Remove --usim-test command line option
Remove the --usim-test option: to my knowledge, it's not used by users,
and is misleadingly placed in various places where there should not be a
difference w.r.t. USIM configuration.
When making the changes, I assumed that --usim-test would be 0 (the
default), which is the case in all executables when not specified in
options except for the nr_dlsim, nr_pbchsmi, nr_ulsim simulators.
The only exception is the initialization of PDCP from the MAC, which
does not seem to be correct (the PDCP is in the CU, so it does not make
sense to call it from the MAC which is in the DU).
Synchronized Real-Time Data Recording Application Addition
The main Features of Synchronized Real-Time Data Recording Application:
- The Data Recording Application runs in parallel to the gNB or UE
server or in another server as Data Recording or Data Lake Server
- It makes use of (an extended version) of OAI’s T-tracer framework
- It communicates with base station, and user terminal
- It synchronizes and combines data traces and control information
(meta-data) from Base station and User terminal into consistent SigMF
data sets
Details on implementation addition:
- T_tracer Extension to Support up to 35 Parameters in the T() macro
- Add traces on gNB PHY UL and related requirement: FD PUSH IQ, FD DMRS, and UL Channel estimates wi/wo interpolation, and Rx Payload bits:
1. The required memories have been created and initialized in:
openair1/PHY/NR_TRANSPORT/nr_ulsch_demodulation.c
2. Those memories: memory for DMRS signals, memory for channel
estimates based on DMRS positions, memory to store slot grid with
channel coefficients based on DMRS positions after interpolation,
and memory to store extracted data including PUSCH + DMRS.
3. The input parameters to nr_pusch_channel_estimation() has been
updated to get the pointer to the added memories of
pusch_dmrs_slot_mem and pusch_ch_est_dmrs_pos_slot_mem
(openair1/PHY/NR_ESTIMATION/nr_ul_estimation.h)
4. The DMRS grid of NR slot and the estimated channel coefficients at
DMRS locations are logged to created memories in:
openair1/PHY/NR_ESTIMATION/nr_ul_channel_estimation.c
5. The function inner_rx in
openair1/PHY/NR_TRANSPORT/nr_ulsch_demodulation.c is updated to log
the extracted ULSCH PRBs and the interpolated channel coefficients.
6. The T-macro() has been called in
(openair1/PHY/NR_TRANSPORT/nr_ulsch_demodulation.c) to trace added
traces: T_GNB_PHY_UL_FD_DMRS, T_GNB_PHY_UL_FD_CHAN_EST_DMRS_POS,
T_GNB_PHY_UL_FD_PUSCH_IQ, T_GNB_PHY_UL_FD_CHAN_EST_DMRS_INTERPL
- Add traces on UE PHY UL and related requirement: Tx Scrambled bits and Tx payload bits
1. Update input parameters to nr_ulsch_encoding () in
nr_transport_proto_ue.h to get number_dmrs_symbols to be added to
meta-data.
2. In nr_ulsch_coding.c: Trace UE PHY UL Payload TX bits (T_message:
T_UE_PHY_UL_PAYLOAD_TX_BITS) including related meta-data and UTC
time.
3. In nr_ulsch_ue.c: Trace UE PHY UL Scrambled TX bits (T_message:
T_UE_PHY_UL_SCRAMBLED_TX_BITS) including related meta-data and UTC
time.
4. Fix the input prameters in nr_ulsch_encoding () called in
(openair1/SIMULATION/NR_PHY/ulschsim.c) since a new input parameter
has been added.
- Add Data Collection Services (T-Tracers for gNB and UE)
1. Add _tracer_app_gnb and t_tracer_app_ue apps to common/utils/T/tracer/Makefile
2. Add Data collection Service for gNB: common/utils/T/tracer/t_tracer_app_gnb.c
3. Add Data Collection Service for UE: common/utils/T/tracer/t_tracer_app_ue.c
- Add Shared memory configuration
(common/utils/T/tracer/shared_memory_config.h), since we will have too
much memory consumption on the stack if the T-tracer is not fast
enough to process the traces or if we have too much traces. As a
result, tracers start to capture data, write data in shared memory,
then close the T-Tracer after getting the recording of N slots. The
Data recording App will read data from the shared memory.
- Add Main app of Data Recording Application v1.0 including: Config
files, SigMF Interface, Config Interface, Sync Service, and def
messages: The data recording App includes mainly the following files:
1. The main app: data_recording_app_v1.0.py that has the data control
service, interact with tracers, data sync and conversion service.
2. The configuration file:
common/utils/data_recording/config/config_data_recording.json
3. The wireless dictionary file:
common/utils/data_recording/config/wireless_link_parameter_map.yaml
4. Config Interface
(common/utils/data_recording/lib/config_interface.py) that reads
the configuration for both gNB and UE.
5. Sync interface (common/utils/data_recording/lib/sync_service.py)
that sync captured data for gNB and UE before saving them in SigMF
format
6. The SigMF Interface
(common/utils/data_recording/lib/sigmf_interface.py) to store
recorded data and meta-data in SigMF format
7. A simple script
(common/utils/data_recording/sync_validation_demo.py) to validate
that the recorded bits from gNB and UE are in Sync
- For more info look to doc/data_recording.md
Remove the --usim-test option: to my knowledge, it's not used by users,
and is misleadingly placed in various places where there should not be a
difference w.r.t. USIM configuration.
When making the changes, I assumed that --usim-test would be 0 (the
default), which is the case in all executables when not specified in
options except for the nr_dlsim, nr_pbchsmi, nr_ulsim simulators.
The only exception is the initialization of PDCP from the MAC, which is
not correct (the PDCP is in the CU, so it does not make sense to call it
from the MAC which is in the DU). Instead, always initialize from main,
even in the NSA case (NSA was covered by the incorrect initialization
from MAC).
This in turn let to some more cleanup around function du_rlc_data_req().
After receiving GTP packets at the DU, they were given to the PDCP to
enqueue the packet at RLC. This does not work anymore, since the PDCP is
not initialized; it also does not seem necessary, as this enqueue
functionality is necessary to decouple PDCP and RLC to avoid deadlocks
in monolithic, but in split-mode, this cannot happen. Instead, call
directly into the RLC when receiving GTP packets.
- Update input parameters to nr_ulsch_encoding () in
nr_transport_proto_ue.h to get number_dmrs_symbols to be added to
meta-data.
- In nr_ulsch_coding.c: Trace UE PHY UL Payload TX bits (T_message:
T_UE_PHY_UL_PAYLOAD_TX_BITS) including related meta-data and UTC time.
- In nr_ulsch_ue.c: Trace UE PHY UL Scrambled TX bits (T_message:
T_UE_PHY_UL_SCRAMBLED_TX_BITS) including related meta-data and UTC
time.
- Fix the input prameters in nr_ulsch_encoding () called in
(openair1/SIMULATION/NR_PHY/ulschsim.c) since a new input parameter
has been added.
Add WLS as a transport option for VNF/PNF, refactor existing options
This MR adds a new transport mechanism for the VNF and PNF split, using
the Intel WLS library, which makes use of DPDK and shared memory to
transport messages. This transport mechanism is what is used to
interface with Radisys ODU, while also working with OAI VNF/PNF.
The VNF and PNF have been refactored so that the transport-specific
functions are isolated from the rest of the logic, and as such,
interchangeable by means of function pointers.
The same change is done for the packing/unpacking procedures, allowing
to switch between FAPI and nFAPI encoding.
This MR also reduces the amount of duplicated code, by refactoring the
VNF with Aerial, which now only has the transport-relevant functions, as
well as the ones that are unique to it ( for example, the unpacking
procedures where a secondary data buffer is used for the message
payloads )
Additionally a new callback is added to the VNF to process
ERROR.indication messages, which currently only indicates which message
originated the error, and which error occurred.
Recently f buffer was changed to hold packed bits and the correspoinding
change in dlsim was not done so the number of error bits before decoding
was incorrectly reported. This commit fixes it.
NR RRC Reject UE
Improvements in procedures to handle RRCReject message both at gNB and
UE. At gNB we send RRC Reject if the UE cannot be given resources to go
into connected mode.
Adds 3 new function pointers to the P5 and P7 PNF structures, allowing to configure which function to use to pack/unpack the messages, as well as unpack the message header, being then used seamlessly regardless of using the FAPI or nFAPI functions.
Changes the header unpack and message unpack functions to return true on success and false on failure, instead of an int
- remove hardcoded values
- adapt gNB RRC code
- adapt gNB MAC code
- adapt physical simulators, where we put the hardcoded values
that were everywhere else before; if there is a need for those
simulators to get configurable values, more work is needed
(but it's doubtful that it is needed)
A number of radio-related RRC parameters, e.g., CellGroupConfig,
"originate" at the DU and are sent to the CU for forwarding to the UE
only. Similarly, there are additional parameters, e.g., RLC-BearerConfig
(part of CellGroupConfig), that originate at the DU.
Hence, the DU decides about these parameters, depending on the MAC state
and capabilities. To clarify this, move the existing
openair2/RRC/NR/nr_rrc_config.{c,h} => openair2/LAYER2/NR_MAC_gNB/nr_radio_config.{c,h}
So that developers know that while those are RRC parameters, they
originate at the DU.
In other words, the CU/RRC is not supposed to use any of these functions
in the RRC. Correspondingly, includes of this file have been cleaned up
(and when they are not necessary at the MAC as well).
Add physim tests into ctest framework
Convert physim testcases listed in
cmake_targets/autotests/test_case_list.xml into ctest. Note: Tests for
dlsim_tm4 are excluded, due to compilation issues.
CTest evaluates success based on return code: '0' = pass, non-zero =
fail. For some simulators, the return code is non-zero even when the
test is successful (as indicated by success messages in the test logs).
For some simulators, the return code is always '0'. This inconsistency
has been address in the first two commits of this MR.
To build and execute these tests:
- using cmake
cd openairinterface5g
mkdir build && cd build # you can also do that in cmake_targets/ran_build/build
cmake .. -GNinja -DENABLE_PHYSIM_TESTS=ON && ninja tests
ctest
- using build_oai: physim-related ctests are activated when PHY
simulators are built (use build_oai with '-P' or '--phy_simulators'
option)
cd openairinterface5g/cmake_targets
./build_oai --ninja --phy_simulators
cd ran_build/build
ctest
Link to newly created 4g and 5g RAN-Physim-Cluster pipelines:
- RAN-Physim-Cluster-4G: https://jenkins-oai.eurecom.fr/job/RAN-Physim-Cluster-4G/
- RAN-Physim-Cluster-5G: https://jenkins-oai.eurecom.fr/job/RAN-Physim-Cluster-5G/
Setting the DEBUG_UE_TIMING flag caused a flood of UE timing logs, resulting
in dlsim output logs as large as 1.5MB per test in CI. This commit comments
out the line where we set the flag to reduce log verbosity.
- Change return value of nr_dlsim, nr_pbchsim and nr_pucchsim to:
- Return '0' if test succeeds (indicated also by "PDSCH/PBCH/PUCCH test
OK" in the test log).
- Return '1' if test fails.
Improvements for LDPC encoding
AVX512 modifications for LDPC encoding: interleaving, ZC384 BG1, output
formatting for encoder, segmentation (memcpy instead of loop). some
improvement in TX for aarch64 in same places where AVX512 support was
added. Also, rate matching and interleaving are done on bytes containing
8 segments and reformatting of the output is done at the end of segment
processing instead of after ldpc encoding.
This improves the overall performance of the NR DL transmitter in gNB
See the merge request description for numbers.
1. nrLDPC_coding_segment_encoder: pointer not aligned on 64 bits
An undefined behavior was happening because of a shift on a __m64 which
was actually a uint32_t which was not aligned on 64 bits.
This commit suggests a fix by properly loading into a __m64 with _mm_set_pi32
before performing the shift.
2. nr_rate_matching: Filler bits detection
Previously, if the bit selection was reaching Ncb and restarting selection from the beginning of encoded data, the filler bits were detected by testing the bytes again NR_NULL which was equal to 2.
But on this branch the bits of up to 8 segments are stored in one byte while doing rate matching so that one byte could be tested to be a filler while it is not.
So this commit changes the way to test filler bits.
Instead of relying on NR_NULL, bit selection now relies on the offset and size of filler bits.
3. nr_ulsch_coding: missing filling f with zeros
The array `f` of the UE UL HARQ process was not filled with zeros between transmissions.
This is a problem if the encoder does not properly overwrite its output.
In order to avoid such situation, this commit adds a memset to fill `f` with zeros before passing it to the encoder.
4. nrLDPC_coding_segment_encoder: missing init of f to zeros
In the encoder, `f` is not filled with zeros before being used.
`f` is instead somehow filled with arbitrary values.
When writing to `output` by 64 bit vectors, some arbitrary bits may be added after the segment.
Then the arbitrary added ones are not overwritten when writing the next segment because of writing with a logical or.
This commit properly fills `f` and `f2` with zeros.
5. nr_ulschsim & nr_dlschsim: Adapt modulation to new encoder output
The QPSK symbols that are feed to the channel model of nr_dlschsim were determined based on the bytes of the encoder output.
With the new output of the encoder, it had to be reworked to determine the QPSK symbols based on the bits of the encoder output.
1. 3GPP Rate-matching/interleaving on 8-segment blocks, deinterleaving at output of multi-segment processing.
2. Bit-packing done in nrLDPC_coding_segment_encoder.c now instead of nr_scrambling.
Interestingly, it improves performance significantly too.
Tested for AVX512 only for now.
- rewrite add_UE_to_list() and rewrite remove_UE_from_list() to simplify
list handling
- return pointer from get_new_nr_ue_inst(); it's completely freed by
delete_nr_ue_data()
- have add_new_UE_RA() add UE context to access list; remove with
nr_release_ra_UE()
- have add_connected_nr_ue() only add UE to connected list; to move from
access to connected use transition_ra_connected_nr_ue(); remove with
mac_remove_nr_ue() as previously
- free memory on problems during RA
- add some documentation
- in NSA/HO: check for pre-configured RA process
- there are asserts in all functions to clarify when a UE has RA
NR UE: add support for continuous frequency offset compensation (in particular NTN LEO Doppler)
- move function add_noise() to file
openair1/SIMULATION/TOOLS/phase_noise.c
- refactor (co)sinus calculation out of phase_noise() into get_sin_cos()
in nr_common.c
- NR UE: continuously estimate the frequency offset using PBCH DMRS
- NR UE: add command line parameter --initial-fo to specify known
initial frequency offset
- NR UE: add support for continuous DL frequency offset compensation
--cont-fo-comp
- NR UE: add support for continuous UL frequency offset
pre-compensation. Depending on the main source of frequency offset,
pre-compensate the UL frequency offset (FO) accordingly. Therefore
modify the flag --cont-fo-comp to accept an integer value, where
* 1 specifies, that the main FO contribution comes from the accuracy of the local oscillator (LO)
* 2 specifies, that the main FO contribution comes from Doppler shift
- NR UE: add PI controller to continuous frequency offset compensation
- rfsimulator: enable and correct NTN LEO doppler simulation
- use SIMD to speed up nr_fo_compensation()
- Extend NTN LEO example in RUNMODEM.md to include continuous Doppler
compensation
- NR UE: in nr_adjust_synch_ue() extend the peak search to +/- cyclic
prefix size, esp. for LEO NTN scenarios the timing drifts during
initial sync, which is mostly compensated with
--ntn-initial-time-drift. But there might be a residual time offset,
and by extending the peak search window we make this more tolerant.
- NR UE: remove unused nr_slot_tx_offset, it is unused since commit
f3e60a14
- NR UE: remove now unnecessary local variable
update_ntn_system_information, directly use
UE->ntn_config_message->update instead
nr_dlsim allocates memory on every SNR iteration without freeing it.
This can lead to memory exhaustion. Fix this by allocating only once
and reuse memory. Also, free it before exit.
Miscelaneous improvements in PHY simulators
1: Add a SIGINT handler to NR PHY simulators
When using T2 virtual functions, it is important to properly stop DPDK
and free the device. Otherwise the virtual functions may be blocked and
a restart of the admin application is necessary.
If not carefully done, such kind of operation can lead to losing cores
to DPDK processes that cannot be stopped. This can lead to situations
where the machine is locked and can only be unlocked by a power cycle.
Always properly stopping DPDK and freeing the device reduces the risk of
such situation to happen.
Up to now, SIGINT was shutting down the PHY simulators without freeing
the device. This commit adds a signal handler to handle SIGINT in a way
that allow to properly free the device.
This feature is added to all the NR PHY simulators whether they use T2
or not in case it is now or later of any use.
2: Make nr_ulschsim functional
There were two issues that were making nr_ulschsim non functional:
1. The channel output was not copied to decoder input (llr array)
2. The test on decoding successful outcome was wrong
The result was that nr_ulschsim was succesfull whatever were its
arguments. This changeset fixes the two issues so that nr_ulschsim is
now functional.
Robert Schmidt