RF emulator and improvements of the timers
RF emulator:
Add an RF library emulating a real device without the need of a real
device. This is thought to enable using phy-test without requiring a
radio device or channel simulation. The library synchronizes on the
real time clock, drops TX and generate noise on RX. Noise generation
can be disabled and the noise level configured.
Timers improvements:
- Fix standard deviation for merged timers
- Add display of ULSCH decoding timers
- Add decoding timer in AAL
Taps client for vrtsim
The MR allows vrtsim to connect to the Raytracing Channel Emulator (CE later).
https://gitlab.eurecom.fr/oai/raytracing-channel-emulator
Additional Ubuntu packages required:
- libnanomsg-dev
- flatbuffers-compiler
- libflatbuffers-dev
Everything was tested on the same host. Follow the steps below to
successfully replicate:
1. Build nr-softmodem dependencies
2. Build nr-softmodem for (gNB, nrUE) with OAI_VRTSIM_TAPS_CLIENTset to ON.
3. Start the core network (Optional) follow this tutorial
4. Start the gNB
sudo ./nr-softmodem -O ../../../targets/PROJECTS/GENERIC-NR-5GC/CONF/gnb.sa.band78.fr1.106PRB.usrpb210.conf --gNBs.[0].min_rxtxtime 6 --device.name vrtsim --vrtsim.role server --vrtsim.timescale 0.5 --vrtsim.taps-socket ipc:///tmp/ru_socket_0
5. Start the nrUE
sudo ./nr-uesoftmodem -C 3619200000 -r 106 --numerology 1 --ssb 516 --band 78 --device.name vrtsim --vrtsim.taps-socket ipc:///tmp/ue_socket_0
6. Start the CE Server, once the UE and gNB are running properly, so the
server can connect to IPC sockets
7. Start CE client; the binaries can be downloaded from a pipeline run
or from this link
8. Move the UE in the CE client to generate a new taps array and
populate the taps in softmodems
Move the channel state management outside the channel implementation.
This simplifies the channel implementation but forces users of the
channel to manage it's state manually.
Add handling for error states in vrtsim which allows to close the
softmodems with CTRL+C without hanging, improving user experience
- Client detection of stale timer.
- Abort state for client and server
- Allow client to exit cleanly if server is down
- Ensure timing_thread is joined only once avoiding exit_function loop
- Free noise_device
- Save previously sent samples for channel modelling
The CI fails sometimes, because some processes/threads are created in
unpredictable order and if the client connects before the server is
ready some tests fail, as seen in the following log:
[UTIL] client/server iq-samples: launch sub-processes
[UTIL] threadCreate() for time client: creating thread (no affinity, default priority)
[UTIL] time manager configuration: [time source: server] [mode: client] [server IP: 127.0.0.1} [server port: 7374]
[UTIL] time client: connection to 127.0.0.1:7374 failed, try again in 1s
[UTIL] threadCreate() for time source iq samples: creating thread (no affinity, default priority)
[UTIL] threadCreate() for time server: creating thread (no affinity, default priority)
[UTIL] time manager configuration: [time source: iq_samples] [mode: server] [server IP: 127.0.0.1} [server port: 7374]
[UTIL] client/server: rlc ticks: 1000 pdcp ticks: 1000 x2ap ticks: 1000
CMDLINE: "/oai-ran/build/common/utils/time_manager/tests/test_time_manager_auto" "--time_management.mode" "server" "--time_management.time_source" "iq_samples"
[UTIL] client/server: rlc ticks: 0 pdcp ticks: 0 x2ap ticks: 0
CMDLINE: "/oai-ran/build/common/utils/time_manager/tests/test_time_manager_auto" "--time_management.mode" "client"
[UTIL] client-server iq-samples: ERROR
The client/server iq-samples test is started.
Then we see this log:
[UTIL] time manager configuration: [time source: server] [mode: client] [server IP: 127.0.0.1} [server port: 7374]
Which means that the client is running.
It tries to connect to the server (not running yet), but fails, so sleeps
for 1 second before attempting another connection to the server:
[UTIL] time client: connection to 127.0.0.1:7374 failed, try again in 1s
The sleep is done by the time_client thread, the main thread of the test
program continues and does a sleep(1).
Then the server arrives:
[UTIL] time manager configuration: [time source: iq_samples] [mode: server] [server IP: 127.0.0.1} [server port: 7374]
It sends all the ticks (to no client, since the client is still sleeping).
Then the test is over, the sleep(1) of the client finishes and the program
exits. The client never connected to the server.
We have this log:
[UTIL] client/server: rlc ticks: 0 pdcp ticks: 0 x2ap ticks: 0
And the test fails.
It's complicated to write an automatic test involving several uncoordinated
programs. The operating system may create processes in any order. We use
some sleeping to wait, but it may be too short or too long, leading to test
failure.
Let's see with more sleep if the CI fails less.
simplify and make more local variables in nr_rx_pdcch() and fix small bugs...
simplify and make more local variables in nr_rx_pdcch() and fix small bugs
(size 9 RE instaed of 12 RE)
T hacks: add new tracer ant0 and a tool to plot ofdm data from usrp or rfsim
(output or input)
The tool ant0 dumps IQ data to a file (or stdout) as float complex or short.
It is needed for a student's project. (Well, we could use record/extract, but
it's simpler.) The tool ofdm-plot is used to plot output or input signal from
gnb in the frequency domain. May be useful to debug, why not.
Fix Dedicated BWPs
- Improve SearchSpace selection
- Improve CORESET selection
- Fix fill 'initialDownlinkBWP->pdsch_Config' when there are
DedicatedBWPs
* This fixes the reported 'RSRP = 0' issue when the UE is in the
InitialBWP and we have DedicatedBWPs configured
- Add a 'common' type SearchSpace in the NR_BWP_DownlinkDedicated IE for
DedicatedBWPs
- Fix hardcoded 'controlResourceSetId' by making it dynamic
- Remove 'searchspaceid' from 'verify_agg_levels' function because it
was used only for LOG
- CI: Add DedicatedBWP to '40 MHz TDD F1+E1 SA' in
'RAN-SA-B200-Module-SABOX-Container'
Synchronized Real-Time Data Recording Application Addition
The main Features of Synchronized Real-Time Data Recording Application:
- The Data Recording Application runs in parallel to the gNB or UE
server or in another server as Data Recording or Data Lake Server
- It makes use of (an extended version) of OAI’s T-tracer framework
- It communicates with base station, and user terminal
- It synchronizes and combines data traces and control information
(meta-data) from Base station and User terminal into consistent SigMF
data sets
Details on implementation addition:
- T_tracer Extension to Support up to 35 Parameters in the T() macro
- Add traces on gNB PHY UL and related requirement: FD PUSH IQ, FD DMRS, and UL Channel estimates wi/wo interpolation, and Rx Payload bits:
1. The required memories have been created and initialized in:
openair1/PHY/NR_TRANSPORT/nr_ulsch_demodulation.c
2. Those memories: memory for DMRS signals, memory for channel
estimates based on DMRS positions, memory to store slot grid with
channel coefficients based on DMRS positions after interpolation,
and memory to store extracted data including PUSCH + DMRS.
3. The input parameters to nr_pusch_channel_estimation() has been
updated to get the pointer to the added memories of
pusch_dmrs_slot_mem and pusch_ch_est_dmrs_pos_slot_mem
(openair1/PHY/NR_ESTIMATION/nr_ul_estimation.h)
4. The DMRS grid of NR slot and the estimated channel coefficients at
DMRS locations are logged to created memories in:
openair1/PHY/NR_ESTIMATION/nr_ul_channel_estimation.c
5. The function inner_rx in
openair1/PHY/NR_TRANSPORT/nr_ulsch_demodulation.c is updated to log
the extracted ULSCH PRBs and the interpolated channel coefficients.
6. The T-macro() has been called in
(openair1/PHY/NR_TRANSPORT/nr_ulsch_demodulation.c) to trace added
traces: T_GNB_PHY_UL_FD_DMRS, T_GNB_PHY_UL_FD_CHAN_EST_DMRS_POS,
T_GNB_PHY_UL_FD_PUSCH_IQ, T_GNB_PHY_UL_FD_CHAN_EST_DMRS_INTERPL
- Add traces on UE PHY UL and related requirement: Tx Scrambled bits and Tx payload bits
1. Update input parameters to nr_ulsch_encoding () in
nr_transport_proto_ue.h to get number_dmrs_symbols to be added to
meta-data.
2. In nr_ulsch_coding.c: Trace UE PHY UL Payload TX bits (T_message:
T_UE_PHY_UL_PAYLOAD_TX_BITS) including related meta-data and UTC
time.
3. In nr_ulsch_ue.c: Trace UE PHY UL Scrambled TX bits (T_message:
T_UE_PHY_UL_SCRAMBLED_TX_BITS) including related meta-data and UTC
time.
4. Fix the input prameters in nr_ulsch_encoding () called in
(openair1/SIMULATION/NR_PHY/ulschsim.c) since a new input parameter
has been added.
- Add Data Collection Services (T-Tracers for gNB and UE)
1. Add _tracer_app_gnb and t_tracer_app_ue apps to common/utils/T/tracer/Makefile
2. Add Data collection Service for gNB: common/utils/T/tracer/t_tracer_app_gnb.c
3. Add Data Collection Service for UE: common/utils/T/tracer/t_tracer_app_ue.c
- Add Shared memory configuration
(common/utils/T/tracer/shared_memory_config.h), since we will have too
much memory consumption on the stack if the T-tracer is not fast
enough to process the traces or if we have too much traces. As a
result, tracers start to capture data, write data in shared memory,
then close the T-Tracer after getting the recording of N slots. The
Data recording App will read data from the shared memory.
- Add Main app of Data Recording Application v1.0 including: Config
files, SigMF Interface, Config Interface, Sync Service, and def
messages: The data recording App includes mainly the following files:
1. The main app: data_recording_app_v1.0.py that has the data control
service, interact with tracers, data sync and conversion service.
2. The configuration file:
common/utils/data_recording/config/config_data_recording.json
3. The wireless dictionary file:
common/utils/data_recording/config/wireless_link_parameter_map.yaml
4. Config Interface
(common/utils/data_recording/lib/config_interface.py) that reads
the configuration for both gNB and UE.
5. Sync interface (common/utils/data_recording/lib/sync_service.py)
that sync captured data for gNB and UE before saving them in SigMF
format
6. The SigMF Interface
(common/utils/data_recording/lib/sigmf_interface.py) to store
recorded data and meta-data in SigMF format
7. A simple script
(common/utils/data_recording/sync_validation_demo.py) to validate
that the recorded bits from gNB and UE are in Sync
- For more info look to doc/data_recording.md
The data recording App includes the following files:
- The main app: data_recording_app_v1.0.py that has the data control service, interact with tracers, and data sync and conversion service
- The configuration file: common/utils/data_recording/config/config_data_recording.json
- The wireless dictionary file: common/utils/data_recording/config/wireless_link_parameter_map.yaml
- The SigMF Interface (common/utils/data_recording/lib/sigmf_interface.py) to store recorded data and meta-data in SigMF format
- A simple script (common/utils/data_recording/sync_validation_demo.py) to validate that the recorded bits from gNB and UE are in Sync
- For more info look to doc/data_recording.md
A squashed commit further updates:
- Enhance Data Recording App structure
- Support the new state machine of T-Tracers
- Add new files for Config Interface, sync Service, def of data recording messages, and common utilties
- Make part of logging on Terminal configurable
- Add _tracer_app_gnb and t_tracer_app_ue apps to common/utils/T/tracer/Makefile
- Add Data collection Service for gNB: common/utils/T/tracer/t_tracer_app_gnb.c
- Add Data Collection Service for UE: common/utils/T/tracer/t_tracer_app_ue.c
- Add Shared memory configuration (common/utils/T/tracer/shared_memory_config.h),
since We will have too much memory consumption on the stack if the
T-tracer is not fast enough to process the traces or if we have too
much traces. As a result, tracers start to capture data, write data in
shared memory, then close the T-Tracer after getting the recording of
N slots
- Support the operation of : COnfig, Record, and Stop
- Poll from Socket and not use get_events to mitigate long-time waiting
if there is no data in buffer or the UE is already released or crashed
- Add extra offset to improve data sync
Measurement gaps
Scheduling measurement gaps to provide sufficient time for the UE to
re-tune its transceiver to the target carrier, complete the set of
measurements and then re-tune its transceiver back to the original
carrier.
The seq_arr_erase_it function was modifying start_it during the free_func loop,
which led to incorrectly apply the free_func on the whole array.
This commit makes sure the elements "freed" are only those in the range and not the whole seq_arr.
* Fixes the logic by introducing a temporary iterator for the free loop,
preserving start_it and end_it for correct offset calculations.
* Adds a regression test in test_seq_arr to verify that only the selected
subrange is erased and that free_func is called exactly on that range.
Fix various bugs and inconsistencies in config read, SCTP, ITTI, GTP
Fix bugs across various layers, mostly layer 3, including memory leaks.
See the commits for more details.
The primary motivation here is to remove a VLA, as in the parent
commits. Closer inspection revealed that (1) it is only used by the LTE
UE, and (2) it only logs some data which we likely don't need. So I
simply removed this function.
Move RRC radio parameters file to DU
A number of radio-related RRC parameters, e.g., CellGroupConfig, "originate" at
the DU and are sent to the CU for forwarding to the UE only. Similarly, there
are additional parameters, e.g., RLC-BearerConfig (part of CellGroupConfig),
that originate at the DU.
Hence, the DU decides about these parameters, depending on the MAC state and
capabilities. To clarify this, move the existing
openair2/RRC/NR/nr_rrc_config.{c,h} => openair2/LAYER2/NR_MAC_gNB/nr_radio_config.{c,h}
So that developers know that while those are RRC parameters, they originate at
the DU.
In other words, the CU/RRC is not supposed to use any of these functions in
the RRC. Correspondingly, includes of this file have been cleaned up (and when
they are not necessary at the MAC as well)
Enhance UE identity management in Initial UE Message and other NGAP improvements
This MR improves the Initial UE Message procedures by supporting simultaneous
presence of both 5G-S-TMSI and GUAMI. Previously, only one identity could be
used, leading to potential AMF selection failures. Now the presenceMask allows
both identities. The AMF selection is prioritizing GUAMI first, then falling
back to 5G-S-TMSI, and finally using PLMN or highest capacity.
Also, fix handling of selected PLMN identity in NGAP.
Refactoring:
- Select the AMF based on (1) absence of UE Identity (2) Presence of GUAMI
(3) Presence of 5G-S-TMSI
- Also refactored the AMF selection logic for better readability and reliability.
- Refactor and cleanup rrc_gNB_send_NGAP_NAS_FIRST_REQ
- Replace ngap_pdu_t with byte_array_t
- Add guami_t to ngran_types.h: refactor NGAP and RRC code to use one common
GUAMI type instead of two identical ones
Also:
- Bugfix in ngap_gNB_handle_ng_setup_response: use correct count for PLMNSupportList
Closes issue #801
A number of radio-related RRC parameters, e.g., CellGroupConfig,
"originate" at the DU and are sent to the CU for forwarding to the UE
only. Similarly, there are additional parameters, e.g., RLC-BearerConfig
(part of CellGroupConfig), that originate at the DU.
Hence, the DU decides about these parameters, depending on the MAC state
and capabilities. To clarify this, move the existing
openair2/RRC/NR/nr_rrc_config.{c,h} => openair2/LAYER2/NR_MAC_gNB/nr_radio_config.{c,h}
So that developers know that while those are RRC parameters, they
originate at the DU.
In other words, the CU/RRC is not supposed to use any of these functions
in the RRC. Correspondingly, includes of this file have been cleaned up
(and when they are not necessary at the MAC as well).
When data is generated by "normal" T tracers, it should be fine.
But it might come from unknown sources and then content might be
"strange". So let's validate input data a bit more.
The way allocation is done (granularity of 64KiB) makes a crash more
difficult to achieve, but if a trace is exactly 64KiB, we miss 4 bytes
in the realloc and then anything may happen.
ngap_pdu_t has the same definition as byte_array_t, so it is
replaced by the latter. This simplify the stack and reduces
duplicated code.
Also, add function create_byte_array to copy from an
existing buffer to a new byte_array_t and replaced
allocCopy with create_byte_array, and adopted
OCTET_STRING_fromBuf whenever possible.
Add CI test to force RRC IDLE and new connection setup with 5G-S-TMSI
This MR is a follow up to !2725 (merged)..
The primary goal is to introduce a CI test that enforces the following behavior:
- Forces the UE to enter RRC_IDLE state.
- Validates the UE's re-connection procedure using the 5G-S-TMSI as the UE
identity.
* Close entity->pdusession_sock manually before pthread_join() to unblock sdap_tun_read_thread()
* Update tun_destroy() to open a fresh socket for interface management, independent from the data socket
* Fix check on success of change_interface_state() in tun_destroy(): is a boolean
* Handle ENODEV error when bringing interface down: treat as success if the interface is already gone.
* tun_destroy() is now handling INTERFACE_DOWN, so `INTERFACE_DOWN` in `tun_config()` is no longer needed
In NR with several layers, we may have MAC PDU of size bigger than 64KB,
which does not fit into a UDP buffer, destroying macpdu2wireshark.
Let's add an option to dump to a pcap file instead of using UDP packets.
The DLT 252 magic is used (check wireshark documentation to understand).
Run macpdu2wireshark with -h to see how to configure wireshark to read
the produced files.
Robert Schmidt