T-Tracer & Data Recording v1.1: UL PHY trace modularization and recording
enhancements
Based on the plan with @schmidtr and @roux-cedric, we agreed to enhance
the data recording in two phases:
- Phase 1: Update Data Recording framework to get All internal Massive
Updates and enhancements through the whole framework public
- Phase2: Restructure T-Tracer Apps and remove all Mems in gNB and UE
Softmodems to Get data on T-tarcer Apps for all activated Messages
symbol-by-symbol, and related Meta-data, then do the aggregation on
the T-tracer App. So, there is No Memory Creation at all on the 5G NR
Stack.
This Merge Request is for Phase 1. We address the comments given in: !3632
Summary: Refactor T-Tracer instrumentation for UL PHY data capture and
improve the data recording application (v1.0 → v1.1). This MR extracts
inline T() macro calls into modular wrapper functions, adds Unix timestamp
fields to trace messages, implements a 4-state shared memory protocol for
gNB/UE tracers, and delivers an improved synchronized recording architecture
in Python.
Changes:
- T-Tracer: Modular UL PHY trace wrappers
- Add T_messages_creator.c/.h with 7 wrapper functions (log_ul_fd_dmrs,
log_ul_fd_chan_est_dmrs_pos, log_ul_fd_pusch_iq,
log_ul_fd_chan_est_dmrs_interpl, log_ul_payload_rx_bits,
log_ul_payload_tx_bits, log_ul_scrambled_tx_bits) and a static
log_ul_common() helper
- Register T_messages_creator.c in the T library CMake build
- Update T_messages.txt definitions: Remove string timestamps and get
time stamp directly from sending_time of T-macro.
- PHY: Replace inline T() calls with wrappers
- gNB (nr_ulsch_demodulation.c, nr_ul_channel_estimation.c,
phy_procedures_nr_gNB.c): replace inline T() calls with
T_messages_creator wrappers
- UE (nr_ulsch_coding.c, nr_ulsch_ue.c): replace inline T() calls with
log_ul_payload_tx_bits and log_ul_scrambled_tx_bits wrappers
- Remove deprecated log_tools.c/.h and its CMake reference
- T-Tracer gNB & UE apps: Enhancements
- Implement 4-state shared memory protocol (WAIT → CONFIG → RECORD → STOP)
for coordination with the data recording app
- Add event_trace_msg_ul_data struct with 27 UL metadata fields
- Refactor event loop to poll()-based with proper signal handling
- Separate project IDs (gNB=2335, UE=2336) and ftok paths for independent
shared memory segments
- Update shared memory sizing: Limit number of records to
NUM_MESSAGES_PER_SLOT=5, 100 slots
- Data Recording App v1.1
- Improved architecture with thread pool and barrier synchronization
- Frame/slot-based message grouping with per-slot validation
- atexit/signal-based cleanup for graceful shared memory detach
- Delete legacy data_recording_app_v1.0.py
- Library improvements
- config_interface.py: remove legacy YAML config reader, streamline to
JSON-only
- shared_memory_interface.py: new module for shared memory
attach/read/protocol
- wireless_parameters_mapper.py: new module for NR parameter mapping
- data_recording_messages_def.py: updated message definitions for
27-field struct
- sigmf_interface.py, sync_service.py, common_utils.py: remove unused
imports and dead code
- Documentation
- Update data_recording.md for v1.1 architecture and usage
- refactor timestamp handling
- Replace string-based SENDING_TIME with integer SEC/NSEC fields.
- Designed the shared memory protocol with a 4-state machine (STATE_WAIT=0, STATE_CONFIG=1, STATE_RECORD=2, STATE_STOP=3) for command/control between the recording app and the T-Tracer service
- Implemented two shared memory segments: one for reading commands (addr_rd) and one for writing captured data (addr_wr)
- Created event_trace_msg_ul_data struct mapping all 27 UL metadata fields (frame, slot, datetime, MCS, DMRS parameters, etc.) by name from the T-Tracer database
- Implemented setup_trace_msg_ul_data() using the G() macro to map field names to indices at startup
- Wrote the main event loop using poll() to avoid busy-waiting, with get_event() to receive T-Tracer events from the softmodem socket
ctest: fix tests when compiling with sanitizers, ignore build directories,
fix spsc_q build
- Fix the directory to the current build directory to enable running with
sanitizers with ctest
- add build* to .gitignore to close#1074
- fix spsc_q compilation on non-C++23 compilers to close#1073
stdatomic.h is a C++23 feature [1]. Thus, "older" compilers such as
gcc-11 (default in Ubuntu 22) do not know this, and can therefore not
correctly compile spsc_q. Fix this by alternatively including atomic for
C++.
Fixes errors such as
stdatomic.h:40:9: error: ‘_Atomic’ does not name a type
[1] https://en.cppreference.com/cpp/header/stdatomic.h
Signed-off-by: Robert Schmidt <robert.schmidt@openairinterface.org>
L1 gNB type0 PDSCH
PDSCH type0 allows frequency allocation via PRB bitmap (type1 is via
start and number of PRBs) allowing for non contiguous allocation in
frequency domain
L1 RX: use queues instead of arrays and linear search for PUCCH, PUSCH, SRS, PRACH
This MR is an attempt to reduce the time L1 RX searches in array for the
next job to process, and instead use a queue. This queue is a FIFO,
because the various jobs (FAPI messages) to process come in order, and
need to be processed in order. The MR (hopefully, to be measured)
reduces the amount of time spent searching for the next UE (because the
next job is always at the beginning of the queue), and should scale
better for many UEs.
It does the following:
- introduce two helper libraries for (1) Frame.Slot calculation (sfn_t),
already introduced in !3521 (merged) commit 3102068e, and (2) a ring
buffer with fixed size
- use sfn_t and ring buffer (gNB->pucch_queue) to remove the linear
array for PUCCH (gNB->pucch)
- use sfn_t and ring buffer (gNB->pusch_queue) for some PUSCH lookups.
Because we need to still store PUSCH contexts, gNB->pusch is still
there
- use sfn_t and ring buffer (gNB->srs_queue) to remove the linear array
for SRS (gNB->srs)
- use sfn_t and ring buffers (gNB->prach_ru_queue and
gNB->prach_l1rx_queue) to remove the linear array for PRACH
(gNB->prach_list)
- some minor cleanups, e.g., additional loops over the PUSCH array,
using const, using pointers instead of indices, etc
Miscellaneous code improvements
- A fix in a yaml config file as reported by @Abdo-Gaber
- proper positioning of static functions in a couple of gNB scheduler
files
- some effort to split NR from LTE code in compilation
- harmonization of macros for unused variables
Add SIB 3,4 support, configurable SIB2 and refactor CU/DU SIB management
This MR makes neighbour and inter-frequency configuration drive how
SIB2/SIB3/SIB4 are built and sent from CU to DUs. It standardizes SIB
payloads as byte_array_t with typed SIB IDs across RRC, F1AP and MAC,
reducing ad‑hoc buffer handling. Neighbour parsing, validation and
lookup are tightened.
Changes
- Minor refactor to gNB neighbour parsing and storage (shared PLMN
extraction, safer allocation, etc).
- Represent SIB containers uniformly as byte_array_t plus nr_sib_type_t,
and adapt F1AP, MAC and RRC users to the new container API.
- Make SIB2 cell-reselection information fully config-driven with
explicit bounds checking and SIB2 ASN.1 building from that config.
- Generate SIB3 intra-frequency neighbours from the per-cell neighbour
list and propagate them from CU to DU over F1, with MAC
decoding/attaching them to SystemInformation.
- Generate SIB4 inter-frequency neighbours from a new frequency_list
plus neighbour SIB3/SIB4 offsets, and propagate them from CU to DU
over F1, with MAC decoding/attaching them.
- Add basic ASN.1 round-trip tests for SIB2/SIB3/SIB4 and SIB4 range
checks, and update RRC docs to describe the neighbour/inter-frequency
configuration model.
Testing:
1. in gNB conf file:
cu_sibs = (2, 3, 4);
2. Update neighbour config file with SIB3/SIB4 conf:
(see documentation)
3. run gNB and UE as usual
---
Logs & configs: see MR on Gitlab
Because it is SPSC, atomic variables are enough to synchronize the two
threads. Concretely put() is modified to ensure that read_idx is
"acquired" so that the read index has been written, including memory, by
the other thread. "Release" ensures that the write operation (including
the memory to the buffer) is written before it is visible to the other
thread in get() (which in turn "acquires" it). For more information, see
also [1].
The rest of the library has been simplified to work only with put() and
get(), reducing total code and the surface for possible bugs.
This (and the previous) version has been tested with the
threadSanitizer:
TSAN_OPTIONS=halt_on_error=1 ./common/utils/ds/tests/test_spsc_q_perf
On my machine, using Google Benchmark, I measure a considerable 5x speed
improvement:
$ /tmp/benchmark/tools/compare.py benchmarks pthread.json atomic.json
Comparing pthread.json to atomic.json
Benchmark Time CPU Time Old Time New CPU Old CPU New
--------------------------------------------------------------------------------------------------------------
BM_spsc_q/10 -0.8201 -0.0740 266779912 47989020 52387 48512
BM_spsc_q/16 -0.8301 -0.0520 249656592 42428540 51462 48784
BM_spsc_q/32 -0.8003 -0.0841 230248798 45972155 53841 49311
BM_spsc_q/64 -0.7995 -0.0506 210429791 42199674 50690 48124
BM_spsc_q/128 -0.7930 -0.1101 205212273 42483155 52745 46936
BM_spsc_q/160 -0.7880 -0.1663 216644738 45938247 53400 44518
OVERALL_GEOMEAN -0.8057 -0.0904 0 0 0 0
[1] https://en.cppreference.com/w/c/atomic/memory_order.html
Assisted-By: Claude:claude-sonnet-4-6
Signed-off-by: Robert Schmidt <robert.schmidt@openairinterface.org>
This data structure is designed to support the use cases of the L1,
i.e., provide a queue which can be quickly used to order L1 jobs in FIFO
order with simple iteration of jobs to treat. It is a Single-Consumer
Single-Producer queue library.
Signed-off-by: Robert Schmidt <robert.schmidt@openairinterface.org>
Unify SIB container handling by introducing a typed SIB enum and storing
SIB payloads as byte_array_t across common NR types, F1AP helpers, MAC,
and RRC DU setup code.
Changes:
- Include byte_array support in nr_common.h, add nr_sib_type_t (NR_SIB_1–NR_SIB_21),
and change nr_SIBs_t to hold only nr_sib_type_t SIB_type.
- Switch f1ap_sib_msg_t in f1ap_messages_types.h to use a byte_array_t SI_container
instead of raw pointer and length fields.
- Update F1AP encode/decode, equality, copy, and free helpers in
f1ap_interface_management.c to work on SI_container.buf / SI_container.len and
use eq_byte_array, copy_byte_array, and free_byte_array.
- Adapt F1AP tests in f1ap_lib_test.c to build and inspect SIB containers via
SI_container.buf and SI_container.len.
- Replace magic SIB numbers with nr_sib_type_t values in gnb_config.c
(get_sys_info and fill_du_sibs) to validate and configure DU SIBs.
- Rework nr_mac_configure_other_sib in NR_MAC_gNB/config.c to decode SIB2
from a byte_array_t container, fix freeing by releasing the decoded sib2 on
failure, and use NR_SIB_2 / NR_SIB_19 for CU/DU SIB selection.
- Add an add_si_msg helper in rrc_gNB_du.c and refactor the SIB2 branch of
rrc_gNB_process_f1_setup_req to populate cell->SI_msg from encoded local
byte array while iterating SIBs with FOR_EACH_SEQ_ARR.
Further unused arguments cleanup
More fixes for unused function arguments (see #1057),
add_compile_options(-Wunused-parameter) added in folders:
- nfapi
- openair2
- openair3
- USRP
- rfsimulator
Cleanup common_lib.h
This change reduces the number of dependecies of common_lib.h and simplifies
the task of implementing an external OAI radio library.
Remove a lot of unused code, defines, and config options
Remove unused header files and defines.
Slightly clean up gnb_config.c by removing all _IDX variables. Remove
these unused parameters:
- MACRLCs.[0].num_cc
- MACRLCs.[0].local_n_portc
- MACRLCs.[0].remove_n_portc
- MACRLCs.[0].remote_s_portc
- MACRLCs.[0].remote_s_portd
- L1s.[0].num_cc
- L1s.[0].local_n_portc
More compilation improvements
Mostly to decouple NR files from LTE headers.
At least in one instance, ulsch_input_buffer_array, using an LTE constant in
NR code was quite dangerous because the buffer in NR could easily be larger
than what the LTE constant foresee.
Extend UICC configuration parsing to provide: routing_indicator,
protection_scheme, home_network_public_key, and home_network_public_key_id.
Use the protection_scheme value to decide what SUCI Profile Scheme to
apply during SUCI generation. Add support for Profile Scheme A which
provides ECIES-based encryption using Curve25519 and X9.63 KDF as outlined
in TS 33.501 Section C.3.4.1 Profile A.
When a configuration file specifies an unsupported Profile Scheme,
the NAS layer triggers a fatal error. This occurs either because
Profile Scheme B is unimplemented or the build uses OpenSSL < 3.0,
which lacks Curve25519 and X9.63 KDF support, ensuring users are
informed of the incompatibility.
Integrate channel pipeline library with vrtsim with two acceleration
options chosen at compile time:
- threadpool if CUDA support is disabled
- CUDA otherwise
- all RAN code, CI code, configuration files, dockerfiles, in CSSL v1.0
- all deployment code (openshift, charts, ancillary files like shell
scripts), in MIT
- documentation in CC-BY-4.0
- exceptions might apply and are listed in NOTICE
- there is a new LICENSES folder with all licenses
- CONTRIBUTIONS.md has been updated accordingly
For automated changes based on OAI PL v1.1:
perl -i~ -0pe 's/\/\*.*Licensed to the OpenAirInterface.*openairinterface.org\n#?/\/*\n * SPDX-License-Identifier: LicenseRef-CSSL-1.0\n/s' **/*.{c,h,cpp}
perl -i~ -0pe 's/\/\*.*Licensed to the OpenAirInterface.*openairinterface.org\n#?/\/*\n * SPDX-License-Identifier: LicenseRef-CSSL-1.0\n/s' **/*.ts
perl -i~ -0pe 's/<!--.*Licensed to the OpenAirInterface.*openairinterface.org\n.*-->/<!-- SPDX-License-Identifier: LicenseRef-CSSL-1.0 -->/s' **/*.xml
The rest (cmake, files with missing license, cmake) manually.
Remove the \file directive, as it is always superfluous because the
current file is implicit [1]:
> If the file name is omitted (i.e. the line after \file is left blank)
> then the documentation block that contains the \file command will belong
> to the file it is located in.
Author names and e-mails are not relevant for us: it can always be
inferred from git blame, and is often outdated.
Eurecom code has been contributed and was under OAI PL v1.0/v1.1.
For the cpack package contact: put generic email address that is
independent of an individual and that will remain reachable.
[1] https://www.doxygen.nl/manual/commands.html#cmdfile
Fix Delta MCS mode, reimplement UL power control
This change set fixes the "delta-MCS" mode, and reimplements the UL power
control loops at gNB, i.e., for PUSCH and PUCCH. It further adds new T traces
to log PUSCH/PUCCH power infrmation, and cleans up code.
The "delta-MCS" mode in OAI refers to configuring a UE with deltaMCS in the
PUSCH-PowerControl IE (see TS 38.331). In this mode, the UE adjusts the PUSCH
power for transmissions to take into account for MCS (see TS 38.213 Sec. 7.1).
In other words, the target SNR can be set to a somewhat low value, and the UE
will still use the right power for UL transmissions. Note that the spec only
foresees this adjustment of UL power when using one layer. In contrast, in the
"normal" mode, the gNB sends TPC commands to keep a UE at a fixed target SNR
(default: 20dB), regardless of the MCS or number of layers. Future work is
planned to completely remove this fixed target SNR, which is not done in this
MR yet.
Further, there is a new power control loop implementation. Prior to this MR, on
each UL transmissions, the gNB uses the SNR to decide about TPC sent to a UE in
the next transmissions. The new implementation relies on averaging the SNR, and
a "TPC in flight" average to account for the slow reaction of the average SNR.
The sum of both quantities is the current SNR that is used by the UE. See the
commit messages or documentation for more information.
Further cleanup to group data (e.g., power control configuration, refactoring of
code to save space, ...) is done. Two new T traces GNB_MAC_PUSCH_POWER_CONTROL
and GNB_MAC_PUCCH_POWER_CONTROL are added; documentation exists to explain how
to graphically plot corresponding graphs.
Remove OSD, some scripts, oaisim references
The README says:
It is just generating the XML file and it is placing it in
$(OPENAIR_TARGETS)/SIMU/EXAMPLE/OSD/WEBXML/ or http://localhost/xmlfile/
so that the oaisim could pars and run the simulation/emulation
oaisim does not exist anymore, and corresponding simulation cannot be done.
Also, it requires XAMPP and similar technologies, that I think nobody uses
in the context of OAI.
Remove also most references to oaisim.
Remove some scripts that are likely not used by anybody, or straight useless.
Jaroslava Fiedlerova