Merge remote-tracking branch 'origin/data_recording_app_optimization_part1' into integration_2026_w17 (!4061)

T-Tracer & Data Recording v1.1: UL PHY trace modularization and recording
enhancements

Based on the plan with @schmidtr  and @roux-cedric, we agreed to enhance
the data recording in two phases:

- Phase 1: Update Data Recording framework to get All internal Massive
  Updates and enhancements through the whole framework public
- Phase2: Restructure T-Tracer Apps and remove all Mems in gNB and UE
  Softmodems to Get data on T-tarcer Apps for all activated Messages
  symbol-by-symbol, and related Meta-data, then do the aggregation on
  the T-tracer App. So, there is No Memory Creation at all on the 5G NR
  Stack.

This Merge Request is for Phase 1. We address the comments given in: !3632

Summary: Refactor T-Tracer instrumentation for UL PHY data capture and
improve the data recording application (v1.0 → v1.1). This MR extracts
inline T() macro calls into modular wrapper functions, adds Unix timestamp
fields to trace messages, implements a 4-state shared memory protocol for
gNB/UE tracers, and delivers an improved synchronized recording architecture
in Python.

Changes:
- T-Tracer: Modular UL PHY trace wrappers
  - Add T_messages_creator.c/.h with 7 wrapper functions (log_ul_fd_dmrs,
    log_ul_fd_chan_est_dmrs_pos, log_ul_fd_pusch_iq,
    log_ul_fd_chan_est_dmrs_interpl, log_ul_payload_rx_bits,
    log_ul_payload_tx_bits, log_ul_scrambled_tx_bits) and a static
    log_ul_common() helper
  - Register T_messages_creator.c in the T library CMake build
  - Update T_messages.txt definitions: Remove string timestamps and get
    time stamp directly from sending_time of T-macro.
  - PHY: Replace inline T() calls with wrappers
  - gNB (nr_ulsch_demodulation.c, nr_ul_channel_estimation.c,
    phy_procedures_nr_gNB.c): replace inline T() calls with
    T_messages_creator wrappers
  - UE (nr_ulsch_coding.c, nr_ulsch_ue.c): replace inline T() calls with
    log_ul_payload_tx_bits and log_ul_scrambled_tx_bits wrappers
  - Remove deprecated log_tools.c/.h and its CMake reference

- T-Tracer gNB & UE apps: Enhancements
  - Implement 4-state shared memory protocol (WAIT → CONFIG → RECORD → STOP)
    for coordination with the data recording app
  - Add event_trace_msg_ul_data struct with 27 UL metadata fields
  - Refactor event loop to poll()-based with proper signal handling
  - Separate project IDs (gNB=2335, UE=2336) and ftok paths for independent
    shared memory segments
  - Update shared memory sizing: Limit number of records to
    NUM_MESSAGES_PER_SLOT=5, 100 slots

- Data Recording App v1.1
  - Improved architecture with thread pool and barrier synchronization
  - Frame/slot-based message grouping with per-slot validation
  - atexit/signal-based cleanup for graceful shared memory detach
  - Delete legacy data_recording_app_v1.0.py

- Library improvements
  - config_interface.py: remove legacy YAML config reader, streamline to
    JSON-only
  - shared_memory_interface.py: new module for shared memory
    attach/read/protocol
  - wireless_parameters_mapper.py: new module for NR parameter mapping
  - data_recording_messages_def.py: updated message definitions for
    27-field struct
  - sigmf_interface.py, sync_service.py, common_utils.py: remove unused
    imports and dead code

- Documentation
  - Update data_recording.md for v1.1 architecture and usage

common/utils/T/CMakeLists.txt

@@ -55,7 +55,7 @@ add_dependencies(T_headers T_IDs.h generate_T)
 target_include_directories(T_headers INTERFACE ${CMAKE_CURRENT_BINARY_DIR} ${CMAKE_CURRENT_SOURCE_DIR})
 
 if(T_TRACER)
-  add_library(T STATIC T.c local_tracer.c T_IDs.h T_messages.txt.h)
+  add_library(T STATIC T.c local_tracer.c T_messages_creator.c T_IDs.h T_messages.txt.h)
   target_link_libraries(T PUBLIC rt)
   target_link_libraries(T PRIVATE CONFIG_LIB T_headers)
 

common/utils/T/T.h

@@ -115,7 +115,7 @@ extern int T_stdout;
 
 /* type used to send arbitrary buffer data */
 typedef struct {
-  void *addr;
+  const void *addr;
   int length;
 } T_buffer;
 

common/utils/T/T_messages.txt

@@ -118,23 +118,23 @@ ID = GNB_PHY_PUCCH_PUSCH_IQ
 ID = GNB_PHY_UL_FD_PUSCH_IQ
     DESC = gNodeB UL PUSCH IQ Data in the frequency domain
     GROUP = ALL:PHY:GRAPHIC:HEAVY:GNB
-    FORMAT = int,frame : int,slot : int,datetime_yyyymmdd : int,datetime_hhmmssmmm : int,frame_type : int,freq_range : int,subcarrier_spacing : int,cyclic_prefix : int,symbols_per_slot : int,Nid_cell : int,rnti : int,rb_size : int,rb_start : int,start_symbol_index : int,nr_of_symbols : int,qam_mod_order : int,mcs_index : int,mcs_table : int,nrOfLayers : int,transform_precoding : int,dmrs_config_type : int,ul_dmrs_symb_pos : int,number_dmrs_symbols : int,dmrs_port : int,dmrs_nscid : int,nb_antennas_rx : int,number_of_bits : buffer,data
+    FORMAT = int,frame : int,slot : int,frame_type : int,freq_range : int,subcarrier_spacing : int,cyclic_prefix : int,symbols_per_slot : int,Nid_cell : int,rnti : int,rb_size : int,rb_start : int,start_symbol_index : int,nr_of_symbols : int,qam_mod_order : int,mcs_index : int,mcs_table : int,nrOfLayers : int,transform_precoding : int,dmrs_config_type : int,ul_dmrs_symb_pos : int,number_dmrs_symbols : int,dmrs_port : int,dmrs_nscid : int,nb_antennas_rx : int,number_of_bits : buffer,data
 ID = GNB_PHY_UL_FD_DMRS
     DESC = gNodeB UL DMRS Signal in the frequency domain
     GROUP = ALL:PHY:GRAPHIC:HEAVY:GNB
-    FORMAT = int,frame : int,slot : int,datetime_yyyymmdd : int,datetime_hhmmssmmm : int,frame_type : int,freq_range : int,subcarrier_spacing : int,cyclic_prefix : int,symbols_per_slot : int,Nid_cell : int,rnti : int,rb_size : int,rb_start : int,start_symbol_index : int,nr_of_symbols : int,qam_mod_order : int,mcs_index : int,mcs_table : int,nrOfLayers : int,transform_precoding : int,dmrs_config_type : int,ul_dmrs_symb_pos : int,number_dmrs_symbols : int,dmrs_port : int,dmrs_nscid : int,nb_antennas_rx : int,number_of_bits : buffer,data
+    FORMAT = int,frame : int,slot : int,frame_type : int,freq_range : int,subcarrier_spacing : int,cyclic_prefix : int,symbols_per_slot : int,Nid_cell : int,rnti : int,rb_size : int,rb_start : int,start_symbol_index : int,nr_of_symbols : int,qam_mod_order : int,mcs_index : int,mcs_table : int,nrOfLayers : int,transform_precoding : int,dmrs_config_type : int,ul_dmrs_symb_pos : int,number_dmrs_symbols : int,dmrs_port : int,dmrs_nscid : int,nb_antennas_rx : int,number_of_bits : buffer,data
 ID = GNB_PHY_UL_FD_CHAN_EST_DMRS_POS
     DESC = gNodeB channel estimation in the frequency domain based on DMRS and at DMRS pilots location without any further interploation.
     GROUP = ALL:PHY:GRAPHIC:HEAVY:GNB
-    FORMAT = int,frame : int,slot : int,datetime_yyyymmdd : int,datetime_hhmmssmmm : int,frame_type : int,freq_range : int,subcarrier_spacing : int,cyclic_prefix : int,symbols_per_slot : int,Nid_cell : int,rnti : int,rb_size : int,rb_start : int,start_symbol_index : int,nr_of_symbols : int,qam_mod_order : int,mcs_index : int,mcs_table : int,nrOfLayers : int,transform_precoding : int,dmrs_config_type : int,ul_dmrs_symb_pos : int,number_dmrs_symbols : int,dmrs_port : int,dmrs_nscid : int,nb_antennas_rx : int,number_of_bits : buffer,data
+    FORMAT = int,frame : int,slot : int,frame_type : int,freq_range : int,subcarrier_spacing : int,cyclic_prefix : int,symbols_per_slot : int,Nid_cell : int,rnti : int,rb_size : int,rb_start : int,start_symbol_index : int,nr_of_symbols : int,qam_mod_order : int,mcs_index : int,mcs_table : int,nrOfLayers : int,transform_precoding : int,dmrs_config_type : int,ul_dmrs_symb_pos : int,number_dmrs_symbols : int,dmrs_port : int,dmrs_nscid : int,nb_antennas_rx : int,number_of_bits : buffer,data
 ID = GNB_PHY_UL_FD_CHAN_EST_DMRS_INTERPL
     DESC = gNodeB channel estimation in the frequency domain based on DMRS and interploation.
     GROUP = ALL:PHY:GRAPHIC:HEAVY:GNB
-    FORMAT = int,frame : int,slot : int,datetime_yyyymmdd : int,datetime_hhmmssmmm : int,frame_type : int,freq_range : int,subcarrier_spacing : int,cyclic_prefix : int,symbols_per_slot : int,Nid_cell : int,rnti : int,rb_size : int,rb_start : int,start_symbol_index : int,nr_of_symbols : int,qam_mod_order : int,mcs_index : int,mcs_table : int,nrOfLayers : int,transform_precoding : int,dmrs_config_type : int,ul_dmrs_symb_pos : int,number_dmrs_symbols : int,dmrs_port : int,dmrs_nscid : int,nb_antennas_rx : int,number_of_bits : buffer,dataI
+    FORMAT = int,frame : int,slot : int,frame_type : int,freq_range : int,subcarrier_spacing : int,cyclic_prefix : int,symbols_per_slot : int,Nid_cell : int,rnti : int,rb_size : int,rb_start : int,start_symbol_index : int,nr_of_symbols : int,qam_mod_order : int,mcs_index : int,mcs_table : int,nrOfLayers : int,transform_precoding : int,dmrs_config_type : int,ul_dmrs_symb_pos : int,number_dmrs_symbols : int,dmrs_port : int,dmrs_nscid : int,nb_antennas_rx : int,number_of_bits : buffer,dataI
 ID = GNB_PHY_UL_PAYLOAD_RX_BITS
     DESC = gNodeB slot payload after decoder
     GROUP = ALL:PHY:GRAPHIC:HEAVY:GNB
-    FORMAT = int,frame : int,slot : int,datetime_yyyymmdd : int,datetime_hhmmssmmm : int,frame_type : int,freq_range : int,subcarrier_spacing : int,cyclic_prefix : int,symbols_per_slot : int,Nid_cell : int,rnti : int,rb_size : int,rb_start : int,start_symbol_index : int,nr_of_symbols : int,qam_mod_order : int,mcs_index : int,mcs_table : int,nrOfLayers : int,transform_precoding : int,dmrs_config_type : int,ul_dmrs_symb_pos : int,number_dmrs_symbols : int,dmrs_port : int,dmrs_nscid : int,nb_antennas_rx : int,number_of_bits : buffer,data
+    FORMAT = int,frame : int,slot : int,frame_type : int,freq_range : int,subcarrier_spacing : int,cyclic_prefix : int,symbols_per_slot : int,Nid_cell : int,rnti : int,rb_size : int,rb_start : int,start_symbol_index : int,nr_of_symbols : int,qam_mod_order : int,mcs_index : int,mcs_table : int,nrOfLayers : int,transform_precoding : int,dmrs_config_type : int,ul_dmrs_symb_pos : int,number_dmrs_symbols : int,dmrs_port : int,dmrs_nscid : int,nb_antennas_rx : int,number_of_bits : buffer,data
 ID = GNB_PHY_UL_FREQ_CHANNEL_ESTIMATE
     DESC = gNodeB channel estimation in the frequency domain
     GROUP = ALL:PHY:GRAPHIC:HEAVY:GNB
@@ -1474,11 +1474,11 @@ ID = UE_PHY_MEAS
 ID = UE_PHY_UL_PAYLOAD_TX_BITS
     DESC = UE UL Tx bits in one slot before decoder and scrambler
     GROUP = ALL:PHY:GRAPHIC:HEAVY:UE
-    FORMAT = int,frame : int,slot : int,datetime_yyyymmdd : int,datetime_hhmmssmmm : int,frame_type : int,freq_range : int,subcarrier_spacing : int,cyclic_prefix : int,symbols_per_slot : int,Nid_cell : int,rnti : int,rb_size : int,rb_start : int,start_symbol_index : int,nr_of_symbols : int,qam_mod_order : int,mcs_index : int,mcs_table : int,nrOfLayers : int,transform_precoding : int,dmrs_config_type : int,ul_dmrs_symb_pos : int,number_dmrs_symbols : int,dmrs_port : int,dmrs_nscid : int,nb_antennas_tx : int,number_of_bits : buffer,data
+    FORMAT = int,frame : int,slot : int,frame_type : int,freq_range : int,subcarrier_spacing : int,cyclic_prefix : int,symbols_per_slot : int,Nid_cell : int,rnti : int,rb_size : int,rb_start : int,start_symbol_index : int,nr_of_symbols : int,qam_mod_order : int,mcs_index : int,mcs_table : int,nrOfLayers : int,transform_precoding : int,dmrs_config_type : int,ul_dmrs_symb_pos : int,number_dmrs_symbols : int,dmrs_port : int,dmrs_nscid : int,nb_antennas_tx : int,number_of_bits : buffer,data
 ID = UE_PHY_UL_SCRAMBLED_TX_BITS
     DESC = UE UL Tx bits in one slot after scrambler
     GROUP = ALL:PHY:GRAPHIC:HEAVY:UE
-    FORMAT = int,frame : int,slot : int,datetime_yyyymmdd : int,datetime_hhmmssmmm : int,frame_type : int,freq_range : int,subcarrier_spacing : int,cyclic_prefix : int,symbols_per_slot : int,Nid_cell : int,rnti : int,rb_size : int,rb_start : int,start_symbol_index : int,nr_of_symbols : int,qam_mod_order : int,mcs_index : int,mcs_table : int,nrOfLayers : int,transform_precoding : int,dmrs_config_type : int,ul_dmrs_symb_pos : int,number_dmrs_symbols : int,dmrs_port : int,dmrs_nscid : int,nb_antennas_tx : int,number_of_bits : buffer,data
+    FORMAT = int,frame : int,slot : int,frame_type : int,freq_range : int,subcarrier_spacing : int,cyclic_prefix : int,symbols_per_slot : int,Nid_cell : int,rnti : int,rb_size : int,rb_start : int,start_symbol_index : int,nr_of_symbols : int,qam_mod_order : int,mcs_index : int,mcs_table : int,nrOfLayers : int,transform_precoding : int,dmrs_config_type : int,ul_dmrs_symb_pos : int,number_dmrs_symbols : int,dmrs_port : int,dmrs_nscid : int,nb_antennas_tx : int,number_of_bits : buffer,data
 
 #for debug/test - not used
 ID = first

common/utils/T/T_messages_creator.c

@@ -0,0 +1,220 @@
+/*
+ * SPDX-License-Identifier: LicenseRef-CSSL-1.0
+ */
+
+/*!
+ * \brief Helper functions for creating T-Tracer UL PHY trace messages, it is used by data recording application
+ */
+
+/*
+ * Each function logs one UL PHY trace message via the T() macro.
+ * All messages share the same field format (Data Collection Trace Messages Structure):
+ *   int,frame : int,slot :
+ *   int,frame_type : int,freq_range : int,subcarrier_spacing : int,cyclic_prefix : int,symbols_per_slot :
+ *   int,Nid_cell : int,rnti :
+ *   int,rb_size : int,rb_start : int,start_symbol_index : int,nr_of_symbols :
+ *   int,qam_mod_order : int,mcs_index : int,mcs_table : int,nrOfLayers :
+ *   int,transform_precoding : int,dmrs_config_type : int,ul_dmrs_symb_pos : int,number_dmrs_symbols : int,dmrs_port :
+ *   int,dmrs_nscid : int,nb_antennas_rx : int,number_of_bits : buffer,data
+ */
+
+#include "T.h"
+#include "T_messages_creator.h"
+
+#if T_TRACER
+
+// Internal helper: emit the common UL metadata + buffer via T()
+// Note: The T() macro internally checks T_ACTIVE(trace_id) before sending,
+// so callers do not need an additional T_ACTIVE() guard.
+static inline void log_ul_common(int trace_id,
+                                 int frame,
+                                 int slot,
+                                 const NR_DL_FRAME_PARMS *frame_parms,
+                                 const nfapi_nr_pusch_pdu_t *rel15_ul,
+                                 int number_dmrs_symbols,
+                                 int dmrs_port,
+                                 int number_of_bits,
+                                 const void *data,
+                                 int data_len)
+{
+  T(trace_id,
+    T_INT((int)frame),
+    T_INT((int)slot),
+    T_INT((int)frame_parms->frame_type),
+    T_INT((int)frame_parms->freq_range),
+    T_INT((int)rel15_ul->subcarrier_spacing),
+    T_INT((int)rel15_ul->cyclic_prefix),
+    T_INT((int)frame_parms->symbols_per_slot),
+    T_INT((int)frame_parms->Nid_cell),
+    T_INT((int)rel15_ul->rnti),
+    T_INT((int)rel15_ul->rb_size),
+    T_INT((int)rel15_ul->rb_start),
+    T_INT((int)rel15_ul->start_symbol_index),
+    T_INT((int)rel15_ul->nr_of_symbols),
+    T_INT((int)rel15_ul->qam_mod_order),
+    T_INT((int)rel15_ul->mcs_index),
+    T_INT((int)rel15_ul->mcs_table),
+    T_INT((int)rel15_ul->nrOfLayers),
+    T_INT((int)rel15_ul->transform_precoding),
+    T_INT((int)rel15_ul->dmrs_config_type),
+    T_INT((int)rel15_ul->ul_dmrs_symb_pos),
+    T_INT((int)number_dmrs_symbols),
+    T_INT((int)dmrs_port),
+    T_INT((int)rel15_ul->scid),
+    T_INT((int)frame_parms->nb_antennas_rx),
+    T_INT((int)number_of_bits),
+    T_BUFFER(data, data_len));
+}
+
+void log_ul_fd_dmrs(int frame,
+                    int slot,
+                    const NR_DL_FRAME_PARMS *frame_parms,
+                    const nfapi_nr_pusch_pdu_t *rel15_ul,
+                    int number_dmrs_symbols,
+                    int dmrs_port,
+                    const c16_t *data,
+                    int data_len)
+{
+  log_ul_common(T_GNB_PHY_UL_FD_DMRS,
+                frame, slot, frame_parms, rel15_ul,
+                number_dmrs_symbols, dmrs_port, 0, data, data_len);
+}
+
+void log_ul_fd_chan_est_dmrs_pos(int frame,
+                                int slot,
+                                const NR_DL_FRAME_PARMS *frame_parms,
+                                const nfapi_nr_pusch_pdu_t *rel15_ul,
+                                int number_dmrs_symbols,
+                                int dmrs_port,
+                                const c16_t *data,
+                                int data_len)
+{
+  log_ul_common(T_GNB_PHY_UL_FD_CHAN_EST_DMRS_POS,
+                frame, slot, frame_parms, rel15_ul,
+                number_dmrs_symbols, dmrs_port, 0, data, data_len);
+}
+
+void log_ul_fd_pusch_iq(int frame,
+                        int slot,
+                        const NR_DL_FRAME_PARMS *frame_parms,
+                        const nfapi_nr_pusch_pdu_t *rel15_ul,
+                        int number_dmrs_symbols,
+                        int dmrs_port,
+                        const c16_t *data,
+                        int data_len)
+{
+  log_ul_common(T_GNB_PHY_UL_FD_PUSCH_IQ,
+                frame, slot, frame_parms, rel15_ul,
+                number_dmrs_symbols, dmrs_port, 0, data, data_len);
+}
+
+void log_ul_fd_chan_est_dmrs_interpl(int frame,
+                                    int slot,
+                                    const NR_DL_FRAME_PARMS *frame_parms,
+                                    const nfapi_nr_pusch_pdu_t *rel15_ul,
+                                    int number_dmrs_symbols,
+                                    int dmrs_port,
+                                    const c16_t *data,
+                                    int data_len)
+{
+  log_ul_common(T_GNB_PHY_UL_FD_CHAN_EST_DMRS_INTERPL,
+                frame, slot, frame_parms, rel15_ul,
+                number_dmrs_symbols, dmrs_port, 0, data, data_len);
+}
+
+void log_ul_payload_rx_bits(int frame,
+                           int slot,
+                           const NR_DL_FRAME_PARMS *frame_parms,
+                           const nfapi_nr_pusch_pdu_t *rel15_ul,
+                           int number_dmrs_symbols,
+                           int dmrs_port,
+                           const uint8_t *data,
+                           int tb_size)
+{
+  log_ul_common(T_GNB_PHY_UL_PAYLOAD_RX_BITS,
+                frame, slot, frame_parms, rel15_ul,
+                number_dmrs_symbols, dmrs_port,
+                tb_size << 3, data, tb_size);
+}
+
+// UE-side: uses nfapi_nr_ue_pusch_pdu_t, derives subcarrier_spacing index
+// from frame_parms, and logs nb_antennas_tx instead of nb_antennas_rx.
+void log_ul_payload_tx_bits(int frame,
+                           int slot,
+                           const NR_DL_FRAME_PARMS *frame_parms,
+                           const nfapi_nr_ue_pusch_pdu_t *pusch_pdu,
+                           int number_dmrs_symbols,
+                           int dmrs_port,
+                           const uint8_t *data,
+                           int tb_size)
+{
+  int subcarrier_spacing_index = frame_parms->subcarrier_spacing / 15000 - 1;
+  T(T_UE_PHY_UL_PAYLOAD_TX_BITS,
+    T_INT((int)frame),
+    T_INT((int)slot),
+    T_INT((int)frame_parms->frame_type),
+    T_INT((int)frame_parms->freq_range),
+    T_INT((int)subcarrier_spacing_index),
+    T_INT((int)pusch_pdu->cyclic_prefix),
+    T_INT((int)frame_parms->symbols_per_slot),
+    T_INT((int)frame_parms->Nid_cell),
+    T_INT((int)pusch_pdu->rnti),
+    T_INT((int)pusch_pdu->rb_size),
+    T_INT((int)pusch_pdu->rb_start),
+    T_INT((int)pusch_pdu->start_symbol_index),
+    T_INT((int)pusch_pdu->nr_of_symbols),
+    T_INT((int)pusch_pdu->qam_mod_order),
+    T_INT((int)pusch_pdu->mcs_index),
+    T_INT((int)pusch_pdu->mcs_table),
+    T_INT((int)pusch_pdu->nrOfLayers),
+    T_INT((int)pusch_pdu->transform_precoding),
+    T_INT((int)pusch_pdu->dmrs_config_type),
+    T_INT((int)pusch_pdu->ul_dmrs_symb_pos),
+    T_INT((int)number_dmrs_symbols),
+    T_INT((int)dmrs_port),
+    T_INT((int)pusch_pdu->scid),
+    T_INT((int)frame_parms->nb_antennas_tx),
+    T_INT((int)(tb_size << 3)),
+    T_BUFFER(data, tb_size));
+}
+
+void log_ul_scrambled_tx_bits(int frame,
+                             int slot,
+                             const NR_DL_FRAME_PARMS *frame_parms,
+                             const nfapi_nr_ue_pusch_pdu_t *pusch_pdu,
+                             int number_dmrs_symbols,
+                             int dmrs_port,
+                             const uint8_t *data,
+                             int number_of_bits)
+{
+  int subcarrier_spacing_index = frame_parms->subcarrier_spacing / 15000 - 1;
+  T(T_UE_PHY_UL_SCRAMBLED_TX_BITS,
+    T_INT((int)frame),
+    T_INT((int)slot),
+    T_INT((int)frame_parms->frame_type),
+    T_INT((int)frame_parms->freq_range),
+    T_INT((int)subcarrier_spacing_index),
+    T_INT((int)pusch_pdu->cyclic_prefix),
+    T_INT((int)frame_parms->symbols_per_slot),
+    T_INT((int)frame_parms->Nid_cell),
+    T_INT((int)pusch_pdu->rnti),
+    T_INT((int)pusch_pdu->rb_size),
+    T_INT((int)pusch_pdu->rb_start),
+    T_INT((int)pusch_pdu->start_symbol_index),
+    T_INT((int)pusch_pdu->nr_of_symbols),
+    T_INT((int)pusch_pdu->qam_mod_order),
+    T_INT((int)pusch_pdu->mcs_index),
+    T_INT((int)pusch_pdu->mcs_table),
+    T_INT((int)pusch_pdu->nrOfLayers),
+    T_INT((int)pusch_pdu->transform_precoding),
+    T_INT((int)pusch_pdu->dmrs_config_type),
+    T_INT((int)pusch_pdu->ul_dmrs_symb_pos),
+    T_INT((int)number_dmrs_symbols),
+    T_INT((int)dmrs_port),
+    T_INT((int)pusch_pdu->scid),
+    T_INT((int)frame_parms->nb_antennas_tx),
+    T_INT((int)number_of_bits),
+    T_BUFFER(data, number_of_bits / 8));
+}
+
+#endif /* T_TRACER */

common/utils/T/T_messages_creator.h

@@ -0,0 +1,195 @@
+/*
+ * SPDX-License-Identifier: LicenseRef-CSSL-1.0
+ */
+
+/*!
+ * \brief Helper functions for creating T-Tracer UL PHY trace messages, it is used by data recording application
+ */
+
+#ifndef T_MESSAGES_CREATOR_H
+#define T_MESSAGES_CREATOR_H
+
+#if T_TRACER
+
+#include <stdint.h>
+#include "common/platform_types.h"
+#include "common/utils/nr/nr_common.h"
+#include "PHY/defs_nr_common.h"
+#include "nfapi_nr_interface_scf.h"
+#include "fapi_nr_ue_interface.h"
+
+/**
+ * @brief Log PUSCH UL DMRS using T-Tracer
+ *
+ * This function creates and sends a T_GNB_PHY_UL_FD_DMRS message containing
+ * PUSCH DMRS symbols in frequency domain.
+ *
+ * @param frame Frame number
+ * @param slot Slot number
+ * @param frame_parms Pointer to frame parameters structure
+ * @param rel15_ul Pointer to PUSCH PDU structure
+ * @param number_dmrs_symbols Number of DMRS symbols
+ * @param dmrs_port DMRS port number
+ * @param data Pointer to DMRS data buffer (c16_t complex samples)
+ * @param data_len Size of the data buffer in bytes
+ */
+void log_ul_fd_dmrs(int frame,
+                    int slot,
+                    const NR_DL_FRAME_PARMS *frame_parms,
+                    const nfapi_nr_pusch_pdu_t *rel15_ul,
+                    int number_dmrs_symbols,
+                    int dmrs_port,
+                    const c16_t *data,
+                    int data_len);
+
+/**
+ * @brief Log PUSCH UL channel estimates at DMRS positions using T-Tracer
+ *
+ * This function creates and sends a T_GNB_PHY_UL_FD_CHAN_EST_DMRS_POS message
+ * containing channel estimates at DMRS positions in frequency domain.
+ *
+ * @param frame Frame number
+ * @param slot Slot number
+ * @param frame_parms Pointer to frame parameters structure
+ * @param rel15_ul Pointer to PUSCH PDU structure
+ * @param number_dmrs_symbols Number of DMRS symbols
+ * @param dmrs_port DMRS port number
+ * @param data Pointer to channel estimate data buffer (c16_t complex samples)
+ * @param data_len Size of the data buffer in bytes
+ */
+void log_ul_fd_chan_est_dmrs_pos(int frame,
+                                int slot,
+                                const NR_DL_FRAME_PARMS *frame_parms,
+                                const nfapi_nr_pusch_pdu_t *rel15_ul,
+                                int number_dmrs_symbols,
+                                int dmrs_port,
+                                const c16_t *data,
+                                int data_len);
+
+/**
+ * @brief Log PUSCH UL IQ data using T-Tracer
+ *
+ * This function creates and sends a T_GNB_PHY_UL_FD_PUSCH_IQ message
+ * containing PUSCH IQ samples in frequency domain.
+ *
+ * @param frame Frame number
+ * @param slot Slot number
+ * @param frame_parms Pointer to frame parameters structure
+ * @param rel15_ul Pointer to PUSCH PDU structure
+ * @param number_dmrs_symbols Number of DMRS symbols
+ * @param dmrs_port DMRS port number
+ * @param data Pointer to PUSCH IQ data buffer (c16_t complex samples)
+ * @param data_len Size of the data buffer in bytes
+ */
+void log_ul_fd_pusch_iq(int frame,
+                        int slot,
+                        const NR_DL_FRAME_PARMS *frame_parms,
+                        const nfapi_nr_pusch_pdu_t *rel15_ul,
+                        int number_dmrs_symbols,
+                        int dmrs_port,
+                        const c16_t *data,
+                        int data_len);
+
+/**
+ * @brief Log PUSCH UL interpolated channel estimates using T-Tracer
+ *
+ * This function creates and sends a T_GNB_PHY_UL_FD_CHAN_EST_DMRS_INTERPL message
+ * containing interpolated channel estimates across all subcarriers in frequency domain.
+ *
+ * @param frame Frame number
+ * @param slot Slot number
+ * @param frame_parms Pointer to frame parameters structure
+ * @param rel15_ul Pointer to PUSCH PDU structure
+ * @param number_dmrs_symbols Number of DMRS symbols
+ * @param dmrs_port DMRS port number
+ * @param data Pointer to interpolated channel estimate data buffer (c16_t complex samples)
+ * @param data_len Size of the data buffer in bytes
+ */
+void log_ul_fd_chan_est_dmrs_interpl(int frame,
+                                    int slot,
+                                    const NR_DL_FRAME_PARMS *frame_parms,
+                                    const nfapi_nr_pusch_pdu_t *rel15_ul,
+                                    int number_dmrs_symbols,
+                                    int dmrs_port,
+                                    const c16_t *data,
+                                    int data_len);
+
+/**
+ * @brief Log PUSCH UL received payload bits using T-Tracer
+ *
+ * This function creates and sends a T_GNB_PHY_UL_PAYLOAD_RX_BITS message
+ * containing the decoded transport block payload data.
+ *
+ * @param frame Frame number
+ * @param slot Slot number
+ * @param frame_parms Pointer to frame parameters structure
+ * @param rel15_ul Pointer to PUSCH PDU structure
+ * @param number_dmrs_symbols Number of DMRS symbols
+ * @param dmrs_port DMRS port number
+ * @param data Pointer to decoded transport block data (uint8_t bytes)
+ * @param tb_size Transport block size in bytes
+ */
+void log_ul_payload_rx_bits(int frame,
+                           int slot,
+                           const NR_DL_FRAME_PARMS *frame_parms,
+                           const nfapi_nr_pusch_pdu_t *rel15_ul,
+                           int number_dmrs_symbols,
+                           int dmrs_port,
+                           const uint8_t *data,
+                           int tb_size);
+
+/**
+ * @brief Log UE PUSCH UL transmitted payload bits using T-Tracer
+ *
+ * This function creates and sends a T_UE_PHY_UL_PAYLOAD_TX_BITS message
+ * containing the transport block payload data before encoding.
+ * Note: subcarrier_spacing is derived from frame_parms->subcarrier_spacing
+ * and nb_antennas_tx is used (UE transmit side).
+ *
+ * @param frame Frame number
+ * @param slot Slot number
+ * @param frame_parms Pointer to frame parameters structure
+ * @param pusch_pdu Pointer to UE PUSCH PDU structure
+ * @param number_dmrs_symbols Number of DMRS symbols
+ * @param dmrs_port DMRS port number
+ * @param data Pointer to transport block payload data (uint8_t bytes)
+ * @param tb_size Transport block size in bytes
+ */
+void log_ul_payload_tx_bits(int frame,
+                           int slot,
+                           const NR_DL_FRAME_PARMS *frame_parms,
+                           const nfapi_nr_ue_pusch_pdu_t *pusch_pdu,
+                           int number_dmrs_symbols,
+                           int dmrs_port,
+                           const uint8_t *data,
+                           int tb_size);
+
+/**
+ * @brief Log UE PUSCH UL scrambled transmit bits using T-Tracer
+ *
+ * This function creates and sends a T_UE_PHY_UL_SCRAMBLED_TX_BITS message
+ * containing the scrambled codeword bits after scrambling.
+ * Note: subcarrier_spacing is derived from frame_parms->subcarrier_spacing
+ * and nb_antennas_tx is used (UE transmit side).
+ *
+ * @param frame Frame number
+ * @param slot Slot number
+ * @param frame_parms Pointer to frame parameters structure
+ * @param pusch_pdu Pointer to UE PUSCH PDU structure
+ * @param number_dmrs_symbols Number of DMRS symbols
+ * @param dmrs_port DMRS port number
+ * @param data Pointer to scrambled codeword data (uint8_t bytes)
+ * @param number_of_bits Number of scrambled bits
+ */
+void log_ul_scrambled_tx_bits(int frame,
+                             int slot,
+                             const NR_DL_FRAME_PARMS *frame_parms,
+                             const nfapi_nr_ue_pusch_pdu_t *pusch_pdu,
+                             int number_dmrs_symbols,
+                             int dmrs_port,
+                             const uint8_t *data,
+                             int number_of_bits);
+
+#endif /* T_TRACER */
+
+#endif /* T_MESSAGES_CREATOR_H */

common/utils/T/tracer/shared_memory_config.h

@@ -18,10 +18,13 @@
 #ifndef _SHARED_MEMORY_CONFIG_H_
 #define _SHARED_MEMORY_CONFIG_H_
 
-#define SHMSIZE ((122.88e6 / (100 * 20)) * 1000 * 8) // Assume capture in 1000, each I+Q represented by 8 byts
-//  122.88e6: 10ms at 100 MHz BW of 5G NR
-// 20: 20 slots in 10ms
-// subcarrier spacing: 30 KHz
+#define NUM_MESSAGES_PER_SLOT 5
+#define SHMSIZE ((122.88e6 / (100 * 20)) * 100 * NUM_MESSAGES_PER_SLOT * 8) // ~229 MiB
+//  122.88e6: sample rate at 100 MHz BW of 5G NR
+//  100 * 20: 2000 slots/s (20 slots/frame, 100 frames/s) at SCS 30 kHz
+//  100: number of slots to capture
+//  NUM_MESSAGES_PER_SLOT: messages captured per slot
+//  8: bytes per complex I+Q sample
 
 // for gNB T Tracer App
 #define GETKEYDIR1_gNB ("/tmp/gnb_app1")

common/utils/T/tracer/t_tracer_app_gnb.c

@@ -3,7 +3,7 @@
  */
 
 /*!
- * \brief T-Tracer gnb service to capture tracee Messages from gNB, it is used by data recording application
+ * \brief T-Tracer gnb service to capture trace Messages from gNB, it is used by data recording application
  */
 
 #include <stdio.h>
@@ -34,6 +34,21 @@
 // the buffer in the stack from the previous record
 #define DISCARD_RECORD_DURATION_MS 10
 
+// Sentinel values for field_descriptor: source is e.sending_time
+#define FIELD_SENDING_TIME_SEC  -2
+#define FIELD_SENDING_TIME_NSEC -3
+
+// State machine constants for shared memory protocol
+#define STATE_WAIT   0
+#define STATE_CONFIG 1
+#define STATE_RECORD 2
+#define STATE_STOP   3
+
+// Visual separator for state machine transitions in console output
+#define STATE_SEPARATOR "========================================"
+#define PRINT_STATE_BANNER(state_name) \
+  printf("\n%s\n  [STATE: %s]\n%s\n", STATE_SEPARATOR, state_name, STATE_SEPARATOR)
+
 // Combine bytes in - little-endian format
 int combine_bytes(const uint8_t *bytes, size_t num_bytes)
 {
@@ -44,19 +59,11 @@ int combine_bytes(const uint8_t *bytes, size_t num_bytes)
   return result;
 }
 
-// Convert an integer to an array of bytes - little-endian format
-void int_to_bytes(int num, uint8_t *bytes, size_t num_bytes)
+// Check if a message ID is present in a given list of message IDs
+bool is_message_in_list(int msg_id_list[], int n_msgs, int msg_id)
 {
-  for (size_t i = 0; i < num_bytes; ++i) {
-    bytes[i] = (num >> (i * 8)) & 0xFF;
-  }
-}
-
-// Check if the message is in the list of bits messages
-bool is_bits_messages(int traces_bits_support_data_Collection_format_idx[], int n_bits_msgs, int msg_id)
-{
-  for (int i = 0; i < n_bits_msgs; i++) {
-    if (msg_id == traces_bits_support_data_Collection_format_idx[i]) {
+  for (int i = 0; i < n_msgs; i++) {
+    if (msg_id == msg_id_list[i]) {
       return true;
     }
   }
@@ -71,12 +78,6 @@ struct timespec get_current_time()
   return time;
 }
 
-// Function to convert timespec to microseconds
-long long timespec_to_microseconds(struct timespec time)
-{
-  return (time.tv_sec * 1000000LL) + (time.tv_nsec / 1000);
-}
-
 // Calculate the time difference in milliseconds
 double calculate_time_difference(struct timespec start, struct timespec end)
 {
@@ -88,7 +89,7 @@ double calculate_time_difference(struct timespec start, struct timespec end)
 // Get Time Stamp in microseconds in YYYYMMDDHHMMSSmmmuuu format
 char *get_time_stamp_usec(char time_stamp_str[])
 {
-  // initialization to measure time stamp --This part should be moved to inilization part
+  // initialization to measure time stamp
   time_t my_time;
   struct tm *timeinfo;
   time(&my_time);
@@ -104,31 +105,12 @@ char *get_time_stamp_usec(char time_stamp_str[])
   uint8_t hour = timeinfo->tm_hour;
   uint8_t min = timeinfo->tm_min;
   uint8_t sec = timeinfo->tm_sec;
-  uint16_t usec = (tv.tv_usec);
-  // printf ("Time stamp: %d_%d_%d_%d_%d_%d_%d \n",year,mon,mday,hour,min,sec,usec);
-  // sprintf(time_stamp_str, "%d_%d_%d_%d_%d_%d_%d",year,mon,mday,hour,min,sec,usec);
-  sprintf(time_stamp_str, "%04d%02d%02d%02d%02d%02d%06d", year, mon, mday, hour, min, sec, usec);
+  uint32_t usec = (tv.tv_usec);
+  sprintf(time_stamp_str, "%04d%02d%02d%02d%02d%02d%06u", year, mon, mday, hour, min, sec, usec);
 
   return time_stamp_str;
 }
 
-// Convert timestamp string to integer
-int convert_time_stamp_to_int(const char *timestamp)
-{
-  return atoi(timestamp);
-}
-
-// Split timestamp string and convert to integer
-int split_time_stamp_and_convert_to_int(char time_stamp_str[], int shift, int length)
-{
-  char time_part[length + 1]; // Buffer to hold the date part YYYYMMDD or HHMMSSmmm
-  // Copy the first 8 or 9 characters (YYYYMMDD) to HHMMSSmmm
-  strncpy(time_part, time_stamp_str + shift, length);
-  time_part[length] = '\0'; // Null-terminate the string
-  // Convert timestamp string to integer
-  return convert_time_stamp_to_int(time_part);
-}
-
 void err_exit(char *buf)
 {
   fprintf(stderr, "%s\n", buf);
@@ -153,7 +135,7 @@ int create_shm(char **addrN, const char *shm_path, int projectId)
   shm_id = shmget(key, SHMSIZE, IPC_CREAT | IPC_EXCL | 0664);
   if (shm_id == -1) {
     if (errno == EEXIST) {
-      printf("Error: shared memeory already exist\n");
+      printf("Error: shared memory already exists\n");
       shm_id = shmget(key, 0, 0);
       printf("reference shm_id = %d\n", shm_id);
     } else {
@@ -210,75 +192,28 @@ void usage(void)
   exit(1);
 }
 
-// class to store the message and the action to enable capture selected data
-// 0: do not record message
-// 1: record message
-struct trace_struct {
-  char on_off_name[100];
-  int on_off_action;
-};
-
 // struct for trace message based on Data Collection Trace Messages Structure
-// you need to define the vararibles of each message
+// you need to define the variables of each message
 typedef struct {
   /* Data Collection Trace Message Structure  */
   int frame;
   int slot;
-  int datetime_yyyymmdd;
-  int datetime_hhmmssmmm;
   int frame_type, freq_range, subcarrier_spacing, cyclic_prefix, symbols_per_slot;
   int Nid_cell, rnti;
   int rb_size, rb_start, start_symbol_index, nr_of_symbols;
   int qam_mod_order, mcs_index, mcs_table, nrOfLayers, transform_precoding;
   int dmrs_config_type, ul_dmrs_symb_pos, number_dmrs_symbols;
   int dmrs_port, dmrs_nscid, nb_antennas_rx, number_of_bits;
-  int data_size, data;
-} event_trace_msg_data;
+  int data;
+} event_trace_msg_ul_data;
 
-void setup_trace_msg_data(event_trace_msg_data *d, void *database)
+void setup_trace_msg_ul_data(event_trace_msg_ul_data *d, void *database)
 {
   database_event_format f;
   int i;
 
-  /* Data Collection Trace Message Structure */
-  // FORMAT = int,frame : int,slot : int,datetime_yyyymmdd : int,datetime_hhmmssmmm :
-  // int,frame_type : int,freq_range : int,subcarrier_spacing : int,cyclic_prefix : int,symbols_per_slot :
-  // int,Nid_cell : int,rnti :
-  // int,rb_size : int,rb_start : int,start_symbol_index : int,nr_of_symbols :
-  // int,qam_mod_order : int,mcs_index : int,mcs_table : int,nrOfLayers :
-  // int,transform_precoding : int,dmrs_config_type : int,ul_dmrs_symb_pos :  int,number_dmrs_symbols :
-  // int,dmrs_port : int,dmrs_nscid :
-  // int,nb_antennas_rx : int,number_of_bits : buffer,data
-
-  // Initialize the data structure
-  d->frame = -1;
-  d->slot = -1;
-  d->datetime_yyyymmdd = -1;
-  d->datetime_hhmmssmmm = -1;
-  d->frame_type = -1;
-  d->freq_range = -1;
-  d->subcarrier_spacing = -1;
-  d->cyclic_prefix = -1;
-  d->symbols_per_slot = -1;
-  d->Nid_cell = -1;
-  d->rnti = -1;
-  d->rb_size = -1;
-  d->rb_start = -1;
-  d->start_symbol_index = -1;
-  d->nr_of_symbols = -1;
-  d->qam_mod_order = -1;
-  d->mcs_index = -1;
-  d->mcs_table = -1;
-  d->nrOfLayers = -1;
-  d->transform_precoding = -1;
-  d->dmrs_config_type = -1;
-  d->ul_dmrs_symb_pos = -1;
-  d->number_dmrs_symbols = -1;
-  d->dmrs_port = -1;
-  d->dmrs_nscid = -1;
-  d->nb_antennas_rx = -1;
-  d->number_of_bits = -1;
-  d->data = -1;
+  // Initialize all fields to -1 (marks unset indices)
+  memset(d, -1, sizeof(*d));
 
 /* this macro looks for a particular element and checks its type */
 #define G(var_name, var_type, var)                                      \
@@ -291,35 +226,14 @@ void setup_trace_msg_data(event_trace_msg_data *d, void *database)
     continue;                                                           \
   }
 
-  /* ------------------------------*/
-  /* Create Macro for Data Collection Trace Message */
-  /* ------------------------------*/
-  // Data Collection Trace Message Structure
-  // Example: GNB_PHY_UL_FD_PUSCH_IQ,  GNB_PHY_UL_FD_DMRS_ID,
-  //          GNB_PHY_UL_FD_CHAN_EST_DMRS_POS, GNB_PHY_UL_FD_CHAN_EST_DMRS_INTERPL
-  //          GNB_PHY_UL_PAYLOAD_RX_BITS
-  //          UE_PHY_UL_SCRAMBLED_TX_BITS
-  //          UE_PHY_UL_PAYLOAD_TX_BITS
-  // Get a template of any message based on Data Collection Trace Messages Structure
+  // Get a template of any UL message based on Data Collection Trace Messages Structure
   int Trace_MSG_ID = event_id_from_name(database, "GNB_PHY_UL_FD_PUSCH_IQ");
   f = get_format(database, Trace_MSG_ID);
 
-  /* get the elements of the trace
-   * the value is an index in the event, see below */
-  // FORMAT = int,frame : int,slot : int,datetime_yyyymmdd : int,datetime_hhmmssmmm :
-  // int,frame_type : int,freq_range : int,subcarrier_spacing : int,cyclic_prefix : int,symbols_per_slot :
-  // int,Nid_cell : int,rnti :
-  // int,rb_size : int,rb_start : int,start_symbol_index : int,nr_of_symbols :
-  // int,qam_mod_order : int,mcs_index : int,mcs_table : int,nrOfLayers :
-  // int,transform_precoding : int,dmrs_config_type : int,ul_dmrs_symb_pos :  int,number_dmrs_symbols :
-  // int,dmrs_port : int,dmrs_nscid :
-  // int,nb_antennas_rx : int,number_of_bits : buffer,data
-
+  // Map each field name to its index in the event structure
   for (i = 0; i < f.count; i++) {
     G("frame", "int", d->frame)
     G("slot", "int", d->slot)
-    G("datetime_yyyymmdd", "int", d->datetime_yyyymmdd)
-    G("datetime_hhmmssmmm", "int", d->datetime_hhmmssmmm)
     G("frame_type", "int", d->frame_type)
     G("freq_range", "int", d->freq_range)
     G("subcarrier_spacing", "int", d->subcarrier_spacing)
@@ -345,20 +259,106 @@ void setup_trace_msg_data(event_trace_msg_data *d, void *database)
     G("number_of_bits", "int", d->number_of_bits)
     G("data", "buffer", d->data)
   }
-  // if (d->frame == -1 || d->slot == -1) goto error;
 #undef G
-  return;
 }
 
-// Function to check if a value is in the array
-int isValueInArray(int value, int arr[], int size)
+// -------------------------------------------------------------------
+// Table-driven metadata shared memory writer and debug printer
+// -------------------------------------------------------------------
+
+// Descriptor for one metadata field to write to shared memory
+typedef struct {
+  int field_idx;     // index into event e.e[] array
+  size_t wire_size;  // number of bytes to write to shm
+} field_descriptor;
+
+// Number of UL metadata fields written to shared memory (excludes msg_id and buffer)
+#define N_UL_METADATA_FIELDS 27
+
+// Build UL field descriptor array from a populated event_trace_msg_ul_data struct.
+// Must be called after setup_trace_msg_ul_data() so that field indices are populated.
+void build_ul_field_descriptors(field_descriptor *fields, const event_trace_msg_ul_data *d)
 {
-  for (int i = 0; i < size; i++) {
-    if (arr[i] == value) {
-      return 1; // Value found
+  int i = 0;
+  fields[i++] = (field_descriptor){ d->frame, sizeof(uint16_t) };
+  fields[i++] = (field_descriptor){ d->slot, sizeof(uint8_t) };
+  fields[i++] = (field_descriptor){ FIELD_SENDING_TIME_SEC, sizeof(uint32_t) };
+  fields[i++] = (field_descriptor){ FIELD_SENDING_TIME_NSEC, sizeof(uint32_t) };
+  fields[i++] = (field_descriptor){ d->frame_type, sizeof(uint8_t) };
+  fields[i++] = (field_descriptor){ d->freq_range, sizeof(uint8_t) };
+  fields[i++] = (field_descriptor){ d->subcarrier_spacing, sizeof(uint8_t) };
+  fields[i++] = (field_descriptor){ d->cyclic_prefix, sizeof(uint8_t) };
+  fields[i++] = (field_descriptor){ d->symbols_per_slot, sizeof(uint8_t) };
+  fields[i++] = (field_descriptor){ d->Nid_cell, sizeof(uint16_t) };
+  fields[i++] = (field_descriptor){ d->rnti, sizeof(uint16_t) };
+  fields[i++] = (field_descriptor){ d->rb_size, sizeof(uint16_t) };
+  fields[i++] = (field_descriptor){ d->rb_start, sizeof(uint16_t) };
+  fields[i++] = (field_descriptor){ d->start_symbol_index, sizeof(uint8_t) };
+  fields[i++] = (field_descriptor){ d->nr_of_symbols, sizeof(uint8_t) };
+  fields[i++] = (field_descriptor){ d->qam_mod_order, sizeof(uint8_t) };
+  fields[i++] = (field_descriptor){ d->mcs_index, sizeof(uint8_t) };
+  fields[i++] = (field_descriptor){ d->mcs_table, sizeof(uint8_t) };
+  fields[i++] = (field_descriptor){ d->nrOfLayers, sizeof(uint8_t) };
+  fields[i++] = (field_descriptor){ d->transform_precoding, sizeof(uint8_t) };
+  fields[i++] = (field_descriptor){ d->dmrs_config_type, sizeof(uint8_t) };
+  fields[i++] = (field_descriptor){ d->ul_dmrs_symb_pos, sizeof(uint16_t) };
+  fields[i++] = (field_descriptor){ d->number_dmrs_symbols, sizeof(uint8_t) };
+  fields[i++] = (field_descriptor){ d->dmrs_port, sizeof(uint16_t) };
+  fields[i++] = (field_descriptor){ d->dmrs_nscid, sizeof(uint16_t) };
+  fields[i++] = (field_descriptor){ d->nb_antennas_rx, sizeof(uint8_t) };
+  fields[i++] = (field_descriptor){ d->number_of_bits, sizeof(uint32_t) };
+}
+
+// Print UL metadata fields for debug output
+void print_ul_metadata_debug(event e, const event_trace_msg_ul_data *d, void *database)
+{
+  printf("\nUnix TS: %ld.%09ld", e.sending_time.tv_sec, e.sending_time.tv_nsec);
+  printf("\nMessage Info: msg_id %s (%d) \n", event_name_from_id(database, e.type), e.type);
+  printf("frame %d, slot %d, unix_ts %ld.%09ld\n",
+         e.e[d->frame].i, e.e[d->slot].i,
+         e.sending_time.tv_sec, e.sending_time.tv_nsec);
+  printf("frame_type %d, freq_range %d, subcarrier_spacing %d, cyclic_prefix %d, symbols_per_slot %d\n",
+         e.e[d->frame_type].i, e.e[d->freq_range].i,
+         e.e[d->subcarrier_spacing].i, e.e[d->cyclic_prefix].i,
+         e.e[d->symbols_per_slot].i);
+  printf("Nid_cell %d, rnti %d, rb_size %d, rb_start %d, start_symbol_index %d, nr_of_symbols %d\n",
+         e.e[d->Nid_cell].i, e.e[d->rnti].i,
+         e.e[d->rb_size].i, e.e[d->rb_start].i,
+         e.e[d->start_symbol_index].i, e.e[d->nr_of_symbols].i);
+  printf("qam_mod_order %d, mcs_index %d, mcs_table %d, nrOfLayers %d, transform_precoding %d\n",
+         e.e[d->qam_mod_order].i, e.e[d->mcs_index].i,
+         e.e[d->mcs_table].i, e.e[d->nrOfLayers].i,
+         e.e[d->transform_precoding].i);
+  printf("dmrs_config_type %d, ul_dmrs_symb_pos %d, number_dmrs_symbols %d, dmrs_port %d, dmrs_nscid %d\n",
+         e.e[d->dmrs_config_type].i, e.e[d->ul_dmrs_symb_pos].i,
+         e.e[d->number_dmrs_symbols].i, e.e[d->dmrs_port].i,
+         e.e[d->dmrs_nscid].i);
+  printf("nb_antennas_rx %d, number_of_bits %d, data size %d\n",
+         e.e[d->nb_antennas_rx].i, e.e[d->number_of_bits].i,
+         e.e[d->data].bsize);
+}
+
+// Write metadata fields to shared memory using field descriptor table.
+// Writes msg_id first, then loops through all fields.
+void write_metadata_to_shm(char *addr_wr, unsigned int *bufIdx_wr, event e,
+                           const field_descriptor *fields, int n_fields)
+{
+  // Write message ID
+  memcpy(&addr_wr[*bufIdx_wr], &e.type, sizeof(uint16_t));
+  *bufIdx_wr += sizeof(uint16_t);
+  // Write all metadata fields
+  for (int i = 0; i < n_fields; i++) {
+    if (fields[i].field_idx == FIELD_SENDING_TIME_SEC) {
+      uint32_t ts_sec = (uint32_t)e.sending_time.tv_sec;
+      memcpy(&addr_wr[*bufIdx_wr], &ts_sec, fields[i].wire_size);
+    } else if (fields[i].field_idx == FIELD_SENDING_TIME_NSEC) {
+      uint32_t ts_nsec = (uint32_t)e.sending_time.tv_nsec;
+      memcpy(&addr_wr[*bufIdx_wr], &ts_nsec, fields[i].wire_size);
+    } else {
+      memcpy(&addr_wr[*bufIdx_wr], &e.e[fields[i].field_idx].i, fields[i].wire_size);
     }
+    *bufIdx_wr += fields[i].wire_size;
   }
-  return 0; // Value not found
 }
 
 void reestablish_connection(int *socket, char *ip, int port, int number_of_events, int *is_on)
@@ -386,40 +386,24 @@ int main(int n, char **v)
   // trace_msgs_support_data_Collection_format
   // it is used to parse the requested messages if it is based
   // on Data Collection Trace Messages Structure and supported tracer messages indices
-  char *traces_iq_support_data_Collection_format[] = {"GNB_PHY_UL_FD_PUSCH_IQ",
-                                                            "GNB_PHY_UL_FD_DMRS",
-                                                            "GNB_PHY_UL_FD_CHAN_EST_DMRS_POS",
-                                                            "GNB_PHY_UL_FD_CHAN_EST_DMRS_INTERPL"};
-  char *traces_bits_support_data_Collection_format[] = {"GNB_PHY_UL_PAYLOAD_RX_BITS",
-                                                              "UE_PHY_UL_SCRAMBLED_TX_BITS",
-                                                              "UE_PHY_UL_PAYLOAD_TX_BITS"};
-  // extra number of records to simlify Sync between base station and UE synchronization records.
+  char *traces_ul_support_data_Collection_format[] = {"GNB_PHY_UL_FD_PUSCH_IQ",
+                                                      "GNB_PHY_UL_FD_DMRS",
+                                                      "GNB_PHY_UL_FD_CHAN_EST_DMRS_POS",
+                                                      "GNB_PHY_UL_FD_CHAN_EST_DMRS_INTERPL",
+                                                      "GNB_PHY_UL_PAYLOAD_RX_BITS",
+                                                      "UE_PHY_UL_SCRAMBLED_TX_BITS",
+                                                      "UE_PHY_UL_PAYLOAD_TX_BITS"};
+  // extra number of records to simplify sync between base station and UE synchronization records.
   // if you have network delay, you can increase the number of records to capture
   int max_sync_offset = 6; // 6 frames ~ 60 ms
-  // all supported messages
-  // Calculate the size of the combined array
-  int n_iq_msgs = sizeof(traces_iq_support_data_Collection_format) / sizeof(traces_iq_support_data_Collection_format[0]);
-  int n_bits_msgs = sizeof(traces_bits_support_data_Collection_format) / sizeof(traces_bits_support_data_Collection_format[0]);
-  int n_msgs_based_data_Collection_format = n_iq_msgs + n_bits_msgs;
-
-  // Create the combined array
-  char *traces_support_data_Collection_format[n_msgs_based_data_Collection_format];
-  // Copy IQ messages to the combined array
-  for (int i = 0; i < n_iq_msgs; i++) {
-    traces_support_data_Collection_format[i] = traces_iq_support_data_Collection_format[i];
-  }
-
-  // Copy Bits messages to the combined array
-  for (int i = 0; i < n_bits_msgs; i++) {
-    traces_support_data_Collection_format[i + n_iq_msgs] = traces_bits_support_data_Collection_format[i];
-  }
+  // Calculate the size of the message array
+  int n_ul_msgs = sizeof(traces_ul_support_data_Collection_format) / sizeof(traces_ul_support_data_Collection_format[0]);
 
   uint16_t msg_id = 0;
   uint16_t start_frame_number = 0;
   uint32_t number_records = 0; // number of records to capture, it is number of slots
   // array to store the requested tracer messages indices
   int req_tracer_msgs_indices[100] = {0};
-  // define variables --> to do: add all of them to be class of pointers
   char *database_filename = NULL;
   void *database;
   char *ip = DEFAULT_REMOTE_IP;
@@ -432,18 +416,19 @@ int main(int n, char **v)
   char trace_time_stamp_str[30];
 
   // data structure for the trace messages based on Data Collection Trace Messages Structure
-  event_trace_msg_data trace_msg_data;
+  event_trace_msg_ul_data trace_msg_ul_data;
 
-  // initlization variables
+  // initialization variables
   unsigned int bufIdx_wr = 0;
   unsigned int bufIdx_rd = 0;
   uint8_t num_req_tracer_msgs = 0;
 
-  // initilaze shared memory
+  // initialize shared memory
   char *addr_wr, *addr_rd;
-  printf("\n Data Collection Service: Initializing shared memory ...");
-  printf("\n Directory 1: %s, Directory 2: %s", GETKEYDIR1_gNB, GETKEYDIR2_gNB);
-  printf("\n Project ID: %d\n", PROJECTID_gNB);
+  PRINT_STATE_BANNER("INIT");
+  printf("  Data Collection Service: Initializing shared memory ...\n");
+  printf("  Directory 1: %s, Directory 2: %s\n", GETKEYDIR1_gNB, GETKEYDIR2_gNB);
+  printf("  Project ID: %d\n", PROJECTID_gNB);
   int shm_id_wr = create_shm(&addr_wr, GETKEYDIR1_gNB, PROJECTID_gNB);
   int shm_id_rd = create_shm(&addr_rd, GETKEYDIR2_gNB, PROJECTID_gNB);
   del_shm(addr_wr, shm_id_wr);
@@ -500,20 +485,18 @@ int main(int n, char **v)
   addr_rd[0] = 0; // important to check if we have new requested messages
 
   // read requested tracer msg indices from memory
-  printf("\n Data Collection Service: Waiting for messages request ...\n");
-  // 0: Wait
-  // 1: config
-  // 2: record
-  // 3: stop
+  PRINT_STATE_BANNER("WAIT");
+  printf("  Data Collection Service: Waiting for config message ...\n");
   // Wait for Action Config
   while (1) {
-    if ((uint8_t)(addr_rd[0]) == 0) {
+    if ((uint8_t)(addr_rd[0]) == STATE_WAIT) {
       usleep(50); // sleep for 50 us: 0.5 ms slot duration
     }
     // config state
-    else if ((uint8_t)(addr_rd[0]) == 1) {
+    else if ((uint8_t)(addr_rd[0]) == STATE_CONFIG) {
       get_time_stamp_usec(trace_time_stamp_str);
-      printf("\n Received config message. Time Stamp: %s", trace_time_stamp_str);
+      PRINT_STATE_BANNER("CONFIG");
+      printf("  Received config message. Time Stamp: %s\n", trace_time_stamp_str);
 
       // get the IP address length in bytes
       bufIdx_rd = 1;
@@ -530,11 +513,11 @@ int main(int n, char **v)
 
       // get the array bytes of the port number : 2 bytes
       uint8_t port_number_bytes[2] = {addr_rd[bufIdx_rd], addr_rd[bufIdx_rd + 1]};
-      bufIdx_rd += 2; // + 2 bytes = start frame number
+      bufIdx_rd += 2; // + 2 bytes = port number
 
       port = combine_bytes(port_number_bytes, 2);
-      printf("\n Parameters: IP Address length: %d, IP Address: %s, Port Number: %d \n", ip_address_length, ip_address, port);
-      addr_rd[0] = 0; // reset memory : to wait for record action
+      printf("  Parameters: IP Address length: %d, IP Address: %s, Port Number: %d\n", ip_address_length, ip_address, port);
+      addr_rd[0] = STATE_WAIT; // reset memory : to wait for record action
       break;
     }
   }
@@ -545,37 +528,39 @@ int main(int n, char **v)
 
   /* connect to the nr-softmodem */
   socket = connect_to(ip, port);
-  printf("\n Connected to nr-softmodem");
+  printf("  Connected to nr-softmodem\n");
 
   // Read Action record or stop
-  printf("\n Data Collection Service: Waiting for record message ...\n");
+  PRINT_STATE_BANNER("WAIT");
+  printf("  Data Collection Service: Waiting for record/stop message ...\n");
   while (1) {
     // wait for Action record
-    if ((uint8_t)(addr_rd[0]) == 0) {
+    if ((uint8_t)(addr_rd[0]) == STATE_WAIT) {
       usleep(50); // sleep for 50 us: 0.5 ms slot duration
     }
 
     // quit state
-    else if ((uint8_t)(addr_rd[0]) == 3) {
-      printf("\n Received 'stop' command. Exiting...");
-      printf("\n ");
+    else if ((uint8_t)(addr_rd[0]) == STATE_STOP) {
+      PRINT_STATE_BANNER("STOP");
+      printf("  Received 'stop' command. Exiting...\n");
       // Clean up and exit
       break;
     }
     // record state
-    else if ((uint8_t)(addr_rd[0]) == 2) {
+    else if ((uint8_t)(addr_rd[0]) == STATE_RECORD) {
       // clear remote buffer if there is
       clear_remote_config();
       get_time_stamp_usec(trace_time_stamp_str);
-      printf("\n Received record message. Time Stamp: %s", trace_time_stamp_str);
+      PRINT_STATE_BANNER("RECORD");
+      printf("  Received record message. Time Stamp: %s\n", trace_time_stamp_str);
 
       // read number of requested messages
       bufIdx_rd = 1;
       num_req_tracer_msgs = addr_rd[bufIdx_rd];
       bufIdx_rd += 1;
-      printf("\n Number of requested tracer messages: %d,", num_req_tracer_msgs);
+      printf("  Number of requested tracer messages: %d\n", num_req_tracer_msgs);
       // reset memory : action to wait for next record action
-      addr_rd[0] = 0;
+      addr_rd[0] = STATE_WAIT;
 
       // read tracer msg IDs - every message ID has been stored in two bytes
       for (uint8_t msg_n = 0; msg_n < num_req_tracer_msgs; msg_n++) {
@@ -607,40 +592,36 @@ int main(int n, char **v)
       printf("start_frame: %d\n", start_frame_number);
 
       /* activate the trace in this array */
-      printf("\n Activate Tracer messages in on_off array: ");
+      printf("  Activating tracer messages:\n");
       for (i = 0; i < num_req_tracer_msgs; i++) {
         char *on_off_name = event_name_from_id(database, req_tracer_msgs_indices[i]);
         on_off(database, on_off_name, is_on, 1);
-        printf("%d, %s, ", req_tracer_msgs_indices[i], on_off_name);
-        // on_off(database, "GNB_PHY_DL_OUTPUT_SIGNAL", is_on, 1);
+        printf("    %d: %s\n", req_tracer_msgs_indices[i], on_off_name);
       }
 
-      // get the event IDs for the bit messages
-      int traces_bits_support_data_Collection_format_idx[n_bits_msgs];
-      for (int i = 0; i < n_bits_msgs; i++) {
-        traces_bits_support_data_Collection_format_idx[i] =
-            event_id_from_name(database, traces_bits_support_data_Collection_format[i]);
-      }
-
-      // all supported messages
-      int traces_support_data_Collection_format_idx[n_msgs_based_data_Collection_format];
-      for (int i = 0; i < n_msgs_based_data_Collection_format; i++) {
-        traces_support_data_Collection_format_idx[i] = event_id_from_name(database, traces_support_data_Collection_format[i]);
+      // Build UL message ID array for generic UL dispatch
+      int ul_msg_ids[n_ul_msgs];
+      for (int i = 0; i < n_ul_msgs; i++) {
+        ul_msg_ids[i] = event_id_from_name(database, traces_ul_support_data_Collection_format[i]);
       }
 
       // setup data for the trace messages
-      setup_trace_msg_data(&trace_msg_data, database);
-      printf("\n Setup Trace Data Done");
+      setup_trace_msg_ul_data(&trace_msg_ul_data, database);
+      printf("  Setup UL trace data done\n");
+
+      // Build field descriptor table for table-driven shared memory writes
+      field_descriptor ul_fields[N_UL_METADATA_FIELDS];
+      build_ul_field_descriptors(ul_fields, &trace_msg_ul_data);
 
       // Get the start time
       struct timespec start_time = get_current_time();
 
-      /* activate the tracee in the nr-softmodem */
+      /* activate the traces in the nr-softmodem */
       activate_traces(socket, number_of_events, is_on);
-      printf("\n Activated Traces in nr-softmodem");
+      printf("  Activated traces in nr-softmodem\n");
 
       /* a buffer needed to receive events from the nr-softmodem */
-      OBUF ebuf = {osize : 0, omaxsize : 0, obuf : NULL};
+      OBUF ebuf = {.osize = 0, .omaxsize = 0, .obuf = NULL};
 
       /* read events */
       int nrecord_idx = 0;
@@ -655,8 +636,8 @@ int main(int n, char **v)
       int current_frame = 0, prev_frame = 0, current_slot = 0, prev_slot = 0;
       // offset to sync between base station and UE synchronization records or power measurements
       int sync_offset_index = 0; // increase the index only if the index of frame changes after getting all records
-      // since we will use the frame differece to do extra records, we should be sure that the last slot is recorded completely
-      printf("\n\n Data Collection Service: Start reading messages ...");
+      // since we will use the frame difference to do extra records, we should be sure that the last slot is recorded completely
+      printf("\n  Data Collection Service: Start reading messages ...\n");
       struct pollfd event_poll_fd;
       event_poll_fd.fd = socket;
       event_poll_fd.events = POLLIN;
@@ -676,27 +657,24 @@ int main(int n, char **v)
             reestablish_connection(&socket, ip, port, number_of_events, is_on);
             continue; // Skip further processing and retry
           }
-          //-------------------------
-          // GNB_PHY_UL_FD_PUSCH_IQ,  GNB_PHY_UL_FD_DMRS_ID, GNB_PHY_UL_FD_CHAN_EST_DMRS_POS,
-          // UE_PHY_UL_SCRAMBLED_TX_BITS, GNB_PHY_UL_PAYLOAD_RX_BITS, UE_PHY_UL_PAYLOAD_TX_BITS
-          //-------------------------
+
           // is it a requested message
-          if (isValueInArray(e.type, req_tracer_msgs_indices, num_req_tracer_msgs)) {
+          if (is_message_in_list(req_tracer_msgs_indices, num_req_tracer_msgs, e.type)) {
             // is it based on Data Collection Trace Messages Structure
-            if (isValueInArray(e.type, traces_support_data_Collection_format_idx, n_msgs_based_data_Collection_format)) {
+            if (is_message_in_list(ul_msg_ids, n_ul_msgs, e.type)) {
               // Start recording from the next slot to mitigate capturing partial data
               // check if the current frame and slot are different from the previous frame and slot
               // Then, increase the record index
               if (start_recording == false) {
                 if (got_ref_frame_slot == false) {
-                  ref_frame = e.e[trace_msg_data.frame].i;
-                  ref_slot = e.e[trace_msg_data.slot].i;
+                  ref_frame = e.e[trace_msg_ul_data.frame].i;
+                  ref_slot = e.e[trace_msg_ul_data.slot].i;
                   printf("\nMessage Info: msg_id %s (%d) \n", event_name_from_id(database, e.type), e.type);
                   got_ref_frame_slot = true;
                 }
 
-                current_frame = e.e[trace_msg_data.frame].i;
-                current_slot = e.e[trace_msg_data.slot].i;
+                current_frame = e.e[trace_msg_ul_data.frame].i;
+                current_slot = e.e[trace_msg_ul_data.slot].i;
                 frame_difference = (current_frame - ref_frame + MAX_FRAME_INDEX + 1) % (MAX_FRAME_INDEX + 1);
                 slot_difference = (current_slot - ref_slot + MAX_SLOT_INDEX + 1) % (MAX_SLOT_INDEX + 1);
                 printf("\n First frame.slot: %d.%d, current frame.slot: %d.%d, diff frame.slot: %d.%d",
@@ -709,129 +687,30 @@ int main(int n, char **v)
 
                 if ((ref_frame != current_frame) || (ref_slot != current_slot)) {
                   start_recording = true;
-                  printf("\n Start recording from frame: %d, slot: %d ", e.e[trace_msg_data.frame].i, e.e[trace_msg_data.slot].i);
+                  printf("\n Start recording from frame: %d, slot: %d ", e.e[trace_msg_ul_data.frame].i, e.e[trace_msg_ul_data.slot].i);
                 }
               }
               // start recording from the next frame to mitigate capturing partial data
               if (start_recording == true) {
-                /* this is how to access the elements of the Data Collection trace messages.
-                 * we use e.e[<element>] and then the correct suffix, here
-                 * it's .i for the integer and .b for the buffer and .bsize for the buffer size
-                 * see in event.h the structure event_arg
-                 */
-                unsigned char *buf = e.e[trace_msg_data.data].b;
-
                 printf("\n\nRecord number: %d", nrecord_idx);
 #ifdef DEBUG_BUFFER
                 printf("\nBuffer index in bytes: %d", bufIdx_wr);
 #endif
-
-                // add general message header:  message ID,
-                // T-Tracer Message format
-                // FORMAT = int,frame : int,slot : int,datetime_yyyymmdd : int,datetime_hhmmssmmm :
-                // int,frame_type : int,freq_range : int,subcarrier_spacing : int,cyclic_prefix : int,symbols_per_slot :
-                // int,Nid_cell : int,rnti :
-                // int,rb_size : int,rb_start : int,start_symbol_index : int,nr_of_symbols :
-                // int,qam_mod_order : int,mcs_index : int,mcs_table : int,nrOfLayers :
-                // int,transform_precoding : int,dmrs_config_type : int,ul_dmrs_symb_pos :  int,number_dmrs_symbols :
-                // int,dmrs_port : int,dmrs_nscid :
-                // int,nb_antennas_rx : int,number_of_bits : buffer,data
-                memcpy(&addr_wr[bufIdx_wr], &e.type, sizeof(uint16_t));
-                bufIdx_wr += sizeof(uint16_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.frame].i, sizeof(uint16_t));
-                bufIdx_wr += sizeof(uint16_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.slot].i, sizeof(uint8_t));
-                bufIdx_wr += sizeof(uint8_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.datetime_yyyymmdd].i, sizeof(uint32_t));
+                // --- UL IQ/Bits handler ---
+                // Write metadata + buffer to shared memory
+                write_metadata_to_shm(addr_wr, &bufIdx_wr, e, ul_fields, N_UL_METADATA_FIELDS);
+                // Write buffer: size (uint32_t) + raw data
+                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_ul_data.data].bsize, sizeof(uint32_t));
                 bufIdx_wr += sizeof(uint32_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.datetime_hhmmssmmm].i, sizeof(uint32_t));
-                bufIdx_wr += sizeof(uint32_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.frame_type].i, sizeof(uint8_t));
-                bufIdx_wr += sizeof(uint8_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.freq_range].i, sizeof(uint8_t));
-                bufIdx_wr += sizeof(uint8_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.subcarrier_spacing].i, sizeof(uint8_t));
-                bufIdx_wr += sizeof(uint8_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.cyclic_prefix].i, sizeof(uint8_t));
-                bufIdx_wr += sizeof(uint8_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.symbols_per_slot].i, sizeof(uint8_t));
-                bufIdx_wr += sizeof(uint8_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.Nid_cell].i, sizeof(uint16_t));
-                bufIdx_wr += sizeof(uint16_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.rnti].i, sizeof(uint16_t));
-                bufIdx_wr += sizeof(uint16_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.rb_size].i, sizeof(uint16_t));
-                bufIdx_wr += sizeof(uint16_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.rb_start].i, sizeof(uint16_t));
-                bufIdx_wr += sizeof(uint16_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.start_symbol_index].i, sizeof(uint8_t));
-                bufIdx_wr += sizeof(uint8_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.nr_of_symbols].i, sizeof(uint8_t));
-                bufIdx_wr += sizeof(uint8_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.qam_mod_order].i, sizeof(uint8_t));
-                bufIdx_wr += sizeof(uint8_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.mcs_index].i, sizeof(uint8_t));
-                bufIdx_wr += sizeof(uint8_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.mcs_table].i, sizeof(uint8_t));
-                bufIdx_wr += sizeof(uint8_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.nrOfLayers].i, sizeof(uint8_t));
-                bufIdx_wr += sizeof(uint8_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.transform_precoding].i, sizeof(uint8_t));
-                bufIdx_wr += sizeof(uint8_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.dmrs_config_type].i, sizeof(uint8_t));
-                bufIdx_wr += sizeof(uint8_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.ul_dmrs_symb_pos].i, sizeof(uint16_t));
-                bufIdx_wr += sizeof(uint16_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.number_dmrs_symbols].i, sizeof(uint8_t));
-                bufIdx_wr += sizeof(uint8_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.dmrs_port].i, sizeof(uint16_t));
-                bufIdx_wr += sizeof(uint16_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.dmrs_nscid].i, sizeof(uint16_t));
-                bufIdx_wr += sizeof(uint16_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.nb_antennas_rx].i, sizeof(uint8_t));
-                bufIdx_wr += sizeof(uint8_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.number_of_bits].i, sizeof(uint32_t));
-                bufIdx_wr += sizeof(uint32_t);
-
-                printf("\nTime Stamp: %d_%d", e.e[trace_msg_data.datetime_yyyymmdd].i, e.e[trace_msg_data.datetime_hhmmssmmm].i);
-
-                // add message body: length in bytes + recorded data
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.data].bsize, sizeof(uint32_t));
-                bufIdx_wr += sizeof(uint32_t);
-
-                // read data from buffer and convert from unsigned char * array to int 16 using the right endianness
-                // T-tracer: Little Endian
-                // For BITS Messages, example: UE_PHY_UL_SCRAMBLED_TX_BITS: data in bytes
-
-                if (is_bits_messages(traces_bits_support_data_Collection_format_idx, n_bits_msgs, e.type)) {
-                  for (int byte_idx = 0; byte_idx < e.e[trace_msg_data.data].bsize; byte_idx += 1) {
-                    // printf("%d, ", buf[byte_idx]);
-                    memcpy(&addr_wr[bufIdx_wr], &buf[byte_idx], sizeof(uint8_t));
-                    bufIdx_wr += sizeof(uint8_t);
-                  }
-
-                } else {
-                  for (int byte_idx = 0; byte_idx < e.e[trace_msg_data.data].bsize; byte_idx += 2) {
-                    // For a little-endian system:
-                    memcpy(&addr_wr[bufIdx_wr], &buf[byte_idx], sizeof(uint8_t));
-                    bufIdx_wr += sizeof(uint8_t);
-                    memcpy(&addr_wr[bufIdx_wr], &buf[byte_idx + 1], sizeof(uint8_t));
-                    bufIdx_wr += sizeof(uint8_t);
-                  }
-                }
-                /*
-                for (int byte_idx_i = 0; byte_idx_i < e.e[d.nr_ul_fd_dmrs_data].bsize; byte_idx_i+=4) {
-                  int16_t I = buf[byte_idx_i] | (buf[byte_idx_i+1] << 8);
-                  int16_t Q = buf[byte_idx_i+2] | (buf[byte_idx_i+3] << 8);
-                  printf ("idx %d, ", byte_idx_i);
-                  printf ("\n%d", I);
-                  printf ("\n%d", Q);
-                }
-                */
+                memcpy(&addr_wr[bufIdx_wr], e.e[trace_msg_ul_data.data].b, e.e[trace_msg_ul_data.data].bsize);
+                bufIdx_wr += e.e[trace_msg_ul_data.data].bsize;
+#ifdef DEBUG_T_Tracer
+                print_ul_metadata_debug(e, &trace_msg_ul_data, database);
+#endif
                 // check if the current frame and slot are different from the previous frame and slot
                 // Then, increase the record index
-                current_frame = e.e[trace_msg_data.frame].i;
-                current_slot = e.e[trace_msg_data.slot].i;
+                current_frame = e.e[trace_msg_ul_data.frame].i;
+                current_slot = e.e[trace_msg_ul_data.slot].i;
 
                 // increase sync offset index if the current frame is different from the previous frame
                 if ((current_frame != prev_frame) && nrecord_idx >= number_records) {
@@ -844,43 +723,7 @@ int main(int n, char **v)
                   prev_frame = current_frame;
                   prev_slot = current_slot;
                 }
-#ifdef DEBUG_T_Tracer
-                printf("\nMessage Info: msg_id %s (%d) \n", event_name_from_id(database, e.type), e.type);
-                printf("frame %d, slot %d, datetime %d_%d\n",
-                       e.e[trace_msg_data.frame].i,
-                       e.e[trace_msg_data.slot].i,
-                       e.e[trace_msg_data.datetime_yyyymmdd].i,
-                       e.e[trace_msg_data.datetime_hhmmssmmm].i);
-                printf("frame_type %d, freq_range %d, subcarrier_spacing %d, cyclic_prefix %d, symbols_per_slot %d\n",
-                       e.e[trace_msg_data.frame_type].i,
-                       e.e[trace_msg_data.freq_range].i,
-                       e.e[trace_msg_data.subcarrier_spacing].i,
-                       e.e[trace_msg_data.cyclic_prefix].i,
-                       e.e[trace_msg_data.symbols_per_slot].i);
-                printf("Nid_cell %d, rnti %d, rb_size %d, rb_start %d, start_symbol_index %d, nr_of_symbols %d\n",
-                       e.e[trace_msg_data.Nid_cell].i,
-                       e.e[trace_msg_data.rnti].i,
-                       e.e[trace_msg_data.rb_size].i,
-                       e.e[trace_msg_data.rb_start].i,
-                       e.e[trace_msg_data.start_symbol_index].i,
-                       e.e[trace_msg_data.nr_of_symbols].i);
-                printf("qam_mod_order %d, mcs_index %d, mcs_table %d, nrOfLayers %d, transform_precoding %d\n",
-                       e.e[trace_msg_data.qam_mod_order].i,
-                       e.e[trace_msg_data.mcs_index].i,
-                       e.e[trace_msg_data.mcs_table].i,
-                       e.e[trace_msg_data.nrOfLayers].i,
-                       e.e[trace_msg_data.transform_precoding].i);
-                printf("dmrs_config_type %d, ul_dmrs_symb_pos %d, number_dmrs_symbols %d, dmrs_port %d, dmrs_nscid %d\n",
-                       e.e[trace_msg_data.dmrs_config_type].i,
-                       e.e[trace_msg_data.ul_dmrs_symb_pos].i,
-                       e.e[trace_msg_data.number_dmrs_symbols].i,
-                       e.e[trace_msg_data.dmrs_port].i,
-                       e.e[trace_msg_data.dmrs_nscid].i);
-                printf("nb_antennas_rx %d, number_of_bits %d, data size %d\n",
-                       e.e[trace_msg_data.nb_antennas_rx].i,
-                       e.e[trace_msg_data.number_of_bits].i,
-                       e.e[trace_msg_data.data].bsize);
-#endif
+
               } // End of start recording flag
             } // end of if statement for the supported messages based on Data Collection Trace Messages Structure
             else {
@@ -891,29 +734,28 @@ int main(int n, char **v)
         } // end while loop of reading events
         else {
           // No data, just loop and check time
-          usleep(100); // optional: avoid busy-waiting
+          usleep(50); // optional: avoid busy-waiting
         }
       } // End of while loop to read events
 
-      // de-activate the tracee in the nr-softmodem
-      printf("\n De-activated Tracer message:\n");
+      // de-activate the traces in the nr-softmodem
+      printf("\n  De-activating tracer messages\n");
       for (i = 0; i < num_req_tracer_msgs; i++) {
         char *on_off_name = event_name_from_id(database, req_tracer_msgs_indices[i]);
         on_off(database, on_off_name, is_on, 0);
-        // printf("\n %d, %s, ", req_tracer_msgs_indices[i], on_off_name);
-        // on_off(database, "GNB_PHY_DL_OUTPUT_SIGNAL", is_on, 1);
       }
-      // De-activate the tracee in the nr-softmodem
       activate_traces(socket, number_of_events, is_on);
-      printf("\n De-activated Traces");
+      printf("  De-activated traces\n");
       // Get the end time
       struct timespec end_time = get_current_time();
       // Calculate the time difference
       double time_diff = calculate_time_difference(start_time, end_time);
-      printf("Total Time difference: %.2f ms\n", time_diff);
-      printf("Time difference per record: %.2f ms\n", time_diff / (number_records + max_sync_offset));
+      printf("  Total time: %.2f ms\n", time_diff);
+      printf("  Time per record: %.2f ms\n", time_diff / (number_records + max_sync_offset));
 
+      //-----------------------------
       // discard stale or previous record data for the first DISCARD_RECORD_DURATION_MS
+      //-----------------------------
       struct timespec record_start, record_now;
       clock_gettime(CLOCK_MONOTONIC, &record_start);
 
@@ -934,12 +776,11 @@ int main(int n, char **v)
           }
         } else {
           // No data, just loop and check time
-          usleep(100); // optional: avoid busy-waiting
+          usleep(50); // optional: avoid busy-waiting
         }
       } // End of while loop to discard stale records
     }
-  } // End a while loop to check for the "stop" command
-  // de-activate the tracee in the nr-softmodem
+  } // End of while loop for record/stop
 
   free_database(database);
   free(is_on);

common/utils/T/tracer/t_tracer_app_ue.c

@@ -3,7 +3,7 @@
  */
 
 /*!
- * \brief T-Tracer UE service to capture tracee Messages from UE, it is used by data recording application
+ * \brief T-Tracer UE service to capture trace Messages from UE, it is used by data recording application
  */
 
 #include <stdio.h>
@@ -32,7 +32,21 @@
 #define DEBUG_BUFFER
 // Duration to discard recording in milliseconds to mitigate stale or previous record data
 #define DISCARD_RECORD_DURATION_MS 10
-// #include <linux/time.h>
+
+// Sentinel values for field_descriptor: source is e.sending_time
+#define FIELD_SENDING_TIME_SEC  -2
+#define FIELD_SENDING_TIME_NSEC -3
+
+// State machine constants for shared memory protocol
+#define STATE_WAIT   0
+#define STATE_CONFIG 1
+#define STATE_RECORD 2
+#define STATE_STOP   3
+
+// Visual separator for state machine transitions in console output
+#define STATE_SEPARATOR "========================================"
+#define PRINT_STATE_BANNER(state_name) \
+  printf("\n%s\n  [STATE: %s]\n%s\n", STATE_SEPARATOR, state_name, STATE_SEPARATOR)
 
 // Combine bytes in little-endian format
 int combine_bytes(const uint8_t *bytes, size_t num_bytes)
@@ -44,19 +58,11 @@ int combine_bytes(const uint8_t *bytes, size_t num_bytes)
   return result;
 }
 
-// Convert an integer to an array of bytes in little-endian format
-void int_to_bytes(int num, uint8_t *bytes, size_t num_bytes)
+// Check if a message ID is present in an array of message IDs (generic array lookup)
+bool is_message_in_list(int msg_id_list[], int list_size, int msg_id)
 {
-  for (size_t i = 0; i < num_bytes; ++i) {
-    bytes[i] = (num >> (i * 8)) & 0xFF;
-  }
-}
-
-// Check if the message is in the list of bits messages
-bool is_bits_messages(int traces_bits_support_data_Collection_format_idx[], int n_bits_msgs, int msg_id)
-{
-  for (int i = 0; i < n_bits_msgs; i++) {
-    if (msg_id == traces_bits_support_data_Collection_format_idx[i]) {
+  for (int i = 0; i < list_size; i++) {
+    if (msg_id == msg_id_list[i]) {
       return true;
     }
   }
@@ -71,12 +77,6 @@ struct timespec get_current_time()
   return time;
 }
 
-// Function to convert timespec to microseconds
-long long timespec_to_microseconds(struct timespec time)
-{
-  return (time.tv_sec * 1000000LL) + (time.tv_nsec / 1000);
-}
-
 // Calculate the time difference in milliseconds
 double calculate_time_difference(struct timespec start, struct timespec end)
 {
@@ -88,7 +88,7 @@ double calculate_time_difference(struct timespec start, struct timespec end)
 // Get Time Stamp in microseconds in YYYYMMDDHHMMSSmmmuuu format
 char *get_time_stamp_usec(char time_stamp_str[])
 {
-  // initialization to measure time stamp --This part should be moved to inilization part
+  // initialization to measure time stamp
   time_t my_time;
   struct tm *timeinfo;
   time(&my_time);
@@ -104,31 +104,12 @@ char *get_time_stamp_usec(char time_stamp_str[])
   uint8_t hour = timeinfo->tm_hour;
   uint8_t min = timeinfo->tm_min;
   uint8_t sec = timeinfo->tm_sec;
-  uint16_t usec = (tv.tv_usec);
-  // printf ("Time stamp: %d_%d_%d_%d_%d_%d_%d \n",year,mon,mday,hour,min,sec,usec);
-  // sprintf(time_stamp_str, "%d_%d_%d_%d_%d_%d_%d",year,mon,mday,hour,min,sec,usec);
-  sprintf(time_stamp_str, "%04d%02d%02d%02d%02d%02d%06d", year, mon, mday, hour, min, sec, usec);
+  uint32_t usec = (tv.tv_usec);
+  sprintf(time_stamp_str, "%04d%02d%02d%02d%02d%02d%06u", year, mon, mday, hour, min, sec, usec);
 
   return time_stamp_str;
 }
 
-// Convert timestamp string to integer
-int convert_time_stamp_to_int(const char *timestamp)
-{
-  return atoi(timestamp);
-}
-
-// Split timestamp string and convert to integer
-int split_time_stamp_and_convert_to_int(char time_stamp_str[], int shift, int length)
-{
-  char time_part[length + 1]; // Buffer to hold the date part YYYYMMDD or HHMMSSmmm
-  // Copy the first 8 or 9 characters (YYYYMMDD) to HHMMSSmmm
-  strncpy(time_part, time_stamp_str + shift, length);
-  time_part[length] = '\0'; // Null-terminate the string
-  // Convert timestamp string to integer
-  return convert_time_stamp_to_int(time_part);
-}
-
 void err_exit(char *buf)
 {
   fprintf(stderr, "%s\n", buf);
@@ -153,7 +134,7 @@ int create_shm(char **addrN, const char *shm_path, int projectId)
   shm_id = shmget(key, SHMSIZE, IPC_CREAT | IPC_EXCL | 0664);
   if (shm_id == -1) {
     if (errno == EEXIST) {
-      printf("Error: shared memeory already exist\n");
+      printf("Error: shared memory already exists\n");
       shm_id = shmget(key, 0, 0);
       printf("reference shm_id = %d\n", shm_id);
     } else {
@@ -210,75 +191,28 @@ void usage(void)
   exit(1);
 }
 
-// class to store the message and the action to enable capture selected data
-// 0: do not record message
-// 1: record message
-struct trace_struct {
-  char on_off_name[100];
-  int on_off_action;
-};
-
 // struct for trace message based on Data Collection Trace Messages Structure
-// you need to define the vararibles of each message
+// you need to define the variables of each message
 typedef struct {
   /* Data Collection Trace Message Structure  */
   int frame;
   int slot;
-  int datetime_yyyymmdd;
-  int datetime_hhmmssmmm;
   int frame_type, freq_range, subcarrier_spacing, cyclic_prefix, symbols_per_slot;
   int Nid_cell, rnti;
   int rb_size, rb_start, start_symbol_index, nr_of_symbols;
   int qam_mod_order, mcs_index, mcs_table, nrOfLayers, transform_precoding;
   int dmrs_config_type, ul_dmrs_symb_pos, number_dmrs_symbols;
   int dmrs_port, dmrs_nscid, nb_antennas_tx, number_of_bits;
-  int data_size, data;
-} event_trace_msg_data;
+  int data;
+} event_trace_msg_ul_data;
 
-void setup_trace_msg_data(event_trace_msg_data *d, void *database)
+void setup_trace_msg_ul_data(event_trace_msg_ul_data *d, void *database)
 {
   database_event_format f;
   int i;
 
-  /* Data Collection Trace Message Structure */
-  // FORMAT = int,frame : int,slot : int,datetime_yyyymmdd : int,datetime_hhmmssmmm :
-  // int,frame_type : int,freq_range : int,subcarrier_spacing : int,cyclic_prefix : int,symbols_per_slot :
-  // int,Nid_cell : int,rnti :
-  // int,rb_size : int,rb_start : int,start_symbol_index : int,nr_of_symbols :
-  // int,qam_mod_order : int,mcs_index : int,mcs_table : int,nrOfLayers :
-  // int,transform_precoding : int,dmrs_config_type : int,ul_dmrs_symb_pos :  int,number_dmrs_symbols :
-  // int,dmrs_port : int,dmrs_nscid :
-  // int,nb_antennas_tx : int,number_of_bits : buffer,data
-
-  // Initialize the data structure
-  d->frame = -1;
-  d->slot = -1;
-  d->datetime_yyyymmdd = -1;
-  d->datetime_hhmmssmmm = -1;
-  d->frame_type = -1;
-  d->freq_range = -1;
-  d->subcarrier_spacing = -1;
-  d->cyclic_prefix = -1;
-  d->symbols_per_slot = -1;
-  d->Nid_cell = -1;
-  d->rnti = -1;
-  d->rb_size = -1;
-  d->rb_start = -1;
-  d->start_symbol_index = -1;
-  d->nr_of_symbols = -1;
-  d->qam_mod_order = -1;
-  d->mcs_index = -1;
-  d->mcs_table = -1;
-  d->nrOfLayers = -1;
-  d->transform_precoding = -1;
-  d->dmrs_config_type = -1;
-  d->ul_dmrs_symb_pos = -1;
-  d->number_dmrs_symbols = -1;
-  d->dmrs_port = -1;
-  d->dmrs_nscid = -1;
-  d->nb_antennas_tx = -1;
-  d->number_of_bits = -1;
-  d->data = -1;
+  // Initialize all fields to -1 (marks unset indices)
+  memset(d, -1, sizeof(*d));
 
 /* this macro looks for a particular element and checks its type */
 #define G(var_name, var_type, var)                                      \
@@ -291,35 +225,14 @@ void setup_trace_msg_data(event_trace_msg_data *d, void *database)
     continue;                                                           \
   }
 
-  /* ------------------------------*/
-  /* Create Macro for Data Collection Trace Message */
-  /* ------------------------------*/
-  // Data Collection Trace Message Structure
-  // Example: GNB_PHY_UL_FD_PUSCH_IQ,  GNB_PHY_UL_FD_DMRS_ID,
-  //          GNB_PHY_UL_FD_CHAN_EST_DMRS_POS, GNB_PHY_UL_FD_CHAN_EST_DMRS_INTERPL
-  //          GNB_PHY_UL_PAYLOAD_RX_BITS
-  //          UE_PHY_UL_SCRAMBLED_TX_BITS
-  //          UE_PHY_UL_PAYLOAD_TX_BITS
-  // Get a template of any message based on Data Collection Trace Messages Structure
+  // Get a template of any UL message based on Data Collection Trace Messages Structure
   int Trace_MSG_ID = event_id_from_name(database, "GNB_PHY_UL_FD_PUSCH_IQ");
   f = get_format(database, Trace_MSG_ID);
 
-  /* get the elements of the trace
-   * the value is an index in the event, see below */
-  // FORMAT = int,frame : int,slot : int,datetime_yyyymmdd : int,datetime_hhmmssmmm :
-  // int,frame_type : int,freq_range : int,subcarrier_spacing : int,cyclic_prefix : int,symbols_per_slot :
-  // int,Nid_cell : int,rnti :
-  // int,rb_size : int,rb_start : int,start_symbol_index : int,nr_of_symbols :
-  // int,qam_mod_order : int,mcs_index : int,mcs_table : int,nrOfLayers :
-  // int,transform_precoding : int,dmrs_config_type : int,ul_dmrs_symb_pos :  int,number_dmrs_symbols :
-  // int,dmrs_port : int,dmrs_nscid :
-  // int,nb_antennas_tx : int,number_of_bits : buffer,data
-
+  // Map each field name to its index in the event structure
   for (i = 0; i < f.count; i++) {
     G("frame", "int", d->frame)
     G("slot", "int", d->slot)
-    G("datetime_yyyymmdd", "int", d->datetime_yyyymmdd)
-    G("datetime_hhmmssmmm", "int", d->datetime_hhmmssmmm)
     G("frame_type", "int", d->frame_type)
     G("freq_range", "int", d->freq_range)
     G("subcarrier_spacing", "int", d->subcarrier_spacing)
@@ -347,20 +260,108 @@ void setup_trace_msg_data(event_trace_msg_data *d, void *database)
     G("number_of_bits", "int", d->number_of_bits)
     G("data", "buffer", d->data)
   }
-  // if (d->frame == -1 || d->slot == -1) goto error;
 #undef G
-  return;
 }
 
-// Function to check if a value is in the array
-int isValueInArray(int value, int arr[], int size)
+// -------------------------------------------------------------------
+// Table-driven UL metadata shared memory writer and debug printer
+// -------------------------------------------------------------------
+
+// Number of UL metadata fields written to shared memory (excludes msg_id and buffer)
+#define N_UL_METADATA_FIELDS 27
+
+// Descriptor for one metadata field to write to shared memory
+typedef struct {
+  int field_idx;     // index into event e.e[] array
+  size_t wire_size;  // number of bytes to write to shm
+} field_descriptor;
+
+// Build UL field descriptor array from a populated event_trace_msg_ul_data struct.
+// Must be called after setup_trace_msg_ul_data() so that field indices are populated.
+// Note: nb_antennas_tx maps to database field "nb_antennas_rx" (see G macro workaround above)
+void build_ul_field_descriptors(field_descriptor *fields, const event_trace_msg_ul_data *d)
 {
-  for (int i = 0; i < size; i++) {
-    if (arr[i] == value) {
-      return 1; // Value found
+  int i = 0;
+  fields[i++] = (field_descriptor){ d->frame, sizeof(uint16_t) };
+  fields[i++] = (field_descriptor){ d->slot, sizeof(uint8_t) };
+  fields[i++] = (field_descriptor){ FIELD_SENDING_TIME_SEC, sizeof(uint32_t) };
+  fields[i++] = (field_descriptor){ FIELD_SENDING_TIME_NSEC, sizeof(uint32_t) };
+  fields[i++] = (field_descriptor){ d->frame_type, sizeof(uint8_t) };
+  fields[i++] = (field_descriptor){ d->freq_range, sizeof(uint8_t) };
+  fields[i++] = (field_descriptor){ d->subcarrier_spacing, sizeof(uint8_t) };
+  fields[i++] = (field_descriptor){ d->cyclic_prefix, sizeof(uint8_t) };
+  fields[i++] = (field_descriptor){ d->symbols_per_slot, sizeof(uint8_t) };
+  fields[i++] = (field_descriptor){ d->Nid_cell, sizeof(uint16_t) };
+  fields[i++] = (field_descriptor){ d->rnti, sizeof(uint16_t) };
+  fields[i++] = (field_descriptor){ d->rb_size, sizeof(uint16_t) };
+  fields[i++] = (field_descriptor){ d->rb_start, sizeof(uint16_t) };
+  fields[i++] = (field_descriptor){ d->start_symbol_index, sizeof(uint8_t) };
+  fields[i++] = (field_descriptor){ d->nr_of_symbols, sizeof(uint8_t) };
+  fields[i++] = (field_descriptor){ d->qam_mod_order, sizeof(uint8_t) };
+  fields[i++] = (field_descriptor){ d->mcs_index, sizeof(uint8_t) };
+  fields[i++] = (field_descriptor){ d->mcs_table, sizeof(uint8_t) };
+  fields[i++] = (field_descriptor){ d->nrOfLayers, sizeof(uint8_t) };
+  fields[i++] = (field_descriptor){ d->transform_precoding, sizeof(uint8_t) };
+  fields[i++] = (field_descriptor){ d->dmrs_config_type, sizeof(uint8_t) };
+  fields[i++] = (field_descriptor){ d->ul_dmrs_symb_pos, sizeof(uint16_t) };
+  fields[i++] = (field_descriptor){ d->number_dmrs_symbols, sizeof(uint8_t) };
+  fields[i++] = (field_descriptor){ d->dmrs_port, sizeof(uint16_t) };
+  fields[i++] = (field_descriptor){ d->dmrs_nscid, sizeof(uint16_t) };
+  fields[i++] = (field_descriptor){ d->nb_antennas_tx, sizeof(uint8_t) };
+  fields[i++] = (field_descriptor){ d->number_of_bits, sizeof(uint32_t) };
+}
+
+// Print UL metadata fields for debug output
+// Note: nb_antennas_tx is the UE transmit antenna count (maps to database field "nb_antennas_rx")
+void print_ul_metadata_debug(event e, const event_trace_msg_ul_data *d, void *database)
+{
+  printf("\nUnix TS: %ld.%09ld", e.sending_time.tv_sec, e.sending_time.tv_nsec);
+  printf("\nMessage Info: msg_id %s (%d) \n", event_name_from_id(database, e.type), e.type);
+  printf("frame %d, slot %d, unix_ts %ld.%09ld\n",
+         e.e[d->frame].i, e.e[d->slot].i,
+         e.sending_time.tv_sec, e.sending_time.tv_nsec);
+  printf("frame_type %d, freq_range %d, subcarrier_spacing %d, cyclic_prefix %d, symbols_per_slot %d\n",
+         e.e[d->frame_type].i, e.e[d->freq_range].i,
+         e.e[d->subcarrier_spacing].i, e.e[d->cyclic_prefix].i,
+         e.e[d->symbols_per_slot].i);
+  printf("Nid_cell %d, rnti %d, rb_size %d, rb_start %d, start_symbol_index %d, nr_of_symbols %d\n",
+         e.e[d->Nid_cell].i, e.e[d->rnti].i,
+         e.e[d->rb_size].i, e.e[d->rb_start].i,
+         e.e[d->start_symbol_index].i, e.e[d->nr_of_symbols].i);
+  printf("qam_mod_order %d, mcs_index %d, mcs_table %d, nrOfLayers %d, transform_precoding %d\n",
+         e.e[d->qam_mod_order].i, e.e[d->mcs_index].i,
+         e.e[d->mcs_table].i, e.e[d->nrOfLayers].i,
+         e.e[d->transform_precoding].i);
+  printf("dmrs_config_type %d, ul_dmrs_symb_pos %d, number_dmrs_symbols %d, dmrs_port %d, dmrs_nscid %d\n",
+         e.e[d->dmrs_config_type].i, e.e[d->ul_dmrs_symb_pos].i,
+         e.e[d->number_dmrs_symbols].i, e.e[d->dmrs_port].i,
+         e.e[d->dmrs_nscid].i);
+  printf("nb_antennas_tx %d, number_of_bits %d, data size %d\n",
+         e.e[d->nb_antennas_tx].i, e.e[d->number_of_bits].i,
+         e.e[d->data].bsize);
+}
+
+// Write metadata fields to shared memory using field descriptor table.
+// Writes msg_id first, then loops through all fields.
+void write_metadata_to_shm(char *addr_wr, unsigned int *bufIdx_wr, event e,
+                           const field_descriptor *fields, int n_fields)
+{
+  // Write message ID
+  memcpy(&addr_wr[*bufIdx_wr], &e.type, sizeof(uint16_t));
+  *bufIdx_wr += sizeof(uint16_t);
+  // Write all metadata fields
+  for (int i = 0; i < n_fields; i++) {
+    if (fields[i].field_idx == FIELD_SENDING_TIME_SEC) {
+      uint32_t ts_sec = (uint32_t)e.sending_time.tv_sec;
+      memcpy(&addr_wr[*bufIdx_wr], &ts_sec, fields[i].wire_size);
+    } else if (fields[i].field_idx == FIELD_SENDING_TIME_NSEC) {
+      uint32_t ts_nsec = (uint32_t)e.sending_time.tv_nsec;
+      memcpy(&addr_wr[*bufIdx_wr], &ts_nsec, fields[i].wire_size);
+    } else {
+      memcpy(&addr_wr[*bufIdx_wr], &e.e[fields[i].field_idx].i, fields[i].wire_size);
     }
+    *bufIdx_wr += fields[i].wire_size;
   }
-  return 0; // Value not found
 }
 
 void reestablish_connection(int *socket, char *ip, int port, int number_of_events, int *is_on)
@@ -388,40 +389,24 @@ int main(int n, char **v)
   // trace_msgs_support_data_Collection_format
   // it is used to parse the requested messages if it is based
   // on Data Collection Trace Messages Structure and supported tracer messages indices
-  char *traces_iq_support_data_Collection_format[] = {"GNB_PHY_UL_FD_PUSCH_IQ",
-                                                            "GNB_PHY_UL_FD_DMRS",
-                                                            "GNB_PHY_UL_FD_CHAN_EST_DMRS_POS",
-                                                            "GNB_PHY_UL_FD_CHAN_EST_DMRS_INTERPL"};
-  char *traces_bits_support_data_Collection_format[] = {"GNB_PHY_UL_PAYLOAD_RX_BITS",
-                                                              "UE_PHY_UL_SCRAMBLED_TX_BITS",
-                                                              "UE_PHY_UL_PAYLOAD_TX_BITS"};
-  // extra number of records to simlify Sync between base station and UE synchronization records.
+  char *traces_ul_support_data_Collection_format[] = {"GNB_PHY_UL_FD_PUSCH_IQ",
+                                                      "GNB_PHY_UL_FD_DMRS",
+                                                      "GNB_PHY_UL_FD_CHAN_EST_DMRS_POS",
+                                                      "GNB_PHY_UL_FD_CHAN_EST_DMRS_INTERPL",
+                                                      "GNB_PHY_UL_PAYLOAD_RX_BITS",
+                                                      "UE_PHY_UL_SCRAMBLED_TX_BITS",
+                                                      "UE_PHY_UL_PAYLOAD_TX_BITS"};
+  // extra number of records to simplify sync between base station and UE synchronization records.
   // if you have network delay, you can increase the number of records to capture
   int max_sync_offset = 6; // 6 frames ~ 60 ms
-  // all supported messages
-  // Calculate the size of the combined array
-  int n_iq_msgs = sizeof(traces_iq_support_data_Collection_format) / sizeof(traces_iq_support_data_Collection_format[0]);
-  int n_bits_msgs = sizeof(traces_bits_support_data_Collection_format) / sizeof(traces_bits_support_data_Collection_format[0]);
-  int n_msgs_based_data_Collection_format = n_iq_msgs + n_bits_msgs;
-
-  // Create the combined array
-  char *traces_support_data_Collection_format[n_msgs_based_data_Collection_format];
-  // Copy IQ messages to the combined array
-  for (int i = 0; i < n_iq_msgs; i++) {
-    traces_support_data_Collection_format[i] = traces_iq_support_data_Collection_format[i];
-  }
-
-  // Copy Bits messages to the combined array
-  for (int i = 0; i < n_bits_msgs; i++) {
-    traces_support_data_Collection_format[i + n_iq_msgs] = traces_bits_support_data_Collection_format[i];
-  }
+  // Calculate the size of the UL message array
+  int n_ul_msgs = sizeof(traces_ul_support_data_Collection_format) / sizeof(traces_ul_support_data_Collection_format[0]);
 
   uint16_t msg_id = 0;
   uint16_t start_frame_number = 0;
   uint32_t number_records = 0; // number of records to capture, it is number of slots
   // array to store the requested tracer messages indices
   int req_tracer_msgs_indices[100] = {0};
-  // define variables --> to do: add all of them to be class of pointers
   char *database_filename = NULL;
   void *database;
   char *ip = DEFAULT_REMOTE_IP;
@@ -434,18 +419,19 @@ int main(int n, char **v)
   char trace_time_stamp_str[30];
 
   // data structure for the trace messages based on Data Collection Trace Messages Structure
-  event_trace_msg_data trace_msg_data;
+  event_trace_msg_ul_data trace_msg_ul_data;
 
-  // initlization variables
+  // initialization variables
   unsigned int bufIdx_wr = 0;
   unsigned int bufIdx_rd = 0;
   uint8_t num_req_tracer_msgs = 0;
 
-  // initilaze shared memory
+  // initialize shared memory
   char *addr_wr, *addr_rd;
-  printf("\n Data Collection Service: Initializing shared memory ...");
-  printf("\n Directory 1: %s, Directory 2: %s", GETKEYDIR1_UE, GETKEYDIR2_UE);
-  printf("\n Project ID: %d\n", PROJECTID_UE);
+  PRINT_STATE_BANNER("INIT");
+  printf("  Data Collection Service: Initializing shared memory ...\n");
+  printf("  Directory 1: %s, Directory 2: %s\n", GETKEYDIR1_UE, GETKEYDIR2_UE);
+  printf("  Project ID: %d\n", PROJECTID_UE);
   int shm_id_wr = create_shm(&addr_wr, GETKEYDIR1_UE, PROJECTID_UE);
   int shm_id_rd = create_shm(&addr_rd, GETKEYDIR2_UE, PROJECTID_UE);
   del_shm(addr_wr, shm_id_wr);
@@ -502,20 +488,18 @@ int main(int n, char **v)
   addr_rd[0] = 0; // important to check if we have new requested messages
 
   // read requested tracer msg indices from memory
-  printf("\n Data Collection Service: Waiting for messages request ...\n");
-  // 0: Wait
-  // 1: config
-  // 2: record
-  // 3: stop
+  PRINT_STATE_BANNER("WAIT");
+  printf("  Data Collection Service: Waiting for config message ...\n");
   // Wait for Action Config
   while (1) {
-    if ((uint8_t)(addr_rd[0]) == 0) {
+    if ((uint8_t)(addr_rd[0]) == STATE_WAIT) {
       usleep(50); // sleep for 50 us: 0.5 ms slot duration
     }
     // config state
-    else if ((uint8_t)(addr_rd[0]) == 1) {
+    else if ((uint8_t)(addr_rd[0]) == STATE_CONFIG) {
       get_time_stamp_usec(trace_time_stamp_str);
-      printf("\n Received config message. Time Stamp: %s", trace_time_stamp_str);
+      PRINT_STATE_BANNER("CONFIG");
+      printf("  Received config message. Time Stamp: %s\n", trace_time_stamp_str);
 
       // get the IP address length in bytes
       bufIdx_rd = 1;
@@ -532,11 +516,11 @@ int main(int n, char **v)
 
       // get the array bytes of the port number : 2 bytes
       uint8_t port_number_bytes[2] = {addr_rd[bufIdx_rd], addr_rd[bufIdx_rd + 1]};
-      bufIdx_rd += 2; // + 2 bytes = start frame number
+      bufIdx_rd += 2; // + 2 bytes = port number
 
       port = combine_bytes(port_number_bytes, 2);
-      printf("\n Parameters: IP Address length: %d, IP Address: %s, Port Number: %d \n", ip_address_length, ip_address, port);
-      addr_rd[0] = 0; // reset memory : to wait for record action
+      printf("  Parameters: IP Address length: %d, IP Address: %s, Port Number: %d\n", ip_address_length, ip_address, port);
+      addr_rd[0] = STATE_WAIT; // reset memory : to wait for record action
       break;
     }
   }
@@ -547,37 +531,39 @@ int main(int n, char **v)
 
   /* connect to the nr-softmodem */
   socket = connect_to(ip, port);
-  printf("\n Connected to nr-UEsoftmodem");
+  printf("  Connected to nr-uesoftmodem\n");
 
   // Read Action record or stop
-  printf("\n Data Collection Service: Waiting for record message ...\n");
+  PRINT_STATE_BANNER("WAIT");
+  printf("  Data Collection Service: Waiting for record/stop message ...\n");
   while (1) {
     // wait for Action record
-    if ((uint8_t)(addr_rd[0]) == 0) {
+    if ((uint8_t)(addr_rd[0]) == STATE_WAIT) {
       usleep(50); // sleep for 50 us: 0.5 ms slot duration
     }
 
     // quit state
-    else if ((uint8_t)(addr_rd[0]) == 3) {
-      printf("\n Received 'stop' command. Exiting...");
-      printf("\n ");
+    else if ((uint8_t)(addr_rd[0]) == STATE_STOP) {
+      PRINT_STATE_BANNER("STOP");
+      printf("  Received 'stop' command. Exiting...\n");
       // Clean up and exit
       break;
     }
     // record state
-    else if ((uint8_t)(addr_rd[0]) == 2) {
+    else if ((uint8_t)(addr_rd[0]) == STATE_RECORD) {
       // clear remote buffer if there is
       clear_remote_config();
       get_time_stamp_usec(trace_time_stamp_str);
-      printf("\n Received record message. Time Stamp: %s", trace_time_stamp_str);
+      PRINT_STATE_BANNER("RECORD");
+      printf("  Received record message. Time Stamp: %s\n", trace_time_stamp_str);
 
       // read number of requested messages
       bufIdx_rd = 1;
       num_req_tracer_msgs = addr_rd[bufIdx_rd];
       bufIdx_rd += 1;
-      printf("\n Number of requested tracer messages: %d,", num_req_tracer_msgs);
+      printf("  Number of requested tracer messages: %d\n", num_req_tracer_msgs);
       // reset memory : action to wait for next record action
-      addr_rd[0] = 0;
+      addr_rd[0] = STATE_WAIT;
 
       // read tracer msg IDs - every message ID has been stored in two bytes
       for (uint8_t msg_n = 0; msg_n < num_req_tracer_msgs; msg_n++) {
@@ -609,40 +595,36 @@ int main(int n, char **v)
       printf("start_frame: %d\n", start_frame_number);
 
       /* activate the trace in this array */
-      printf("\n Activate Tracer messages in on_off array: \n");
+      printf("  Activating tracer messages:\n");
       for (i = 0; i < num_req_tracer_msgs; i++) {
         char *on_off_name = event_name_from_id(database, req_tracer_msgs_indices[i]);
         on_off(database, on_off_name, is_on, 1);
-        printf("%d, %s, ", req_tracer_msgs_indices[i], on_off_name);
-        // on_off(database, "GNB_PHY_DL_OUTPUT_SIGNAL", is_on, 1);
+        printf("    %d: %s\n", req_tracer_msgs_indices[i], on_off_name);
       }
 
-      // get the event IDs for the bit messages
-      int traces_bits_support_data_Collection_format_idx[n_bits_msgs];
-      for (int i = 0; i < n_bits_msgs; i++) {
-        traces_bits_support_data_Collection_format_idx[i] =
-            event_id_from_name(database, traces_bits_support_data_Collection_format[i]);
-      }
-
-      // all supported messages
-      int traces_support_data_Collection_format_idx[n_msgs_based_data_Collection_format];
-      for (int i = 0; i < n_msgs_based_data_Collection_format; i++) {
-        traces_support_data_Collection_format_idx[i] = event_id_from_name(database, traces_support_data_Collection_format[i]);
+      // Build UL message ID array for generic UL dispatch
+      int ul_msg_ids[n_ul_msgs];
+      for (int i = 0; i < n_ul_msgs; i++) {
+        ul_msg_ids[i] = event_id_from_name(database, traces_ul_support_data_Collection_format[i]);
       }
 
       // setup data for the trace messages
-      setup_trace_msg_data(&trace_msg_data, database);
-      printf("\n Setup Trace Data Done");
+      setup_trace_msg_ul_data(&trace_msg_ul_data, database);
+      printf("  Setup UL trace data done\n");
+
+      // Build field descriptor table for table-driven shared memory writes
+      field_descriptor ul_fields[N_UL_METADATA_FIELDS];
+      build_ul_field_descriptors(ul_fields, &trace_msg_ul_data);
 
       // Get the start time
       struct timespec start_time = get_current_time();
 
-      /* activate the tracee in the nr-softmodem */
+      /* activate the traces in the nr-uesoftmodem */
       activate_traces(socket, number_of_events, is_on);
-      printf("\n Activated Traces in nr-UEsoftmodem");
+      printf("  Activated traces in nr-uesoftmodem\n");
 
       /* a buffer needed to receive events from the nr-softmodem */
-      OBUF ebuf = {osize : 0, omaxsize : 0, obuf : NULL};
+      OBUF ebuf = {.osize = 0, .omaxsize = 0, .obuf = NULL};
       /* read events */
       int nrecord_idx = 0;
       bufIdx_wr = 0;
@@ -656,8 +638,8 @@ int main(int n, char **v)
       int current_frame = 0, prev_frame = 0, current_slot = 0, prev_slot = 0;
       // offset to sync between base station and UE synchronization records or power measurements
       int sync_offset_index = 0; // increase the index only if the index of frame changes after getting all records
-      // since we will use the frame differece to do extra records, we should be sure that the last slot is recorded completely
-      printf("\n\n Data Collection Service: Start reading messages ...");
+      // since we will use the frame difference to do extra records, we should be sure that the last slot is recorded completely
+      printf("\n  Data Collection Service: Start reading messages ...\n");
       struct pollfd event_poll_fd;
       event_poll_fd.fd = socket;
       event_poll_fd.events = POLLIN;
@@ -677,27 +659,23 @@ int main(int n, char **v)
             continue; // Skip further processing and retry
           }
 
-          //-------------------------
-          // GNB_PHY_UL_FD_PUSCH_IQ,  GNB_PHY_UL_FD_DMRS_ID, GNB_PHY_UL_FD_CHAN_EST_DMRS_POS,
-          // UE_PHY_UL_SCRAMBLED_TX_BITS, GNB_PHY_UL_PAYLOAD_RX_BITS, UE_PHY_UL_PAYLOAD_TX_BITS
-          //-------------------------
           // is it a requested message
-          if (isValueInArray(e.type, req_tracer_msgs_indices, num_req_tracer_msgs)) {
+          if (is_message_in_list(req_tracer_msgs_indices, num_req_tracer_msgs, e.type)) {
             // is it based on Data Collection Trace Messages Structure
-            if (isValueInArray(e.type, traces_support_data_Collection_format_idx, n_msgs_based_data_Collection_format)) {
+            if (is_message_in_list(ul_msg_ids, n_ul_msgs, e.type)) {
               // Start recording from the next slot to mitigate capturing partial data
               // check if the current frame and slot are different from the previous frame and slot
               // Then, increase the record index
               if (start_recording == false) {
                 if (got_ref_frame_slot == false) {
-                  ref_frame = e.e[trace_msg_data.frame].i;
-                  ref_slot = e.e[trace_msg_data.slot].i;
+                  ref_frame = e.e[trace_msg_ul_data.frame].i;
+                  ref_slot = e.e[trace_msg_ul_data.slot].i;
                   printf("\nMessage Info: msg_id %s (%d) \n", event_name_from_id(database, e.type), e.type);
                   got_ref_frame_slot = true;
                 }
 
-                current_frame = e.e[trace_msg_data.frame].i;
-                current_slot = e.e[trace_msg_data.slot].i;
+                current_frame = e.e[trace_msg_ul_data.frame].i;
+                current_slot = e.e[trace_msg_ul_data.slot].i;
                 frame_difference = (current_frame - ref_frame + MAX_FRAME_INDEX + 1) % (MAX_FRAME_INDEX + 1);
                 slot_difference = (current_slot - ref_slot + MAX_SLOT_INDEX + 1) % (MAX_SLOT_INDEX + 1);
                 printf("\n First frame.slot: %d.%d, current frame.slot: %d.%d, diff frame.slot: %d.%d",
@@ -710,130 +688,30 @@ int main(int n, char **v)
 
                 if ((ref_frame != current_frame) || (ref_slot != current_slot)) {
                   start_recording = true;
-                  printf("\n Start recording from frame: %d, slot: %d ", e.e[trace_msg_data.frame].i, e.e[trace_msg_data.slot].i);
+                  printf("\n Start recording from frame: %d, slot: %d ", e.e[trace_msg_ul_data.frame].i, e.e[trace_msg_ul_data.slot].i);
                 }
               }
               // start recording from the next frame to mitigate capturing partial data
               if (start_recording == true) {
-                /* this is how to access the elements of the Data Collection trace messages.
-                 * we use e.e[<element>] and then the correct suffix, here
-                 * it's .i for the integer and .b for the buffer and .bsize for the buffer size
-                 * see in event.h the structure event_arg
-                 */
-                unsigned char *buf = e.e[trace_msg_data.data].b;
-
                 printf("\n\nRecord number: %d", nrecord_idx);
 #ifdef DEBUG_BUFFER
                 printf("\nBuffer index in bytes: %d", bufIdx_wr);
 #endif
-
-                // add general message header:  message ID,
-                // T-Tracer Message format
-                // FORMAT = int,frame : int,slot : int,datetime_yyyymmdd : int,datetime_hhmmssmmm :
-                // int,frame_type : int,freq_range : int,subcarrier_spacing : int,cyclic_prefix : int,symbols_per_slot :
-                // int,Nid_cell : int,rnti :
-                // int,rb_size : int,rb_start : int,start_symbol_index : int,nr_of_symbols :
-                // int,qam_mod_order : int,mcs_index : int,mcs_table : int,nrOfLayers :
-                // int,transform_precoding : int,dmrs_config_type : int,ul_dmrs_symb_pos :  int,number_dmrs_symbols :
-                // int,dmrs_port : int,dmrs_nscid :
-                // int,nb_antennas_tx : int,number_of_bits : buffer,data
-                // printf("\nTX : %d", e.e[trace_msg_data.nb_antennas_tx].i);
-                memcpy(&addr_wr[bufIdx_wr], &e.type, sizeof(uint16_t));
-                bufIdx_wr += sizeof(uint16_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.frame].i, sizeof(uint16_t));
-                bufIdx_wr += sizeof(uint16_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.slot].i, sizeof(uint8_t));
-                bufIdx_wr += sizeof(uint8_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.datetime_yyyymmdd].i, sizeof(uint32_t));
+                // --- UL IQ/Bits handler ---
+                // Write metadata + buffer to shared memory
+                write_metadata_to_shm(addr_wr, &bufIdx_wr, e, ul_fields, N_UL_METADATA_FIELDS);
+                // Write buffer: size (uint32_t) + raw data
+                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_ul_data.data].bsize, sizeof(uint32_t));
                 bufIdx_wr += sizeof(uint32_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.datetime_hhmmssmmm].i, sizeof(uint32_t));
-                bufIdx_wr += sizeof(uint32_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.frame_type].i, sizeof(uint8_t));
-                bufIdx_wr += sizeof(uint8_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.freq_range].i, sizeof(uint8_t));
-                bufIdx_wr += sizeof(uint8_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.subcarrier_spacing].i, sizeof(uint8_t));
-                bufIdx_wr += sizeof(uint8_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.cyclic_prefix].i, sizeof(uint8_t));
-                bufIdx_wr += sizeof(uint8_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.symbols_per_slot].i, sizeof(uint8_t));
-                bufIdx_wr += sizeof(uint8_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.Nid_cell].i, sizeof(uint16_t));
-                bufIdx_wr += sizeof(uint16_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.rnti].i, sizeof(uint16_t));
-                bufIdx_wr += sizeof(uint16_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.rb_size].i, sizeof(uint16_t));
-                bufIdx_wr += sizeof(uint16_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.rb_start].i, sizeof(uint16_t));
-                bufIdx_wr += sizeof(uint16_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.start_symbol_index].i, sizeof(uint8_t));
-                bufIdx_wr += sizeof(uint8_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.nr_of_symbols].i, sizeof(uint8_t));
-                bufIdx_wr += sizeof(uint8_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.qam_mod_order].i, sizeof(uint8_t));
-                bufIdx_wr += sizeof(uint8_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.mcs_index].i, sizeof(uint8_t));
-                bufIdx_wr += sizeof(uint8_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.mcs_table].i, sizeof(uint8_t));
-                bufIdx_wr += sizeof(uint8_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.nrOfLayers].i, sizeof(uint8_t));
-                bufIdx_wr += sizeof(uint8_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.transform_precoding].i, sizeof(uint8_t));
-                bufIdx_wr += sizeof(uint8_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.dmrs_config_type].i, sizeof(uint8_t));
-                bufIdx_wr += sizeof(uint8_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.ul_dmrs_symb_pos].i, sizeof(uint16_t));
-                bufIdx_wr += sizeof(uint16_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.number_dmrs_symbols].i, sizeof(uint8_t));
-                bufIdx_wr += sizeof(uint8_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.dmrs_port].i, sizeof(uint16_t));
-                bufIdx_wr += sizeof(uint16_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.dmrs_nscid].i, sizeof(uint16_t));
-                bufIdx_wr += sizeof(uint16_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.nb_antennas_tx].i, sizeof(uint8_t));
-                bufIdx_wr += sizeof(uint8_t);
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.number_of_bits].i, sizeof(uint32_t));
-                bufIdx_wr += sizeof(uint32_t);
-
-                printf("\nTime Stamp: %d_%d", e.e[trace_msg_data.datetime_yyyymmdd].i, e.e[trace_msg_data.datetime_hhmmssmmm].i);
-
-                // add message body: length in bytes + recorded data
-                memcpy(&addr_wr[bufIdx_wr], &e.e[trace_msg_data.data].bsize, sizeof(uint32_t));
-                bufIdx_wr += sizeof(uint32_t);
-
-                // read data from buffer and convert from unsigned char * array to int 16 using the right endianness
-                // T-tracer: Little Endian
-                // For BITS Messages, example: UE_PHY_UL_SCRAMBLED_TX_BITS: data in bytes
-
-                if (is_bits_messages(traces_bits_support_data_Collection_format_idx, n_bits_msgs, e.type)) {
-                  for (int byte_idx = 0; byte_idx < e.e[trace_msg_data.data].bsize; byte_idx += 1) {
-                    // printf("%d, ", buf[byte_idx]);
-                    memcpy(&addr_wr[bufIdx_wr], &buf[byte_idx], sizeof(uint8_t));
-                    bufIdx_wr += sizeof(uint8_t);
-                  }
-
-                } else {
-                  for (int byte_idx = 0; byte_idx < e.e[trace_msg_data.data].bsize; byte_idx += 2) {
-                    // For a little-endian system:
-                    memcpy(&addr_wr[bufIdx_wr], &buf[byte_idx], sizeof(uint8_t));
-                    bufIdx_wr += sizeof(uint8_t);
-                    memcpy(&addr_wr[bufIdx_wr], &buf[byte_idx + 1], sizeof(uint8_t));
-                    bufIdx_wr += sizeof(uint8_t);
-                  }
-                }
-                /*
-                for (int byte_idx_i = 0; byte_idx_i < e.e[d.nr_ul_fd_dmrs_data].bsize; byte_idx_i+=4) {
-                  int16_t I = buf[byte_idx_i] | (buf[byte_idx_i+1] << 8);
-                  int16_t Q = buf[byte_idx_i+2] | (buf[byte_idx_i+3] << 8);
-                  printf ("idx %d, ", byte_idx_i);
-                  printf ("\n%d", I);
-                  printf ("\n%d", Q);
-                }
-                */
+                memcpy(&addr_wr[bufIdx_wr], e.e[trace_msg_ul_data.data].b, e.e[trace_msg_ul_data.data].bsize);
+                bufIdx_wr += e.e[trace_msg_ul_data.data].bsize;
+#ifdef DEBUG_T_Tracer
+                print_ul_metadata_debug(e, &trace_msg_ul_data, database);
+#endif
                 // check if the current frame and slot are different from the previous frame and slot
                 // Then, increase the record index
-                current_frame = e.e[trace_msg_data.frame].i;
-                current_slot = e.e[trace_msg_data.slot].i;
+                current_frame = e.e[trace_msg_ul_data.frame].i;
+                current_slot = e.e[trace_msg_ul_data.slot].i;
 
                 // increase sync offset index if the current frame is different from the previous frame
                 if ((current_frame != prev_frame) && nrecord_idx >= number_records) {
@@ -846,43 +724,6 @@ int main(int n, char **v)
                   prev_frame = current_frame;
                   prev_slot = current_slot;
                 }
-#ifdef DEBUG_T_Tracer
-                printf("\nMessage Info: msg_id %s (%d) \n", event_name_from_id(database, e.type), e.type);
-                printf("frame %d, slot %d, datetime %d_%d\n",
-                       e.e[trace_msg_data.frame].i,
-                       e.e[trace_msg_data.slot].i,
-                       e.e[trace_msg_data.datetime_yyyymmdd].i,
-                       e.e[trace_msg_data.datetime_hhmmssmmm].i);
-                printf("frame_type %d, freq_range %d, subcarrier_spacing %d, cyclic_prefix %d, symbols_per_slot %d\n",
-                       e.e[trace_msg_data.frame_type].i,
-                       e.e[trace_msg_data.freq_range].i,
-                       e.e[trace_msg_data.subcarrier_spacing].i,
-                       e.e[trace_msg_data.cyclic_prefix].i,
-                       e.e[trace_msg_data.symbols_per_slot].i);
-                printf("Nid_cell %d, rnti %d, rb_size %d, rb_start %d, start_symbol_index %d, nr_of_symbols %d\n",
-                       e.e[trace_msg_data.Nid_cell].i,
-                       e.e[trace_msg_data.rnti].i,
-                       e.e[trace_msg_data.rb_size].i,
-                       e.e[trace_msg_data.rb_start].i,
-                       e.e[trace_msg_data.start_symbol_index].i,
-                       e.e[trace_msg_data.nr_of_symbols].i);
-                printf("qam_mod_order %d, mcs_index %d, mcs_table %d, nrOfLayers %d, transform_precoding %d\n",
-                       e.e[trace_msg_data.qam_mod_order].i,
-                       e.e[trace_msg_data.mcs_index].i,
-                       e.e[trace_msg_data.mcs_table].i,
-                       e.e[trace_msg_data.nrOfLayers].i,
-                       e.e[trace_msg_data.transform_precoding].i);
-                printf("dmrs_config_type %d, ul_dmrs_symb_pos %d, number_dmrs_symbols %d, dmrs_port %d, dmrs_nscid %d\n",
-                       e.e[trace_msg_data.dmrs_config_type].i,
-                       e.e[trace_msg_data.ul_dmrs_symb_pos].i,
-                       e.e[trace_msg_data.number_dmrs_symbols].i,
-                       e.e[trace_msg_data.dmrs_port].i,
-                       e.e[trace_msg_data.dmrs_nscid].i);
-                printf("nb_antennas_tx %d, number_of_bits %d, data size %d\n",
-                       e.e[trace_msg_data.nb_antennas_tx].i,
-                       e.e[trace_msg_data.number_of_bits].i,
-                       e.e[trace_msg_data.data].bsize);
-#endif
               } // End of start recording flag
             } // end of if statement for the supported messages based on Data Collection Trace Messages Structure
             else {
@@ -893,28 +734,25 @@ int main(int n, char **v)
         } // end while loop of reading events
         else {
           // No data, just loop and check time
-          usleep(100); // optional: avoid busy-waiting
+          usleep(50); // optional: avoid busy-waiting
         }
       } // End of while loop to read messages
 
-      // de-activate the tracee in the nr-softmodem
-      printf("\n De-activated Tracer message:\n");
+      // de-activate the traces in the nr-uesoftmodem
+      printf("\n  De-activating tracer messages\n");
       for (i = 0; i < num_req_tracer_msgs; i++) {
         char *on_off_name = event_name_from_id(database, req_tracer_msgs_indices[i]);
         on_off(database, on_off_name, is_on, 0);
-        // printf("\n %d, %s, ", req_tracer_msgs_indices[i], on_off_name);
-        // on_off(database, "GNB_PHY_DL_OUTPUT_SIGNAL", is_on, 1);
       }
-      // De-activate the tracee in the nr-softmodem
       activate_traces(socket, number_of_events, is_on);
-      printf("\n De-activated Traces");
+      printf("  De-activated traces\n");
 
       // Get the end time
       struct timespec end_time = get_current_time();
       // Calculate the time difference
       double time_diff = calculate_time_difference(start_time, end_time);
-      printf("Total Time difference: %.2f ms\n", time_diff);
-      printf("Time difference per record: %.2f ms\n", time_diff / (number_records + max_sync_offset));
+      printf("  Total time: %.2f ms\n", time_diff);
+      printf("  Time per record: %.2f ms\n", time_diff / (number_records + max_sync_offset));
 
       // discard stale or previous record data for the first DISCARD_RECORD_DURATION_MS
       struct timespec record_start, record_now;
@@ -941,10 +779,9 @@ int main(int n, char **v)
         }
       } // End of while loop to discard stale records
     }
-  } // End a while loop to check for the "stop" command
-  // de-activate the tracee in the nr-softmodem
+  } // End of while loop for record/stop
 
-  // free_database(database); //Do on one app, for example on gNB App
+  // free_database(database); // Done on gNB App only
   free(is_on);
   close(socket);
 

common/utils/data_recording/config/config_data_recording.json

@@ -42,43 +42,59 @@
         "t_tracer_message_definition_file": "../T/T_messages.txt",
         "parameter_map_file": "config/wireless_link_parameter_map.yaml",
         "start_frame_number": 5,
-        "base_station": {
-            "requested_tracer_messages": [
-                "GNB_PHY_UL_FD_PUSCH_IQ",
-                "GNB_PHY_UL_FD_DMRS",
-                "GNB_PHY_UL_FD_CHAN_EST_DMRS_POS",
-                "GNB_PHY_UL_FD_CHAN_EST_DMRS_INTERPL",
-                "GNB_PHY_UL_PAYLOAD_RX_BITS"
-            ],
-            "meta_data":{
-                "num_rx_antennas": 1,
-                "tx_gain": 48.0,
-                "rx_gain": 30.0,
-                "hw_type": "USRP X410",
-                "hw_subtype": "ZBX",
-                "seid": "328AB35"
-            }
-        },
-        "user_equipment": {
-            "requested_tracer_messages": [
-                "UE_PHY_UL_SCRAMBLED_TX_BITS",
-                "UE_PHY_UL_PAYLOAD_TX_BITS"
-            ],
-            "meta_data":{
-                "num_tx_antennas": 1,
-                "tx_gain": 48.0,
-                "rx_gain": 40.0,
-                "hw_type": "USRP X410",
-                "hw_subtype": "ZBX",
-                "seid": "323F75F"
-            }
-        },
         "common_meta_data": {
             "sample_rate": 61440000.0,
             "bandwidth": 40000000.0,
             "clock_reference": "external"
         },
-        "tracer_service_baseStation_address": "127.0.0.1:2021",
-        "tracer_service_userEquipment_address": "127.0.0.1:2023"
+        "nodes": [
+            {
+                "id": "gnb_0",
+                "type": "gnb",
+                "requested_tracer_messages": [
+                    "GNB_PHY_UL_FD_PUSCH_IQ",
+                    "GNB_PHY_UL_FD_DMRS",
+                    "GNB_PHY_UL_FD_CHAN_EST_DMRS_POS",
+                    "GNB_PHY_UL_FD_CHAN_EST_DMRS_INTERPL",
+                    "GNB_PHY_UL_PAYLOAD_RX_BITS"],
+                "tracer_service_address": "127.0.0.1:2021",
+                "meta_data": {
+                    "num_tx_antennas": 1,
+                    "num_rx_antennas": 1,
+                    "tx_gain": 48.0,
+                    "rx_gain": 30.0,
+                    "hw_type": "USRP X410",
+                    "hw_subtype": "ZBX",
+                    "seid": "328AB35"
+                },
+                "shared_mem_config": {
+                    "read_path": "/tmp/gnb_app1",
+                    "write_path": "/tmp/gnb_app2",
+                    "project_id": 2335
+                }
+            },
+            {
+                "id": "ue_0",
+                "type": "ue",
+                "requested_tracer_messages": [
+                    "UE_PHY_UL_SCRAMBLED_TX_BITS",
+                    "UE_PHY_UL_PAYLOAD_TX_BITS"],
+                "tracer_service_address": "127.0.0.1:2023",
+                "meta_data": {
+                    "num_tx_antennas": 1,
+                    "num_rx_antennas": 1,
+                    "tx_gain": 48.0,
+                    "rx_gain": 30.0,
+                    "hw_type": "USRP X410",
+                    "hw_subtype": "ZBX",
+                    "seid": "323F75F"
+                },
+                "shared_mem_config": {
+                    "read_path": "/tmp/ue_app1",
+                    "write_path": "/tmp/ue_app2",
+                    "project_id": 2336
+                }
+            }
+        ]
     }
-}
+}

common/utils/data_recording/data_recording_app_v1.0.py

@@ -1,803 +0,0 @@
-# SPDX-License-Identifier: LicenseRef-CSSL-1.0
-
-# ---------------------------------------------------------------------
-# brief main application of synchronized real-time data recording
-
-import sysv_ipc as ipc
-import struct
-import time
-from datetime import datetime
-from termcolor import colored
-import numpy as np
-import json
-import concurrent.futures
-# from concurrent import futures
-import threading
-# import related functions
-from lib import sigmf_interface
-# import library functions
-from lib import sync_service
-from lib import data_recording_messages_def
-from lib import common_utils
-from lib import config_interface
-
-DEBUG_WIRELESS_RECORDED_DATA = True
-DEBUG_BUFFER_READING = False
-
-# globally applicable metadata
-global_info = {
-    "author": "Abdo Gaber",
-    "description": "Synchronized Real-Time Data Recording",
-    "timestamp": 0,
-    "collection_file_prefix": "data-collection",  # collection file name prefix "deap-rx + str(...)"
-    "collection_file": "",  # Reserved to be created in the code: “data-collection_rec-0_TIME-STAMP”
-    "datetime_offset": "",  # datetime offset between current location and UTC/Zulu timezone
-    # Example: "+01:00" for Berlin, Germany
-    "save_config_data_recording_app_json": True,
-    "waveform_generator": "5gnr_oai",
-    "extensions": {},
-}
-
-# Supported OAI Trace messages
-# UL receiver messages
-# gNB IQ Msgs: "GNB_PHY_UL_FD_PUSCH_IQ", "GNB_PHY_UL_FD_DMRS", "GNB_PHY_UL_FD_CHAN_EST_DMRS_POS",
-#               "GNB_PHY_UL_FD_CHAN_EST_DMRS_INTERPL"
-# gNB BITS Msgs: "GNB_PHY_UL_PAYLOAD_RX_BITS"
-# UE BITS Msgs: "UE_PHY_UL_SCRAMBLED_TX_BITS", "UE_PHY_UL_PAYLOAD_TX_BITS"
-
-supported_oai_tracer_messages = {
-    # gNB messages
-    "GNB_PHY_UL_FD_PUSCH_IQ": {
-        "file_name_prefix": "rx-fd-data",
-        "scope": "gNB",
-        "description": "Frequency-domain RX data",
-        "serialization_scheme": ["subcarriers", "ofdm_symbols"],
-    },
-    "GNB_PHY_UL_FD_DMRS": {
-        "file_name_prefix": "tx-pilots-fd-data",
-        "scope": "gNB",
-        "description": "Frequency-domain TX PUSCH DMRS data",
-        "serialization_scheme": ["subcarriers", "ofdm_symbols"],
-    },
-    "GNB_PHY_UL_FD_CHAN_EST_DMRS_POS": {
-        "file_name_prefix": "raw-ce-fd-data",
-        "scope": "gNB",
-        "description": "Frequency-domain raw channel estimates (at DMRS positions)",
-        "serialization_scheme": ["subcarriers", "ofdm_symbols"],
-    },
-    "GNB_PHY_UL_FD_CHAN_EST_DMRS_INTERPL": {
-        "file_name_prefix": "raw-inter-ce-fd-data",
-        "scope": "gNB",
-        "description": "Interpolcated Frequency-domain raw channel estimates",
-        "serialization_scheme": ["subcarriers", "ofdm_symbols"],
-    },
-    "GNB_PHY_UL_PAYLOAD_RX_BITS": {
-        "file_name_prefix": "rx-payload-bits",
-        "scope": "gNB",
-        "description": "Received PUSCH payload bits",
-        "serialization_scheme": ["bits", "subcarriers", "ofdm_symbols"],
-    },
-    # UE messages
-    "UE_PHY_UL_SCRAMBLED_TX_BITS": {
-        "file_name_prefix": "tx-scrambled-bits",
-        "scope": "UE",
-        "description": "Transmitted scrambled PUSCH bits",
-        "serialization_scheme": ["bits", "subcarriers", "ofdm_symbols"],
-    },
-    "UE_PHY_UL_PAYLOAD_TX_BITS": {
-        "file_name_prefix": "tx-payload-bits",
-        "scope": "UE",
-        "description": "Transmitted PUSCH payload bits",
-        "serialization_scheme": ["bits", "subcarriers", "ofdm_symbols"],
-    },
-}
-# -------------------------------------------
-# System configuration: gNB
-project_id_gnb = 2335
-read_shm_path_gnb = "/tmp/gnb_app1"
-write_shm_path_gnb = "/tmp/gnb_app2"
-
-# System configuration: UE
-project_id_ue = 2336
-read_shm_path_ue = "/tmp/ue_app1"
-write_shm_path_ue = "/tmp/ue_app2"
-
-
-# initialize shared memory
-def attach_shm(shm_path, project_id):
-    key = ipc.ftok(shm_path, project_id)
-    shm = ipc.SharedMemory(key, 0, 0)
-    # I found if we do not attach ourselves
-    # it will attach as ReadOnly.
-    shm.attach(0, 0)
-    return shm
-
-
-def detach_shm(shm):
-    try:
-        shm.detach()
-        print("Shared memory detached successfully.")
-    except ipc.ExistentialError:
-        print("Shared memory segment does not exist.")
-
-
-def remove_shm(shm):
-    try:
-        shm.remove()
-        print("Shared memory removed successfully.")
-    except ipc.ExistentialError:
-        print("Shared memory segment does not exist.")
-
-
-# check data if avalible in the shared memory
-def is_data_available_in_memory(shm, bufIdx, general_message_header_length, timeout=20):
-    start_time = time.time()
-    while True:
-        buf = shm.read(bufIdx + general_message_header_length)
-        n_bytes = sum(buf)
-        print("Data Recording App: Waiting for Measurements!")
-        if n_bytes > 0:
-            print("There is data in memory, n_bytes: ", n_bytes)
-            return True
-        if (time.time() - start_time) > timeout:
-            break
-        time.sleep(1)
-    return False
-
-
-# Read data from Shared memory based Data Conversion Service message structure
-def read_data_from_shm(shm, bufIdx, tracer_msgs_identities):
-    # print buffer index
-    if DEBUG_BUFFER_READING:
-        print("Buffer Index: ", bufIdx)
-    # get general message header list
-    general_msg_header_list, general_message_header_length = data_recording_messages_def.get_general_msg_header_list()
-    buf = shm.read(bufIdx + general_message_header_length)
-    n_bytes = sum(buf)
-    if n_bytes == 0:
-        raise Exception('ERROR: No data available in memory')
-    msg_id = struct.unpack('<H', buf[bufIdx:bufIdx+general_msg_header_list.get("msg_id")])[0]
-    bufIdx += general_msg_header_list.get("msg_id")
-    frame = struct.unpack('<H', buf[bufIdx:bufIdx+general_msg_header_list.get("frame")])[0]
-    bufIdx += general_msg_header_list.get("frame")
-    slot = struct.unpack('B', buf[bufIdx:bufIdx+general_msg_header_list.get("slot")])[0]
-    bufIdx += general_msg_header_list.get("slot")
-    # get time stamp:  yyyy mm dd hh mm ss msec
-    nr_trace_time_stamp_yyymmdd = \
-        struct.unpack('<i', buf[bufIdx:bufIdx+general_msg_header_list.get("datetime_yyyymmdd")])[0]
-    bufIdx += general_msg_header_list.get("datetime_yyyymmdd")
-    nr_trace_time_stamp_hhmmssmmm = \
-        struct.unpack('<i', buf[bufIdx:bufIdx+general_msg_header_list.get("datetime_hhmmssmmm")])[0]
-    bufIdx += general_msg_header_list.get("datetime_hhmmssmmm")
-    time_stamp_milli_sec = str(nr_trace_time_stamp_yyymmdd)+"_"+str(nr_trace_time_stamp_hhmmssmmm)
-    # get frame type
-    frame_type = struct.unpack('B', buf[bufIdx:bufIdx+general_msg_header_list.get("frame_type")])[0]
-    bufIdx += general_msg_header_list.get("frame_type")
-    # get frequency range
-    freq_range = struct.unpack('B', buf[bufIdx:bufIdx+general_msg_header_list.get("freq_range")])[0]
-    bufIdx += general_msg_header_list.get("freq_range")
-    # get subcarrier spacing
-    subcarrier_spacing = struct.unpack('B', buf[bufIdx:bufIdx+general_msg_header_list.get("subcarrier_spacing")])[0]
-    bufIdx += general_msg_header_list.get("subcarrier_spacing")
-    # get cyclic prefix
-    cyclic_prefix = struct.unpack('B', buf[bufIdx:bufIdx+general_msg_header_list.get("cyclic_prefix")])[0]
-    bufIdx += general_msg_header_list.get("cyclic_prefix")
-    # get symbols per slot
-    symbols_per_slot = struct.unpack('B', buf[bufIdx:bufIdx+general_msg_header_list.get("symbols_per_slot")])[0]
-    bufIdx += general_msg_header_list.get("symbols_per_slot")
-    # get Nid cell
-    Nid_cell = struct.unpack('<H', buf[bufIdx:bufIdx+general_msg_header_list.get("Nid_cell")])[0]
-    bufIdx += general_msg_header_list.get("Nid_cell")
-    # get rnti
-    rnti = struct.unpack('<H', buf[bufIdx:bufIdx+general_msg_header_list.get("rnti")])[0]
-    bufIdx += general_msg_header_list.get("rnti")
-    # get rb size
-    rb_size = struct.unpack('<H', buf[bufIdx:bufIdx+general_msg_header_list.get("rb_size")])[0]
-    bufIdx += general_msg_header_list.get("rb_size")
-    # get rb start
-    rb_start = struct.unpack('<H', buf[bufIdx:bufIdx+general_msg_header_list.get("rb_start")])[0]
-    bufIdx += general_msg_header_list.get("rb_start")
-    # get start symbol index
-    start_symbol_index = struct.unpack('B', buf[bufIdx:bufIdx+general_msg_header_list.get("start_symbol_index")])[0]
-    bufIdx += general_msg_header_list.get("start_symbol_index")
-    # get number of symbols
-    nr_of_symbols = struct.unpack('B', buf[bufIdx:bufIdx+general_msg_header_list.get("nr_of_symbols")])[0]
-    bufIdx += general_msg_header_list.get("nr_of_symbols")
-    # get qam modulation order
-    qam_mod_order = struct.unpack('B', buf[bufIdx:bufIdx+general_msg_header_list.get("qam_mod_order")])[0]
-    bufIdx += general_msg_header_list.get("qam_mod_order")
-    # get mcs index
-    mcs_index = struct.unpack('B', buf[bufIdx:bufIdx+general_msg_header_list.get("mcs_index")])[0]
-    bufIdx += general_msg_header_list.get("mcs_index")
-    # get mcs table
-    mcs_table = struct.unpack('B', buf[bufIdx:bufIdx+general_msg_header_list.get("mcs_table")])[0]
-    bufIdx += general_msg_header_list.get("mcs_table")
-    # get number of layers
-    nrOfLayers = struct.unpack('B', buf[bufIdx:bufIdx+general_msg_header_list.get("nrOfLayers")])[0]
-    bufIdx += general_msg_header_list.get("nrOfLayers")
-    # get transform precoding
-    transform_precoding = struct.unpack('B', buf[bufIdx:bufIdx+general_msg_header_list.get("transform_precoding")])[0]
-    bufIdx += general_msg_header_list.get("transform_precoding")
-    # get dmrs config type
-    dmrs_config_type = struct.unpack('B', buf[bufIdx:bufIdx+general_msg_header_list.get("dmrs_config_type")])[0]
-    bufIdx += general_msg_header_list.get("dmrs_config_type")
-    # get ul dmrs symb pos
-    ul_dmrs_symb_pos = struct.unpack('<H', buf[bufIdx:bufIdx+general_msg_header_list.get("ul_dmrs_symb_pos")])[0]
-    bufIdx += general_msg_header_list.get("ul_dmrs_symb_pos")
-    # get number dmrs symbols
-    number_dmrs_symbols = struct.unpack('B', buf[bufIdx:bufIdx+general_msg_header_list.get("number_dmrs_symbols")])[0]
-    bufIdx += general_msg_header_list.get("number_dmrs_symbols")
-    # get dmrs port
-    dmrs_port = struct.unpack('<H', buf[bufIdx:bufIdx+general_msg_header_list.get("dmrs_port")])[0]
-    bufIdx += general_msg_header_list.get("dmrs_port")
-    # get dmrs scid
-    dmrs_scid = struct.unpack('<H', buf[bufIdx:bufIdx+general_msg_header_list.get("dmrs_scid")])[0]
-    bufIdx += general_msg_header_list.get("dmrs_scid")
-    # get nb antennas rx for gNB or nb antennas tx for UE
-    nb_antennas = struct.unpack('B', buf[bufIdx:bufIdx+general_msg_header_list.get("nb_antennas")])[0]
-    bufIdx += general_msg_header_list.get("nb_antennas")
-    # get number of bits
-    number_of_bits = struct.unpack('<I', buf[bufIdx:bufIdx+general_msg_header_list.get("number_of_bits")])[0]
-    bufIdx += general_msg_header_list.get("number_of_bits")
-    # get length of bytes
-    length_bytes = struct.unpack('<I', buf[bufIdx:bufIdx+general_msg_header_list.get("length_bytes")])[0]
-    bufIdx += general_msg_header_list.get("length_bytes")
-
-    # print all captured data
-    if DEBUG_WIRELESS_RECORDED_DATA:
-        print(" ")
-        print(f"Time stamp: {time_stamp_milli_sec}")
-        print(f"MSG ID: {msg_id:<5} MSG Name: {tracer_msgs_identities[msg_id]}")
-        print(f"Frame: {frame:<5} Slot: {slot:<5}")
-        print(f"Frame Type: {frame_type:<5} Frequency Range: {freq_range:<5}"
-              f"Subcarrier Spacing: {subcarrier_spacing:<5} Cyclic Prefix: {cyclic_prefix:<5} "
-              f"Symbols per Slot: {symbols_per_slot:<5}")
-        print(f"Nid Cell: {Nid_cell:<5} RNTI: {rnti:<5}")
-        print(f"RB Size: {rb_size:<5} RB Start: {rb_start:<5} Start Symbol Index: {start_symbol_index:<5} "
-              f"Number of Symbols: {nr_of_symbols:<5}")
-        print(f"QAM Modulation Order: {qam_mod_order:<5} MCS Index: {mcs_index:<5} "
-              f"MCS Table: {mcs_table:<5}")
-        print(f"Number of Layers: {nrOfLayers:<5} Transform Precoding: {transform_precoding:<5}")
-        print(f"DMRS Config Type: {dmrs_config_type:<5} UL DMRS Symbol Position: {ul_dmrs_symb_pos:<5} "
-              f"Number of DMRS Symbols: {number_dmrs_symbols:<5}")
-        print(f"DMRS Port: {dmrs_port:<5} DMRS SCID: {dmrs_scid:<5} "
-              f"Number of Antennas: {nb_antennas:<5}")
-        print(f"Number of bits: {number_of_bits:<5} Length of bytes: {length_bytes:<5}")
-    # raise exception if time stamp is zero, it means that the data is not recorded yet
-    if nr_trace_time_stamp_yyymmdd == 0 and nr_trace_time_stamp_hhmmssmmm == 0:
-        raise Exception("ERROR: Time stamp is zero, data is not recorded yet or something wrong, check logs!")
-    # get recorded data
-    buf = shm.read(bufIdx + length_bytes)
-    # bit_msg_index = get_index_of_id(tracer_msgs_identities, "GNB_PHY_UL_PAYLOAD_RX_BITS")
-    captured_data = {}
-    # If message is bit message, store data in bytes
-    # then the field number_of_bits should be not zero
-    if "_BITS" in tracer_msgs_identities[msg_id]:
-        # recorded_data = buf[bufIdx:bufIdx + length_bytes]
-        recorded_data = struct.unpack("<" + int(length_bytes) * 'B', buf[bufIdx:bufIdx + length_bytes])
-        bufIdx += length_bytes
-        # convert data in bytes to bits
-        bits_vector = []
-        for byte in recorded_data:
-            bits_vector.extend([int(bit) for bit in format(int(byte), '08b')])
-        captured_data["sigmf_data_type"] = "ri8_le"
-        # convert to uint8
-        captured_data["recorded_data"] = np.asarray(bits_vector).astype(np.uint8)
-        # recorded_data_formated = recorded_data.astype(np.complex64) # convert to complex64
-    else:
-        recorded_data = struct.unpack("<" + int(length_bytes/2) * 'h', buf[bufIdx:bufIdx + length_bytes])
-        bufIdx += length_bytes
-        # print("IQ data I/Q: ", recorded_data)
-        # Convert real data to complext data
-        # converting list to array
-        recorded_data = np.asarray(recorded_data)
-        # recorded_data_complex = recorded_data
-        recorded_data_complex = common_utils.real_to_complex(recorded_data)
-        captured_data["sigmf_data_type"] = "cf32_le"
-        # convert to complex64
-        captured_data["recorded_data"] = recorded_data_complex.astype(np.complex64)
-    # print("Recorded Data: ", captured_data["recorded_data"])
-    # store data in dictonary
-    captured_data["message_id"] = msg_id
-    captured_data["message_type"] = tracer_msgs_identities[msg_id]
-    captured_data["frame"] = frame
-    captured_data["slot"] = slot
-    captured_data["time_stamp"] = time_stamp_milli_sec
-    captured_data["frame_type"] = frame_type
-    captured_data["freq_range"] = freq_range
-    captured_data["subcarrier_spacing"] = subcarrier_spacing
-    captured_data["cyclic_prefix"] = cyclic_prefix
-    # captured_data["symbols_per_slot"] = symbols_per_slot  ... not used
-    captured_data["Nid_cell"] = Nid_cell
-    captured_data["rnti"] = rnti
-    captured_data["rb_size"] = rb_size
-    captured_data["rb_start"] = rb_start
-    captured_data["start_symbol_index"] = start_symbol_index
-    captured_data["nr_of_symbols"] = nr_of_symbols
-    captured_data["qam_mod_order"] = qam_mod_order
-    captured_data["mcs_index"] = mcs_index
-    captured_data["mcs_table"] = mcs_table
-    captured_data["nrOfLayers"] = nrOfLayers
-    captured_data["transform_precoding"] = transform_precoding
-    captured_data["dmrs_config_type"] = dmrs_config_type
-    captured_data["ul_dmrs_symb_pos"] = ul_dmrs_symb_pos
-    captured_data["number_dmrs_symbols"] = number_dmrs_symbols
-    captured_data["dmrs_port"] = dmrs_port
-    captured_data["dmrs_scid"] = dmrs_scid
-    captured_data["nb_antennas"] = nb_antennas
-    captured_data["number_of_bits"] = number_of_bits
-    return captured_data, bufIdx
-
-
-# Synchronize data between gNB and UE
-def sync_data_conversion_service(
-        shm_reading_gnb, shm_reading_ue, sync_info, config_meta_data, gnb_args, ue_args):
-    # Initialize variables
-    record_idx = 0
-    prev_frame = -1
-    prev_slot = -1
-    ue_bufIdx = 0
-    gnb_bufIdx = 0
-
-    gnb_args.num_requested_tracer_msgs = len(
-        config_meta_data["data_recording_config"]["base_station"][
-            "requested_tracer_messages"])
-    ue_args.num_requested_tracer_msgs = len(
-        config_meta_data["data_recording_config"]["user_equipment"][
-            "requested_tracer_messages"])
-    tracer_msgs_identities = config_meta_data["data_recording_config"][
-        "tracer_msgs_identities"]
-
-    global_info = config_meta_data["data_recording_config"]["global_info"]
-
-    # Get UE data based on the sync data
-    bufIdx = 0
-    timeout_sync = time.time() + 5  # 5 seconds if no sync data found, stop the process
-    while True:
-        # wait for the next record
-        # To do: check if we need to add exta waiting times between different events in case of
-        # data streaming via network such as on UE side or gNB side
-        time.sleep(0.0035)  # 2.3 ms = latency of T tracer to capture data from the RAN
-        ue_bufIdx = bufIdx
-        captured_data, bufIdx = read_data_from_shm(shm_reading_ue, bufIdx, tracer_msgs_identities)
-        if (captured_data["frame"] == sync_info["frame"]
-                and captured_data["slot"] == sync_info["slot"]):
-            break
-        if time.time() > timeout_sync:
-            raise Exception(
-                "ERROR: Data Recording NO Sync Found, check Tracer Services if they are connected!")
-
-    # Get gNB data based on the sync data
-    bufIdx = 0
-    while True:
-        time.sleep(0.0035)
-        gnb_bufIdx = bufIdx
-        captured_data, bufIdx = read_data_from_shm(
-            shm_reading_gnb, bufIdx, tracer_msgs_identities)
-        if (captured_data["frame"] == sync_info["frame"]
-                and captured_data["slot"] == sync_info["slot"]):
-            break
-
-    # Read Synchronized data between gNB and UE
-    while True:  # read all records
-        print("\nRecord number: ", record_idx)
-        if DEBUG_BUFFER_READING:
-            print(f"Buffer Index gNB: {gnb_bufIdx}, Buffer Index UE: {ue_bufIdx}")
-        # wait for the next record
-        # To do: check if we need to add exta waiting times between different events in case of
-        # data streaming via network such as on UE side or gNB side
-        time.sleep(0.0035)  # 2.3 ms = latency of T tracer to capture data from the RAN
-
-        collected_metafiles = []
-        # Read data from gNB T-tracer Application
-        for idx in range(gnb_args.num_requested_tracer_msgs):
-            time.sleep(0.0015)
-            if DEBUG_WIRELESS_RECORDED_DATA:
-                print(f"\nRecord number: {record_idx}, Reading MSG data ", idx)
-            captured_data, gnb_bufIdx = read_data_from_shm(
-                shm_reading_gnb, gnb_bufIdx, tracer_msgs_identities)
-            # drive the collection file time stamp from the first message per record
-            if idx == 0:
-                # Get time stamp
-                time_stamp_ms, time_stamp_ms_file_name = (
-                    sigmf_interface.time_stamp_formating(
-                        captured_data["time_stamp"], global_info["datetime_offset"]))
-                global_info["collection_file"] = (
-                    global_info["collection_file_prefix"]
-                    + "-rec-"
-                    + str(record_idx)
-                    + "-"
-                    + str(time_stamp_ms_file_name)
-                )
-                global_info["timestamp"] = time_stamp_ms
-
-            # Write data into files with the given format
-            if config_meta_data["data_recording_config"]["enable_saving_tracer_messages_sigmf"]:
-                collected_metafiles.append(
-                    sigmf_interface.write_recorded_data_to_sigmf(
-                        captured_data, config_meta_data, global_info, record_idx))
-
-        # Read data from UE T-tracer Application
-        for idx in range(ue_args.num_requested_tracer_msgs):
-            time.sleep(0.0015)
-            if DEBUG_WIRELESS_RECORDED_DATA:
-                print(f"\nRecord number: {record_idx}, Reading MSG data ", idx)
-            captured_data, ue_bufIdx = read_data_from_shm(
-                shm_reading_ue, ue_bufIdx, tracer_msgs_identities)
-
-            # Write data into files with the given format
-            if config_meta_data["data_recording_config"]["enable_saving_tracer_messages_sigmf"]:
-                collected_metafiles.append(
-                    sigmf_interface.write_recorded_data_to_sigmf(
-                        captured_data, config_meta_data, global_info, record_idx))
-
-        # generate SigMF collection file
-        if config_meta_data["data_recording_config"]["enable_saving_tracer_messages_sigmf"]:
-            data_storage_path = config_meta_data["data_recording_config"][
-                "data_storage_path"]
-            description = global_info["description"]
-            sigmf_interface.save_sigmf_collection(
-                collected_metafiles, global_info, description, data_storage_path)
-
-        frame = captured_data["frame"]
-        slot = captured_data["slot"]
-
-        # Check for changes in frame or slot
-        if frame != prev_frame or slot != prev_slot:
-            record_idx += 1
-            # We have reached the end of the data. Break the loop
-            if record_idx >= config_meta_data["data_recording_config"]["num_records"]:
-                break
-        # Update previous frame and slot
-        prev_frame = frame
-        prev_slot = slot
-
-
-# data conversion service
-def data_conversion_service(shm_reading, config_meta_data, args, sync_info, do_sync):
-    # Initialize variables
-    record_idx = 0
-    prev_frame = -1
-    prev_slot = -1
-    bufIdx = 0
-    # Read data from T-tracer Application
-    print("Data Conversion Service: Reading data from T-tracer Application")
-    print("Requested Tracer Messages: ", args.num_requested_tracer_msgs)
-    if args.num_requested_tracer_msgs > 0:
-        num_requested_tracer_msgs = args.num_requested_tracer_msgs
-    else:
-        raise Exception("ERROR: No requested tracer messages found!")
-
-    tracer_msgs_identities = config_meta_data["data_recording_config"][
-        "tracer_msgs_identities"]
-    global_info = config_meta_data["data_recording_config"]["global_info"]
-
-    if do_sync:
-        print(" Find Memory Index of NR MSGs based on Sync info")
-        while True:
-            time.sleep(0.0035)
-            station_bufIdx = bufIdx
-            captured_data, bufIdx = read_data_from_shm(
-                shm_reading, bufIdx, tracer_msgs_identities)
-            print("*Sync Status: NR Captured Data (Frame, slot): (", captured_data["frame"],
-                  ", ", captured_data["slot"], "), Sync Info (Frame, slot): (", sync_info["frame"],
-                  ", ", sync_info["slot"]," )")
-            if (captured_data["frame"] == sync_info["frame"]
-                    and captured_data["slot"] == sync_info["slot"]):
-                print("*sync Pass")
-                break
-            print("*sync Fail")
-        bufIdx = station_bufIdx
-
-    # Read data from T-tracer Application
-    while True:  # read all records
-        print("\nRecord number: ", record_idx)
-        if DEBUG_BUFFER_READING:
-            print(f"Buffer Index: {bufIdx}")
-        # wait for the next record
-        # To do: check if we need to add exta waiting times between different events in case of
-        # data streaming via network such as on UE side or gNB side
-        time.sleep(0.0035)  # 2.3 ms = latency of T tracer to capture data from the RAN
-
-        collected_metafiles = []
-        for idx in range(num_requested_tracer_msgs):
-            time.sleep(0.0015)
-            if DEBUG_WIRELESS_RECORDED_DATA:
-                print(f"\nRecord number: {record_idx}, Reading MSG data ", idx)
-            captured_data, bufIdx = read_data_from_shm(
-                shm_reading, bufIdx, tracer_msgs_identities)
-            # derive the collection file time stamp from the first message per record
-            if idx == 0:
-                # Get time stamp
-                time_stamp_ms, time_stamp_ms_file_name = (
-                    sigmf_interface.time_stamp_formating(
-                        captured_data["time_stamp"], global_info["datetime_offset"]))
-                global_info["collection_file"] = (
-                    global_info["collection_file_prefix"]
-                    + "-rec-"
-                    + str(record_idx)
-                    + "-"
-                    + str(time_stamp_ms_file_name))
-                global_info["timestamp"] = time_stamp_ms
-            # Write data into files with the given format
-            if config_meta_data["data_recording_config"]["enable_saving_tracer_messages_sigmf"]:
-                collected_metafiles.append(
-                    sigmf_interface.write_recorded_data_to_sigmf(
-                        captured_data, config_meta_data, global_info, record_idx))
-
-        frame = captured_data["frame"]
-        slot = captured_data["slot"]
-
-        # generate SigMF collection file
-        if config_meta_data["data_recording_config"]["enable_saving_tracer_messages_sigmf"]:
-            data_storage_path = config_meta_data["data_recording_config"][
-                "data_storage_path"]
-            description = global_info["description"]
-            sigmf_interface.save_sigmf_collection(
-                collected_metafiles, global_info, description, data_storage_path)
-
-        # Check for changes in frame or slot
-        if frame != prev_frame or slot != prev_slot:
-            record_idx += 1
-            # We have reached the end of the data. Break the loop
-            if record_idx >= config_meta_data["data_recording_config"]["num_records"]:
-                break
-        # Update previous frame and slot
-        prev_frame = frame
-        prev_slot = slot
-
-
-# Write Tracer Control Message
-def write_shm(shm, args):
-    # Note: Big Endian >, Little Endian <
-    # Note: unsigned char B, signed char b, short h, int I, long long q
-    # Note: float f, double d, string s,  char c, bool ?
-    # 1: config
-    # 2: record
-    # 3: quit
-    print("Write Shared Memory: ", args.action)
-    if args.action == "config":
-        # Determine the length of the IP address
-        ip_length = len(args.bytes_IPaddress) + 1  # String terminator
-        # Construct the format string dynamically
-        format_string = f"{ip_length}s"
-        shm.write(
-            # Config action
-            struct.pack("<B", 1) +
-            struct.pack("<B", ip_length) +
-            struct.pack(format_string, args.bytes_IPaddress) +
-            struct.pack("<h", int(args.port))
-        )
-        print("T-Tracer Config IP: ", args.bytes_IPaddress, " port: ", args.port)
-    elif args.action == "record":
-        shm.write(
-                # Record action
-                struct.pack("<B", int(2)) +
-                struct.pack("<B", args.num_requested_tracer_msgs) +
-                struct.pack(
-                    "<{}h".format(len(args.req_tracer_msgs_indices)),
-                    *args.req_tracer_msgs_indices,) +
-                struct.pack("<I", args.num_records) +
-                struct.pack("<h", args.start_frame_number)
-        )
-        print("T-Tracer Record: N Messags: ", args.num_requested_tracer_msgs,
-              ", Msg IDs: ", args.req_tracer_msgs_indices,
-              ", Num records: ", args.num_records,
-              ", Start Frame: ", args.start_frame_number,
-              )
-    elif args.action == "quit":
-        shm.write(struct.pack("<B", int(3)))  # Quit action
-        print("T-Tracer Quit")
-    else:
-        print("Unknown action for data recording system!")
-
-
-# write shared memory task
-def write_shm_task(barrier, shm_id, args):
-    # Wait for threads to be ready
-    barrier.wait()
-    # send request to related T-Tracer Application
-    write_shm(shm_id, args)
-
-
-if __name__ == "__main__":
-    # -------------------------------------------
-    # ------------- Configuration --------------
-    # ------------------------------------------
-    # Data Recording Configuration
-    data_recording_config_file = "config/config_data_recording.json"
-    # -------------------------------------------
-    # Configuration
-    # -------------------------------------------
-    # First: get the configuration mode either local or remote
-    # Second: get data recording configuration
-    # Read and parse the JSON file
-    with open(data_recording_config_file, "r") as file:
-        config_meta_data = json.load(file)
-
-    # get Configuration parameters
-    config_meta_data, gnb_args, ue_args = config_interface.get_data_recording_config(config_meta_data)
-
-    # check if lists of requested tracer messages IDs are not empty
-    if not gnb_args.requested_tracer_messages and \
-            not ue_args.requested_tracer_messages:
-        raise Exception("ERROR: No requested tracer messages are provided")
-
-    # check if gnb_requested_tracer_messages is not empty, attach to the shared memory
-    if gnb_args.requested_tracer_messages:
-        # attach to the shared memory
-        shm_writing_gnb = attach_shm(write_shm_path_gnb, project_id_gnb)
-        shm_reading_gnb = attach_shm(read_shm_path_gnb, project_id_gnb)
-
-    # check if ue_requested_tracer_messages is not empty, attach to the shared memory
-    if ue_args.requested_tracer_messages:
-        # attach to the shared memory
-        shm_writing_ue = attach_shm(write_shm_path_ue, project_id_ue)
-        shm_reading_ue = attach_shm(read_shm_path_ue, project_id_ue)
-
-    # get general message header list
-    general_msg_header_list, general_message_header_length = \
-        data_recording_messages_def.get_general_msg_header_list()
-
-    # Add supported OAI Tracer Messages
-    config_meta_data["data_recording_config"]["supported_oai_tracer_messages"] = supported_oai_tracer_messages
-    # Add global info
-    config_meta_data["data_recording_config"]["global_info"] = global_info
-
-    # -------------------------------------------
-    # Initialization
-    # -------------------------------------------
-    # -------------------------------------------
-    # send Tracer Control Message request to T-Tracers Apps
-    # -------------------------------------------
-    #   It consists of the following fields:
-    #       Config action
-    #       IP address length
-    #       IP address
-    #       Port Number
-    # check if gnb_requested_tracer_messages is not empty, config T-Tracer gNB via shared memory
-    if gnb_args.requested_tracer_messages:
-        # Config T-Tracer via shared memory
-        gnb_args.action = "config"
-        write_shm(shm_writing_gnb, gnb_args)
-
-    # check if ue_requested_tracer_messages is not empty, config T-Tracer UE via shared memory
-    if ue_args.requested_tracer_messages:
-        # Config T-Tracer via shared memory
-        ue_args.action = "config"
-        write_shm(shm_writing_ue, ue_args)
-
-    time.sleep(0.5)  # wait for the config to be applied
-
-    # -------------------------------------------
-    # Execution
-    # -------------------------------------------
-    # send Tracer Control Message request to T-Tracers Apps
-    # -------------------------------------------
-    #   It consists of the following fields:
-    #       Record action
-    #       Number of requested Tracer Messages
-    #       Requested Tracer Messages ID 1, …, ID N
-    #       Number of records to be recorded in slots
-    #       Start SFN: Frame Index to start data collection from it, useful for future
-    #       data sync between gNB and UE but not yet used
-    print("Args:")
-    if gnb_args.requested_tracer_messages:
-        gnb_args.action = "record"
-        print("gnb_args: ", gnb_args)
-    if ue_args.requested_tracer_messages:
-        ue_args.action = "record"
-        print("ue_args: ", ue_args)
-
-    start_time = time.time()
-    print("Send data logging request us:", datetime.now().strftime("%Y%m%d-%H%M%S%f"))
-
-    # if requested: gNB and UE Tracer Messages
-    # gNB + UE Tracer Messages
-    if gnb_args.requested_tracer_messages and ue_args.requested_tracer_messages:
-        # Create a barrier to synchronize the threads
-        barrier = threading.Barrier(2)
-        with concurrent.futures.ThreadPoolExecutor() as executor:
-            tracer_ue = executor.submit(write_shm_task, barrier, shm_writing_ue, ue_args)
-            tracer_gnb = executor.submit(write_shm_task, barrier, shm_writing_gnb, gnb_args)
-            # Wait for both functions to complete
-            concurrent.futures.wait([tracer_ue, tracer_gnb])
-
-    # gNB Tracer Messages
-    elif gnb_args.requested_tracer_messages:
-        write_shm(shm_writing_gnb, gnb_args)
-
-    # UE Tracer Messages
-    elif ue_args.requested_tracer_messages:
-        write_shm(shm_writing_ue, ue_args)
-    else:
-        raise Exception("ERROR: No requested tracer messages IDs are provided")
-
-    # -------------------------------------------
-    # Read data from gNB and UE T-tracer Application
-    # -------------------------------------------
-    # Check if data is available in memory
-    # Initialize variables
-    bufIdx = 0
-    timeout = 10  # 10 seconds from now
-
-    # If gNB MSGs are requested
-    if gnb_args.requested_tracer_messages:
-        # Check if data is available in gNB memory
-        is_gnb_data_in_memory = is_data_available_in_memory(
-            shm_reading_gnb, bufIdx, general_message_header_length, timeout)
-        # Report the status of gNB T-Tracer APP locally
-        if not is_gnb_data_in_memory:
-            print("Error: gNB: Check t-Tracer APP of gNB, check IPs and Ports")
-            print("Error: gNB: If IPs and Ports are correct, re-run the hanging app.")
-            print("It seems the socket was not closed properly")
-            raise Exception("ERROR: Time out, check if gNB T-Tracer APP connected to stack")
-    # If UE MSGs are requested
-    if ue_args.requested_tracer_messages:
-        # Check if data is available in UE memory
-        is_ue_data_in_memory = is_data_available_in_memory(
-            shm_reading_ue, bufIdx, general_message_header_length, timeout)
-        # Report the status of UE T-Tracer APP locally
-        if not is_ue_data_in_memory:
-            print("Error: UE: Check t-Tracer APP of UE, check IPs and Ports")
-            print("Error: UE: If IPs and Ports are correct, re-run the hanging app.")
-            print("It seems the socket was not closed properly")
-            raise Exception("ERROR: Time out, check if UE T-Tracer APP connected to stack")
-
-    # -------------------------------------------
-    # Sync data between gNB and UE
-    # -------------------------------------------
-    # write JSON file
-    common_utils.write_config_data_recording_app_json(config_meta_data)
-    sync_info = {}
-    if gnb_args.requested_tracer_messages and ue_args.requested_tracer_messages:
-        # Sync data between gNB and UE
-        sync_info = sync_service.sync_gnb_ue_captured_data(shm_reading_gnb, shm_reading_ue)
-        print("\n***Sync data between gNB and UE: ", sync_info)
-        # Read data from gNB and UE T-tracer Applications
-        sync_data_conversion_service(
-            shm_reading_gnb, shm_reading_ue, sync_info, config_meta_data, gnb_args, ue_args)
-    elif gnb_args.requested_tracer_messages:
-        # Read data from gNB T-tracer Application
-        data_conversion_service(
-            shm_reading_gnb, config_meta_data, gnb_args, sync_info, do_sync=False)
-    elif ue_args.requested_tracer_messages:
-        # Read data from UE T-tracer Application
-        data_conversion_service(
-            shm_reading_ue, config_meta_data, ue_args, sync_info, do_sync=False)
-    else:
-        raise Exception("ERROR: No requested tracer messages IDs are provided")
-
-    # measure Elapsed time
-    time_elapsed = time.time() - start_time
-    time_elapsed_ms = int(time_elapsed * 1000)
-    print(
-        "Elapsed time of getting Requested Messages and writing data and meta data files:",
-        colored(time_elapsed_ms, "yellow"), "ms",)
-
-    # Stop T-Tracer Application function
-    if gnb_args.requested_tracer_messages:
-        gnb_args.action = "quit"
-        write_shm(shm_writing_gnb, gnb_args)
-        # Add Sleep time to ensure that the message sent to the UE T-tracer application is received
-        # before the shared memory is detached
-        time.sleep(0.5)
-        # Clean shared memory
-        detach_shm(shm_reading_gnb)
-        detach_shm(shm_writing_gnb)
-        remove_shm(shm_reading_gnb)
-        remove_shm(shm_writing_gnb)
-    if ue_args.requested_tracer_messages:
-        ue_args.action = "quit"
-        write_shm(shm_writing_ue, ue_args)
-        # Add Sleep time to ensure that the message sent to the UE T-tracer application is received
-        # before the shared memory is detached
-        time.sleep(0.5)
-        # Clean shared memory
-        detach_shm(shm_reading_ue)
-        detach_shm(shm_writing_ue)
-        remove_shm(shm_reading_ue)
-        remove_shm(shm_writing_ue)
-
-    print("End of the RF Data Recording API")
-    pass

common/utils/data_recording/data_recording_app_v1.1.py

@@ -0,0 +1,523 @@
+# SPDX-License-Identifier: LicenseRef-CSSL-1.0
+
+# ---------------------------------------------------------------------
+# brief main application of synchronized real-time data recording
+
+import atexit
+import signal
+import sys
+import time
+from datetime import datetime
+import json
+import concurrent.futures
+from termcolor import colored
+import threading
+# import library functions
+from lib import sigmf_interface
+from lib import sync_service
+from lib import data_recording_messages_def
+from lib import common_utils
+from lib import config_interface
+from lib import shared_memory_interface as shm_interface
+
+DEBUG_WIRELESS_RECORDED_DATA = True
+DEBUG_BUFFER_READING = False 
+
+# globally applicable metadata
+global_info = {
+    "author": "Abdo Gaber",
+    "description": "Synchronized Real-Time Data Recording",
+    "timestamp": 0,
+    "collection_file_prefix": "data-collection",  # collection file name prefix
+    "collection_file": "",  # Reserved to be created in the code: “data-collection_rec-0_TIME-STAMP”
+    "datetime_offset": "",  # datetime offset between current location and UTC/Zulu timezone
+    # Example: "+01:00" for Berlin, Germany
+    "save_config_data_recording_app_json": True,
+    "waveform_generator": "5gnr_oai",
+    "extensions": {},
+}
+
+# Supported OAI Trace messages
+# UL receiver messages
+# gNB IQ Msgs: "GNB_PHY_UL_FD_PUSCH_IQ", "GNB_PHY_UL_FD_DMRS", "GNB_PHY_UL_FD_CHAN_EST_DMRS_POS", 
+#               "GNB_PHY_UL_FD_CHAN_EST_DMRS_INTERPL"
+# gNB BITS Msgs: "GNB_PHY_UL_PAYLOAD_RX_BITS"
+# UE BITS Msgs: "UE_PHY_UL_SCRAMBLED_TX_BITS", "UE_PHY_UL_PAYLOAD_TX_BITS"
+
+supported_oai_tracer_messages = {
+    # gNB messages
+    "GNB_PHY_UL_FD_PUSCH_IQ": {
+        "file_name_prefix": "rx-fd-data",
+        "scope": "gNB",
+        "description": "Frequency-domain RX data",
+        "serialization_scheme": ["subcarriers", "ofdm_symbols"],
+    },
+    "GNB_PHY_UL_FD_DMRS": {
+        "file_name_prefix": "tx-pilots-fd-data",
+        "scope": "gNB",
+        "description": "Frequency-domain TX PUSCH DMRS data",
+        "serialization_scheme": ["subcarriers", "ofdm_symbols"],
+    },
+    "GNB_PHY_UL_FD_CHAN_EST_DMRS_POS": {
+        "file_name_prefix": "raw-ce-fd-data",
+        "scope": "gNB",
+        "description": "Frequency-domain raw channel estimates (at DMRS positions)",
+        "serialization_scheme": ["subcarriers", "ofdm_symbols"],
+    },
+    "GNB_PHY_UL_FD_CHAN_EST_DMRS_INTERPL": {
+        "file_name_prefix": "raw-inter-ce-fd-data",
+        "scope": "gNB",
+        "description": "Interpolcated Frequency-domain raw channel estimates",
+        "serialization_scheme": ["subcarriers", "ofdm_symbols"],
+    },
+    "GNB_PHY_UL_PAYLOAD_RX_BITS": {
+        "file_name_prefix": "rx-payload-bits",
+        "scope": "gNB",
+        "description": "Received PUSCH payload bits",
+        "serialization_scheme": ["bits", "subcarriers", "ofdm_symbols"],
+    },
+    # UE messages
+    "UE_PHY_UL_SCRAMBLED_TX_BITS": {
+        "file_name_prefix": "tx-scrambled-bits",
+        "scope": "UE",
+        "description": "Transmitted scrambled PUSCH bits",
+        "serialization_scheme": ["bits", "subcarriers", "ofdm_symbols"],
+    },
+    "UE_PHY_UL_PAYLOAD_TX_BITS": {
+        "file_name_prefix": "tx-payload-bits",
+        "scope": "UE",
+        "description": "Transmitted PUSCH payload bits",
+        "serialization_scheme": ["bits", "subcarriers", "ofdm_symbols"],
+    },
+}    
+       
+def read_and_store_tracer_messages(shm_reading, bufIdx, num_messages, config_meta_data, 
+                         global_info, update_timestamp=False):
+    """
+    Read tracer messages from shared memory while they belong to the same frame and slot.
+    Continuously reads messages until a different frame/slot is encountered.
+    
+    Args:
+        shm_reading: Shared memory reading handle
+        bufIdx: Current buffer index
+        num_messages: Maximum number of messages to read (used as safeguard)
+        config_meta_data: Configuration metadata
+        global_info: Global information dict (for timestamp on first message)
+        update_timestamp: If True, update global_info timestamp from first message
+    
+    Returns:
+        tuple: (updated_bufIdx, collected_metafiles, last_captured_data)
+    """
+    tracer_msgs_identities = config_meta_data["data_recording_config"]["tracer_msgs_identities"]
+    collected_metafiles = []
+    captured_data = None
+    
+    # Get sync header to check frame and slot
+    sync_header_msg, sync_header_msg_length = data_recording_messages_def.get_sync_header_msg_list()
+    
+    # Read first message to establish the reference frame and slot
+    timeout = 10  # 10 seconds timeout
+    is_data_in_memory = shm_interface.is_data_available_in_memory(
+        shm_reading, bufIdx, sync_header_msg_length, timeout)
+    
+    if not is_data_in_memory:
+        if DEBUG_WIRELESS_RECORDED_DATA:
+            print("Warning: No data available in shared memory")
+        return bufIdx, collected_metafiles, global_info
+    
+    # Get reference frame and slot from first message
+    ref_frame, ref_slot = shm_interface.get_frame_slot_start(
+        shm_reading, bufIdx, sync_header_msg, sync_header_msg_length)
+    
+    if DEBUG_WIRELESS_RECORDED_DATA:
+        print(f"\nRecord number: {global_info['record_idx']}, Reference Frame: {ref_frame}, Slot: {ref_slot}")
+    
+    idx = 0
+    while idx < num_messages:
+        time.sleep(0.0015)
+        
+        # Check if data is available
+        is_data_in_memory = shm_interface.is_data_available_in_memory(
+            shm_reading, bufIdx, sync_header_msg_length, timeout)
+        
+        if not is_data_in_memory:
+            break
+        
+        # Get frame and slot of current message before reading
+        current_frame, current_slot = shm_interface.get_frame_slot_start(
+            shm_reading, bufIdx, sync_header_msg, sync_header_msg_length)
+        
+        # Check if we're still on the same frame and slot
+        if current_frame != ref_frame or current_slot != ref_slot:
+            if DEBUG_WIRELESS_RECORDED_DATA:
+                print(f"Frame/Slot changed: ({ref_frame}, {ref_slot}) -> ({current_frame}, {current_slot}). Stopping at message {idx}")
+            # Keep buffer index pointing to this new frame/slot message (don't increment)
+            break
+        
+        if DEBUG_WIRELESS_RECORDED_DATA:
+            print(f"Reading MSG {idx}: Frame={current_frame}, Slot={current_slot}")
+        
+        # Read the message
+        captured_data, bufIdx = shm_interface.read_data_from_shm(
+            shm_reading, bufIdx, tracer_msgs_identities)
+        
+        # Update timestamp from the first message if requested
+        if idx == 0 and update_timestamp:
+            time_stamp, time_stamp_file_name = (
+                sigmf_interface.time_stamp_formating(
+                    captured_data["unix_capture_ts_sec"],
+                    captured_data["unix_capture_ts_nsec"],
+                    global_info["datetime_offset"]))
+            global_info["collection_file"] = (
+                f"{global_info['collection_file_prefix']}-rec-{global_info['record_idx']}-{time_stamp_file_name}")
+            global_info["timestamp"] = time_stamp
+            global_info["frame"] = captured_data["frame"]
+            global_info["slot"]= captured_data["slot"]
+        
+        # Write data into files with the given format
+        if config_meta_data["data_recording_config"]["enable_saving_tracer_messages_sigmf"]:
+            collected_metafiles.append(sigmf_interface.write_recorded_data_to_sigmf(
+                                        captured_data, config_meta_data, global_info, global_info['record_idx']))
+        
+        idx += 1
+    
+    if DEBUG_WIRELESS_RECORDED_DATA:
+        print(f"Total messages read: {idx} for Frame {ref_frame}, Slot {ref_slot}")
+    
+    return bufIdx, collected_metafiles, global_info
+
+def find_sync_buffer_index(shm_reading, sync_info, tracer_msgs_identities, timeout_sec=None):
+    """
+    Find buffer index that matches sync frame and slot.
+    
+    Args:
+        shm_reading: Shared memory reading handle
+        sync_info: Dictionary with 'frame' and 'slot' keys
+        tracer_msgs_identities: Tracer message identities
+        timeout_sec: Timeout in seconds (None for no timeout)
+    
+    Returns:
+        bufIdx: Buffer index where sync match was found (points to the matching message)
+    
+    Raises:
+        Exception if timeout occurs before sync is found
+    """
+    bufIdx = 0
+    timeout_sync = time.time() + timeout_sec if timeout_sec else None
+    
+    while True:
+        time.sleep(0.0035)  # 2.3 ms = latency of T tracer to capture data from the RAN
+        saved_bufIdx = bufIdx  # Save index BEFORE read
+        captured_data, bufIdx = shm_interface.read_data_from_shm(
+            shm_reading, bufIdx, tracer_msgs_identities)
+        print(f"  [NR Data Sync] NR Data (Frame:{captured_data['frame']}, Slot:{captured_data['slot']}) | "
+              f"Target (Frame:{sync_info['frame']}, Slot:{sync_info['slot']})")
+        
+        if (captured_data["frame"] == sync_info["frame"] and 
+            captured_data["slot"] == sync_info["slot"]):
+            print("  [NR Data Sync] Sync pass")
+            return saved_bufIdx  # Return index pointing to the matching message
+        
+        if timeout_sync and time.time() > timeout_sync:
+            raise Exception(
+                "ERROR: Data Recording NO Sync Found, check Tracer Services if they are connected!")
+
+def send_5g_nr_msgs_request(barrier, shm_writing, shm_reading, args):
+    # send Tracer Control Message request to T-Tracers Apps
+    # -------------------------------------------
+    #   It consists of the following fields:
+    #       Record action
+    #       Number of requested Tracer Messages
+    #       Requested Tracer Messages ID 1, …, ID N
+    #       Number of records to be recorded in slots
+    #       Start SFN: Frame Index to start data collection from it, but not yet used
+    shm_interface.write_shm_task(barrier,shm_writing, args)
+
+def generic_data_conversion_service(node_shm, config_meta_data, tracer_nodes, sync_info=None):
+    """Generic data conversion service for N tracer nodes.
+
+    Args:
+        node_shm: dict {node_id: {"reading": shm, "writing": shm}}
+        config_meta_data: Config metadata dict
+        tracer_nodes: dict {node_id: args} for nodes with tracer messages
+        sync_info: Optional sync info dict with 'frame' and 'slot'. If provided,
+                   each node's buffer index is synced to this frame/slot.
+    """
+    record_idx = 0
+    prev_frame = -1
+    prev_slot = -1
+
+    tracer_msgs_identities = config_meta_data["data_recording_config"]["tracer_msgs_identities"]
+    global_info = config_meta_data["data_recording_config"]["global_info"]
+
+    # Find sync buffer indices for each node if sync_info provided
+    node_buf_idx = {}
+    for node_id in tracer_nodes:
+        if sync_info:
+            print(f"  [NR Data Sync] Finding memory index for node {node_id}")
+            node_buf_idx[node_id] = find_sync_buffer_index(
+                node_shm[node_id]["reading"], sync_info, tracer_msgs_identities, timeout_sec=5)
+        else:
+            node_buf_idx[node_id] = 0
+
+    # Determine num_records from first tracer node
+    num_records = next(iter(tracer_nodes.values())).num_records
+
+    print("\n--- Data Conversion: Reading data from T-tracer Applications ---")
+    for node_id, args in tracer_nodes.items():
+        print(f"  [Data Conversion] {node_id}: Requested tracer messages: {args.num_requested_tracer_msgs}")
+
+    # Read data from all tracer nodes
+    while True:
+        if DEBUG_WIRELESS_RECORDED_DATA:
+            print("\nRecord number: ", record_idx)
+        if DEBUG_BUFFER_READING:
+            for nid in tracer_nodes:
+                print(f"  Buffer Index {nid}: {node_buf_idx[nid]}")
+        time.sleep(0.0035)
+
+        global_info["record_idx"] = record_idx
+        all_metafiles = []
+
+        for i, (node_id, args) in enumerate(tracer_nodes.items()):
+            buf_idx, metafiles, global_info = read_and_store_tracer_messages(
+                node_shm[node_id]["reading"], node_buf_idx[node_id],
+                args.num_requested_tracer_msgs, config_meta_data, global_info,
+                update_timestamp=(i == 0))
+            node_buf_idx[node_id] = buf_idx
+            all_metafiles.extend(metafiles)
+
+        # Generate SigMF collection file
+        if config_meta_data["data_recording_config"]["enable_saving_tracer_messages_sigmf"]:
+            data_storage_path = config_meta_data["data_recording_config"]["data_storage_path"]
+            description = global_info["description"]
+            sigmf_interface.save_sigmf_collection(
+                all_metafiles, global_info, description, data_storage_path)
+
+        frame = global_info["frame"]
+        slot = global_info["slot"]
+
+        if frame != prev_frame or slot != prev_slot:
+            record_idx += 1
+            if record_idx >= num_records:
+                break
+        prev_frame = frame
+        prev_slot = slot
+
+# -------------------------------------------
+# Cleanup — ensures shared memory and resources are released even on Ctrl+C
+# -------------------------------------------
+# Module-level dict populated at runtime with resources that need cleanup.
+# Each key is optional; the cleanup function checks before acting.
+_cleanup_state = {"done": False}
+
+
+def cleanup_resources():
+    """Release T-tracer shared memory resources."""
+    if _cleanup_state["done"]:
+        return
+    _cleanup_state["done"] = True
+    print("\nCleaning up resources...")
+
+    tracer_nodes = _cleanup_state.get("tracer_nodes", {})
+    node_shm = _cleanup_state.get("node_shm", {})
+    for node_id in tracer_nodes:
+        try:
+            tracer_nodes[node_id].action = "quit"
+            shm_interface.write_shm(node_shm[node_id]["writing"], tracer_nodes[node_id])
+            time.sleep(0.5)
+            shm_interface.detach_shm(node_shm[node_id]["reading"])
+            shm_interface.detach_shm(node_shm[node_id]["writing"])
+            shm_interface.remove_shm(node_shm[node_id]["reading"])
+            shm_interface.remove_shm(node_shm[node_id]["writing"])
+        except Exception as e:
+            print(f"Warning: {node_id} shared memory cleanup error: {e}")
+
+
+def register_cleanup():
+    """Register cleanup_resources with atexit and signal handlers."""
+    atexit.register(cleanup_resources)
+
+    def _signal_handler(signum, frame):
+        print(f"\nSignal {signum} received, shutting down...")
+        cleanup_resources()
+        sys.exit(130)
+
+    signal.signal(signal.SIGINT, _signal_handler)
+    signal.signal(signal.SIGTERM, _signal_handler)
+
+
+if __name__ == "__main__":
+    # -------------------------------------------
+    # Data Control Service
+    ## -------------------------------------------
+    # ------------- Configuration --------------
+    # ------------------------------------------
+    # Data Recording Configuration
+    data_recording_config_file = "config/config_data_recording.json"
+    # -------------------------------------------
+    # Configuration
+    # -------------------------------------------
+    # Read and parse the JSON file
+    with open(data_recording_config_file, "r") as file:
+        config_meta_data = json.load(file)
+    # -------------------------------------------
+    # Generic node-based configuration
+    # -------------------------------------------
+    config_meta_data, all_node_args = config_interface.get_data_recording_config(config_meta_data)
+    _nodes_cfg = config_meta_data["data_recording_config"]["nodes"]
+
+    # Build tracer_nodes: only nodes with requested tracer messages
+    tracer_nodes = {nid: args for nid, args in all_node_args.items()
+                    if args.requested_tracer_messages}
+    
+    # Validate: at least one node must have tracer messages
+    any_tracer = bool(tracer_nodes)
+    if not any_tracer:
+        raise Exception("ERROR: No requested tracer messages IDs are provided")
+
+    # -------------------------------------------
+    # Attach shared memory for each tracer node
+    # -------------------------------------------
+    node_shm = {}  # {node_id: {"reading": shm, "writing": shm}}
+    for node_id, args in tracer_nodes.items():
+        node_cfg = next(n for n in _nodes_cfg if n["id"] == node_id)
+        shm_cfg = node_cfg.get("shared_mem_config", {})
+        read_path = shm_cfg.get("read_path")
+        write_path = shm_cfg.get("write_path")
+        proj_id = shm_cfg.get("project_id")
+        if not read_path or not write_path or proj_id is None:
+            raise Exception(
+                f"ERROR: Node '{node_id}' has tracer messages but missing 'shared_mem_config' "
+                f"(read_path, write_path, project_id) in config.")
+        shm_writing = shm_interface.attach_shm(write_path, proj_id)
+        shm_reading = shm_interface.attach_shm(read_path, proj_id)
+        node_shm[node_id] = {"reading": shm_reading, "writing": shm_writing}
+
+    # get general message header list
+    general_msg_header_list, general_message_header_length = \
+        data_recording_messages_def.get_general_msg_header_list()
+
+    # Add supported OAI Tracer Messages
+    config_meta_data["data_recording_config"]["supported_oai_tracer_messages"] = supported_oai_tracer_messages
+    # Add global info
+    config_meta_data["data_recording_config"]["global_info"] = global_info
+
+    # -------------------------------------------
+    # Initialization
+    # -------------------------------------------  
+    # send Tracer Control Message request to T-Tracers Apps
+    #   It consists of the following fields:
+    #       Config action
+    #       IP address length
+    #       IP address
+    #       Port Number
+    # Configure T-Tracers via shared memory for each tracer node
+    for node_id, args in tracer_nodes.items():
+        args.action = "config"
+        shm_interface.write_shm(node_shm[node_id]["writing"], args)
+
+    time.sleep(0.5)  # wait for the config to be applied
+
+    # -------------------------------------------
+    # Execution
+    # -------------------------------------------
+    # send Tracer Control Message request to T-Tracers Apps
+    # -------------------------------------------
+    #   It consists of the following fields:
+    #       Record action
+    #       Number of requested Tracer Messages
+    #       Requested Tracer Messages ID 1, …, ID N
+    #       Number of records to be recorded in slots
+    #       Start SFN: Frame Index to start data collection from it, useful for future
+    #       data sync between gNB and UE but not yet used
+    # -------------------------------------------
+    # -------- Data Collection Service ----------
+    # -------------------------------------------
+    print("Args:")
+    for node_id, args in tracer_nodes.items():
+        args.action = "record"
+        print(f"  {node_id} args: ", args)
+    
+    # Pre-create thread pool if user would like to create a for loop for N iterations 
+    # Outside the loop to avoid initialization overhead.
+    # max_workers and barrier_parties are kept identical: every thread that
+    # enters the pool must also call barrier.wait(), so pool size == barrier size.
+    total_thread_count = len(tracer_nodes)
+    thread_pool = concurrent.futures.ThreadPoolExecutor(max_workers=total_thread_count)
+    barrier = threading.Barrier(total_thread_count) if total_thread_count > 0 else None
+    
+    # Populate cleanup state and register signal handlers
+    _cleanup_state.update({
+        "tracer_nodes": tracer_nodes,
+        "node_shm": node_shm,
+    })
+    register_cleanup()
+
+    # Start data recording iterations
+    start_time = time.time()
+    print("Send data logging request us:", datetime.now().strftime("%Y%m%d-%H%M%S%f"))
+
+    # -------------------------------------------
+    # Submit NR tracer threads for all tracer nodes
+    # -------------------------------------------
+    futures = []
+    for node_id, args in tracer_nodes.items():
+        shm = node_shm[node_id]
+        tracer_thread = thread_pool.submit(
+            send_5g_nr_msgs_request, barrier, shm["writing"], shm["reading"], args)
+        futures.append(tracer_thread)
+
+    # Wait for all threads to complete
+    concurrent.futures.wait(futures)
+
+    # -------------------------------------------
+    # Check data availability for each tracer node
+    # -------------------------------------------
+    bufIdx = 0
+    timeout = 10  # 10 seconds from now
+
+    for node_id in tracer_nodes:
+        is_data_in_memory = shm_interface.is_data_available_in_memory(
+            node_shm[node_id]["reading"], bufIdx, general_message_header_length, timeout)
+
+        if not is_data_in_memory:
+            print(f"Error: {node_id}: Check T-Tracer APP, check IPs and Ports")
+            print(f"Error: {node_id}: If IPs and Ports are correct, re-run the hanging app. "
+                    "It seems the socket was not closed properly")
+            raise Exception(
+                f"ERROR: Time out, check if {node_id} T-Tracer APP connected to stack")
+
+    # -------------------------------------------
+    # Sync and convert data
+    # -------------------------------------------
+    common_utils.write_config_data_recording_app_json(config_meta_data)
+    sync_info = {}
+
+    # Step 1: Sync NR tracer nodes (find latest common frame/slot across all NR sources)
+    if len(tracer_nodes) > 1:
+        node_shm_readings = {nid: node_shm[nid]["reading"] for nid in tracer_nodes}
+        sync_info = sync_service.sync_multiple_nr_nodes(node_shm_readings)
+        print(f"\n--- NR Node Sync: Sync point: frame={sync_info['frame']}, slot={sync_info['slot']} ---")
+
+    # Step 2: Read and store NR tracer data
+    if tracer_nodes:
+        do_sync = bool(sync_info)
+        generic_data_conversion_service(
+            node_shm, config_meta_data, tracer_nodes, sync_info if do_sync else None)
+
+    # measure Elapsed time
+    time_elapsed = time.time() - start_time
+    time_elapsed_ms = int(time_elapsed * 1000)
+    print(
+        "Elapsed time of getting Requested Messages and writing data and meta data files:",
+        colored(time_elapsed_ms, "yellow"),
+        "ms",)
+
+    # Shutdown thread pool after all iterations complete
+    thread_pool.shutdown(wait=True)
+
+    # Run cleanup (also registered with atexit and signal handlers for abnormal exits)
+    cleanup_resources()
+        
+    print("End of the RF Data Recording API")

common/utils/data_recording/lib/common_utils.py

@@ -1,12 +1,11 @@
 # SPDX-License-Identifier: LicenseRef-CSSL-1.0
 
 # ---------------------------------------------------------------------
-# brief Data Recording common utilities
+# brief Common utilities of Data Recording App
 
 import os
 import json
 
-
 def real_to_complex(real_vector):
     # Ensure the length of the real vector is even
     if len(real_vector) % 2 != 0:
@@ -34,7 +33,7 @@ def write_config_data_recording_app_json(config_meta_data):
         # Specify the file name
         output_file = (
             config_meta_data["data_recording_config"]["data_storage_path"]
-            + "config_data_recording_app.json"
+            + "config_data_recording_app_extended.json"
         )
         # Ensure the directory exists
         os.makedirs(os.path.dirname(output_file), exist_ok=True)
@@ -43,6 +42,6 @@ def write_config_data_recording_app_json(config_meta_data):
         with open(output_file, "w") as file:
             try:
                 json.dump(config_meta_data, file, indent=4)
-                print(f"JSON file created successfully at {output_file}")
+                print(f"  [Config] JSON file saved: {output_file}")
             except Exception as e:
-                print(f"Failed to create JSON file: {e}")
+                print(f"  [Config] Failed to create JSON file: {e}")   

common/utils/data_recording/lib/config_interface.py

@@ -1,18 +1,17 @@
 # SPDX-License-Identifier: LicenseRef-CSSL-1.0
 
 # ---------------------------------------------------------------------
-# brief Data Recording App Configuration Interface
+# brief Configuration Interface of Data Recording App
 
-import yaml
 import json
 import argparse
 
-
 # Function to read the main configuration file in JSON format
 def read_main_config_file_json(main_config_file: str) -> dict:
     """
     Reads main config file.
     """
+
     # read general parameter set from yaml config file
     with open(main_config_file, "r") as file:
         main_config = json.load(file)
@@ -20,22 +19,11 @@ def read_main_config_file_json(main_config_file: str) -> dict:
     return main_config
 
 
-# Function to read the main configuration file in YAML format
-def read_main_config_file(main_config_file: str) -> dict:
-    """
-    Reads main config file.
-    """
-    # read general parameter set from yaml config file
-    with open(main_config_file, "r") as file:
-        main_config = yaml.load(file, Loader=yaml.Loader)
-
-    return main_config
-
-
 # Function to parse the OAI T_messages file and get the index of a given string
 def parse_message_file(file_path):
     with open(file_path, "r") as file:
         content = file.readlines()
+
     # Extract lines that start with 'ID' and remove the 'ID = ' prefix
     tracer_msgs_identities = [
         line.strip().replace("ID = ", "")
@@ -65,12 +53,18 @@ def get_requested_tracer_msgs_indices(requested_tracer_messages, tracer_msgs_ide
     return req_tracer_msgs_indices
 
 
-# Function to get the data recording configuration
-def get_data_recording_config(config_meta_data):
-    parser = argparse.ArgumentParser(description="request messages IDs")
-    ue_args = parser.parse_args()
-    gnb_args = parser.parse_args()
+# Helper: return the first node in the nodes list whose type matches the given type string
+def get_node_by_type(nodes, node_type):
+    for node in nodes:
+        if node.get("type") == node_type:
+            return node
+    return None
 
+
+# Function to get the data recording configuration.
+# Returns (config_meta_data, node_args_dict) where node_args_dict is
+# { node_id: Namespace } with one entry per node defined in the config.
+def get_data_recording_config(config_meta_data):
     # get Tracer Messages IDs from the T-Tracer Messages file
     #   MSG list order should be similar to the T-Tracer Messages txt file
     #   Be sure that you are using the same T_messages file that is used in the OAI project
@@ -81,44 +75,33 @@ def get_data_recording_config(config_meta_data):
         config_meta_data["data_recording_config"]["t_tracer_message_definition_file"])
     config_meta_data["data_recording_config"]["tracer_msgs_identities"] = tracer_msgs_identities
 
-    # get requested tracer messages indices for gNB
-    gnb_args.requested_tracer_messages = \
-        config_meta_data["data_recording_config"]["base_station"]["requested_tracer_messages"]
-    # get requested tracer messages indices for UE
-    ue_args.requested_tracer_messages = \
-        config_meta_data["data_recording_config"]["user_equipment"]["requested_tracer_messages"]
+    nodes = config_meta_data["data_recording_config"]["nodes"]
 
-    # check if gnb_requested_tracer_messages is not empty
-    if gnb_args.requested_tracer_messages:
-        config_meta_data["data_recording_config"]["base_station"][
-            "req_tracer_msgs_indices"] = get_requested_tracer_msgs_indices(
-            gnb_args.requested_tracer_messages, tracer_msgs_identities)
+    # Build per-node args generically
+    node_args_dict = {}
+    for node in nodes:
+        parser = argparse.ArgumentParser(description="request messages IDs")
+        node_args = parser.parse_args()
 
-        # get gNB Trace Messages
-        gnb_args.num_records = config_meta_data["data_recording_config"]["num_records"]
-        gnb_args.start_frame_number = config_meta_data["data_recording_config"]["start_frame_number"]
-        gnb_args.req_tracer_msgs_indices = \
-            config_meta_data["data_recording_config"]["base_station"]["req_tracer_msgs_indices"]
-        gnb_args.num_requested_tracer_msgs = len(gnb_args.req_tracer_msgs_indices)
-        # Split the string into IP and port
-        gnb_args.IPaddress, gnb_args.port = config_meta_data["data_recording_config"][
-            "tracer_service_baseStation_address"].split(":")
-        gnb_args.bytes_IPaddress = bytes(gnb_args.IPaddress, "utf-8")
+        node_id = node.get("id")
+        node_args.node_id = node_id
+        node_args.type = node.get("type")
+        node_args.requested_tracer_messages = node.get("requested_tracer_messages", [])
 
-    # check if ue_requested_tracer_messages is not empty
-    if ue_args.requested_tracer_messages:
-        config_meta_data["data_recording_config"]["user_equipment"][
-            "req_tracer_msgs_indices"] = get_requested_tracer_msgs_indices(
-            ue_args.requested_tracer_messages, tracer_msgs_identities)
+        if node_args.requested_tracer_messages:
+            node["req_tracer_msgs_indices"] = get_requested_tracer_msgs_indices(
+                node_args.requested_tracer_messages, tracer_msgs_identities)
 
-        # get UE Trace Messages
-        ue_args.num_records = config_meta_data["data_recording_config"]["num_records"]
-        ue_args.start_frame_number = config_meta_data["data_recording_config"]["start_frame_number"]
-        ue_args.req_tracer_msgs_indices = \
-            config_meta_data["data_recording_config"]["user_equipment"]["req_tracer_msgs_indices"]
-        ue_args.num_requested_tracer_msgs = len(ue_args.req_tracer_msgs_indices)
-        ue_args.IPaddress, ue_args.port = config_meta_data["data_recording_config"][
-            "tracer_service_userEquipment_address"].split(":")
-        ue_args.bytes_IPaddress = bytes(ue_args.IPaddress, "utf-8")
+            node_args.num_records = config_meta_data["data_recording_config"]["num_records"]
+            node_args.start_frame_number = config_meta_data["data_recording_config"]["start_frame_number"]
+            node_args.req_tracer_msgs_indices = node["req_tracer_msgs_indices"]
+            node_args.num_requested_tracer_msgs = len(node_args.req_tracer_msgs_indices)
+            # Split the string into IP and port
+            tracer_addr = node.get("tracer_service_address", "")
+            if tracer_addr:
+                node_args.IPaddress, node_args.port = tracer_addr.split(":")
+                node_args.bytes_IPaddress = bytes(node_args.IPaddress, "utf-8")
 
-    return config_meta_data, gnb_args, ue_args
+        node_args_dict[node_id] = node_args
+
+    return config_meta_data, node_args_dict

common/utils/data_recording/lib/data_recording_messages_def.py

@@ -3,6 +3,32 @@
 # ---------------------------------------------------------------------
 # brief defination of captured data recording messages
 
+# Get Common Sync Header - number of bytes
+def get_sync_header_msg_list():
+    """
+    shared memory layout written from the app:
+    =================================
+    msg_id                  (uint8)  message type ID
+    frame                   (uint16)
+    slot                    (uint8)
+    unix_capture_ts_sec       (uint32)  Unix epoch seconds
+    unix_capture_ts_nsec      (uint32)  nanoseconds [0, 999999999]
+
+    """
+    # Get Common Sync Header - number of bytes
+    sync_header_msg = {
+        "msg_id": 2,
+        "frame": 2,
+        "slot": 1,
+        "unix_capture_ts_sec": 4,
+        "unix_capture_ts_nsec": 4,
+    }
+    # initial number of bytes to read to get data
+    sync_header_msg_length = 0
+    for key, value in sync_header_msg.items():
+        sync_header_msg_length = sync_header_msg_length + value
+    return sync_header_msg, sync_header_msg_length
+
 # Data Collection Trace Messages - General message structure - number of bytes
 def get_general_msg_header_list():
     """
@@ -11,8 +37,8 @@ def get_general_msg_header_list():
     msg_id                  (uint8)  message type ID
     frame                   (uint16)
     slot                    (uint8)
-    datetime_yyyymmdd       (uint32)
-    datetime_hhmmssmmm      (uint32)
+    unix_capture_ts_sec     (uint32)  Unix epoch seconds
+    unix_capture_ts_nsec    (uint32)  nanoseconds [0, 999999999]
     frame_type              (uint8)
     freq_range              (uint8)
     subcarrier_spacing      (uint8)
@@ -46,8 +72,8 @@ def get_general_msg_header_list():
         "msg_id": 2,
         "frame": 2,
         "slot": 1,
-        "datetime_yyyymmdd": 4,
-        "datetime_hhmmssmmm": 4,
+        "unix_capture_ts_sec": 4,
+        "unix_capture_ts_nsec": 4,
         "frame_type": 1,
         "freq_range": 1,
         "subcarrier_spacing": 1,

common/utils/data_recording/lib/shared_memory_interface.py

@@ -0,0 +1,348 @@
+# SPDX-License-Identifier: LicenseRef-CSSL-1.0
+
+# ---------------------------------------------------------------------
+# brief Shared Memory Interface for Data Recording App
+
+import sysv_ipc as ipc
+import time
+import struct
+from lib import data_recording_messages_def
+import numpy as np
+from lib import common_utils
+
+DEBUG_WIRELESS_RECORDED_DATA = True
+DEBUG_BUFFER_READING = False
+
+# initialize shared memory
+def attach_shm(shm_path, project_id):
+    key = ipc.ftok(shm_path, project_id)
+    shm = ipc.SharedMemory(key, 0, 0)
+    shm.attach(0, 0)
+    return shm
+
+
+def detach_shm(shm):
+    try:
+        shm.detach()
+        print("Shared memory detached successfully.")
+    except ipc.ExistentialError:
+        print("Shared memory segment does not exist.")
+
+def remove_shm(shm):
+    try:
+        shm.remove()
+        print("Shared memory removed successfully.")
+    except ipc.ExistentialError:
+        print("Shared memory segment does not exist.")
+
+
+# Write Tracer Control Message
+def write_shm(shm, args):
+    # Note: Big Endian >, Little Endian <
+    # Note: unsigned char B, signed char b, short h, int I, long long q
+    # Note: float f, double d, string s,  char c, bool ?
+    # 1: config
+    # 2: record
+    # 3: quit
+    print(f"  [Shared Memory] Write action: {args.action}")
+    if args.action == "config":
+        # Determine the length of the IP address
+        ip_length = len(args.bytes_IPaddress) + 1  # String terminator
+        # Construct the format string dynamically
+        format_string = f"{ip_length}s"
+        shm.write(
+            # Config action
+            struct.pack("<B", 1) +
+            struct.pack("<B", ip_length) +
+            struct.pack(format_string, args.bytes_IPaddress) +
+            struct.pack("<h", int(args.port))
+        )
+        print(f"  [T-Tracer] Config: IP={args.bytes_IPaddress}, port={args.port}")
+    elif args.action == "record":
+        shm.write(
+                # Record action
+                struct.pack("<B", int(2)) +
+                struct.pack("<B", args.num_requested_tracer_msgs) +
+                struct.pack("<{}h".format(len(args.req_tracer_msgs_indices)),
+                            *args.req_tracer_msgs_indices,) +
+                struct.pack("<I", args.num_records) +
+                struct.pack("<h", args.start_frame_number)
+        )
+        print(f"  [T-Tracer] Record: num_msgs={args.num_requested_tracer_msgs}, "
+              f"msg_ids={args.req_tracer_msgs_indices}, "
+              f"num_records={args.num_records}, "
+              f"start_frame={args.start_frame_number}")
+    elif args.action == "quit":
+        shm.write(struct.pack("<B", int(3)))  # Quit action
+        print("  [T-Tracer] Quit")
+    else:
+        print("Unknown action for data recording system!")
+
+
+# write shared memory task
+def write_shm_task(barrier, shm_id, args):
+    # Wait for threads to be ready
+    barrier.wait()
+    # send request to related T-Tracer Application
+    write_shm(shm_id, args)
+
+# check data if avalible in the shared memory
+def is_data_available_in_memory(shm, bufIdx, general_message_header_length, timeout=10):
+    start_time = time.time()
+    while True:
+        buf = shm.read(bufIdx + general_message_header_length)
+                # Only check bytes in the range [bufIdx, bufIdx + header_length),
+        # not the entire buffer from 0 — earlier records would make sum > 0
+        n_bytes = sum(buf[bufIdx:bufIdx + general_message_header_length])
+        if n_bytes > 0:
+            if DEBUG_BUFFER_READING:
+                print("Data in memory: ", n_bytes, " bytes")
+            return True
+        if (time.time() - start_time) > timeout:
+            break
+        print("Data Recording App: Waiting for Measurements!")
+        time.sleep(1)
+    return False
+
+# Read data from gNB T-tracer Application
+def get_frame_slot_start(shm_reading, bufIdx, general_msg_header_list,
+                         general_message_header_length):
+    buf = shm_reading.read(bufIdx + general_message_header_length)
+    msg_id = struct.unpack("<H", buf[bufIdx: bufIdx + general_msg_header_list.get("msg_id")])[0]
+    bufIdx += general_msg_header_list.get("msg_id")
+    frame = struct.unpack("<H", buf[bufIdx: bufIdx + general_msg_header_list.get("frame")])[0]
+    bufIdx += general_msg_header_list.get("frame")
+    slot = struct.unpack("B", buf[bufIdx: bufIdx + general_msg_header_list.get("slot")])[0]
+    return frame, slot
+
+# get MSG header Info
+def get_msg_header(shm_reading, bufIdx, sync_header_msg, sync_header_length):
+    """Read the sync header from shared memory and return msg_id, frame, slot, and timestamp.
+
+    Args:
+        shm_reading: Shared memory handle.
+        bufIdx: Current buffer index.
+        sync_header_msg: Dict with field names and byte sizes (from get_sync_header_msg_list).
+        sync_header_length: Total byte length of the sync header.
+
+    Returns:
+        tuple: (msg_id, frame, slot, unix_capture_ts_sec, unix_capture_ts_nsec)
+    """
+    buf = shm_reading.read(bufIdx + sync_header_length)
+    offset = bufIdx
+    msg_id    = struct.unpack('<H', buf[offset:offset + sync_header_msg["msg_id"]])[0]
+    offset += sync_header_msg["msg_id"]
+    frame     = struct.unpack('<H', buf[offset:offset + sync_header_msg["frame"]])[0]
+    offset += sync_header_msg["frame"]
+    slot      = struct.unpack('B',  buf[offset:offset + sync_header_msg["slot"]])[0]
+    offset += sync_header_msg["slot"]
+    unix_capture_ts_sec  = struct.unpack('<I', buf[offset:offset + sync_header_msg["unix_capture_ts_sec"]])[0]
+    offset += sync_header_msg["unix_capture_ts_sec"]
+    unix_capture_ts_nsec = struct.unpack('<I', buf[offset:offset + sync_header_msg["unix_capture_ts_nsec"]])[0]
+    return msg_id, frame, slot, unix_capture_ts_sec, unix_capture_ts_nsec
+
+
+# Get MSG ID from Shared memory
+def get_msg_id_from_shm(shm_reading, bufIdx, msg_id_length):
+    buf = shm_reading.read(bufIdx + msg_id_length)
+    msg_id = struct.unpack("<H", buf[bufIdx: bufIdx + msg_id_length])[0]
+    return msg_id
+
+# Read data from Shared memory based Data Conversion Service message structure
+def read_msg_data_from_shm(shm, bufIdx, tracer_msgs_identities):
+    # read sync header to move buffer index
+    # get general message header list
+    general_msg_header_list, general_message_header_length = \
+        data_recording_messages_def.get_general_msg_header_list()
+    buf = shm.read(bufIdx + general_message_header_length)
+    n_bytes = sum(buf)
+    if n_bytes == 0:
+        raise Exception('ERROR: No data available in memory')
+    msg_id = struct.unpack('<H', buf[bufIdx:bufIdx+general_msg_header_list.get("msg_id")])[0]
+    bufIdx += general_msg_header_list.get("msg_id")
+    frame = struct.unpack('<H', buf[bufIdx:bufIdx+general_msg_header_list.get("frame")])[0]
+    bufIdx += general_msg_header_list.get("frame")
+    slot = struct.unpack('B', buf[bufIdx:bufIdx+general_msg_header_list.get("slot")])[0]
+    bufIdx += general_msg_header_list.get("slot")
+    # get Unix capture timestamp (sec, nsec) — Unix epoch 1970
+    unix_capture_ts_sec = \
+        struct.unpack('<I', buf[bufIdx:bufIdx+general_msg_header_list.get("unix_capture_ts_sec")])[0]
+    bufIdx += general_msg_header_list.get("unix_capture_ts_sec")
+    unix_capture_ts_nsec = \
+        struct.unpack('<I', buf[bufIdx:bufIdx+general_msg_header_list.get("unix_capture_ts_nsec")])[0]
+    bufIdx += general_msg_header_list.get("unix_capture_ts_nsec")
+    # Derive timestamp in seconds and nanoseconds (already in Unix epoch)
+    timestamp_seconds = unix_capture_ts_sec
+    timestamp_nseconds = unix_capture_ts_nsec
+    # get frame type
+    frame_type = struct.unpack('B', buf[bufIdx:bufIdx+general_msg_header_list.get("frame_type")])[0]
+    bufIdx += general_msg_header_list.get("frame_type")
+    # get frequency range
+    freq_range = struct.unpack('B', buf[bufIdx:bufIdx+general_msg_header_list.get("freq_range")])[0]
+    bufIdx += general_msg_header_list.get("freq_range")
+    # get subcarrier spacing
+    subcarrier_spacing = struct.unpack('B', buf[bufIdx:bufIdx+general_msg_header_list.get("subcarrier_spacing")])[0]
+    bufIdx += general_msg_header_list.get("subcarrier_spacing")
+    # get cyclic prefix
+    cyclic_prefix = struct.unpack('B', buf[bufIdx:bufIdx+general_msg_header_list.get("cyclic_prefix")])[0]
+    bufIdx += general_msg_header_list.get("cyclic_prefix")
+    # get symbols per slot
+    symbols_per_slot = struct.unpack('B', buf[bufIdx:bufIdx+general_msg_header_list.get("symbols_per_slot")])[0]
+    bufIdx += general_msg_header_list.get("symbols_per_slot")
+    # get Nid cell
+    Nid_cell = struct.unpack('<H', buf[bufIdx:bufIdx+general_msg_header_list.get("Nid_cell")])[0]
+    bufIdx += general_msg_header_list.get("Nid_cell")
+    # get rnti
+    rnti = struct.unpack('<H', buf[bufIdx:bufIdx+general_msg_header_list.get("rnti")])[0]
+    bufIdx += general_msg_header_list.get("rnti")
+    # get rb size
+    rb_size = struct.unpack('<H', buf[bufIdx:bufIdx+general_msg_header_list.get("rb_size")])[0]
+    bufIdx += general_msg_header_list.get("rb_size")
+    # get rb start
+    rb_start = struct.unpack('<H', buf[bufIdx:bufIdx+general_msg_header_list.get("rb_start")])[0]
+    bufIdx += general_msg_header_list.get("rb_start")
+    # get start symbol index    
+    start_symbol_index = struct.unpack('B', buf[bufIdx:bufIdx+general_msg_header_list.get("start_symbol_index")])[0]
+    bufIdx += general_msg_header_list.get("start_symbol_index")
+    # get number of symbols
+    nr_of_symbols = struct.unpack('B', buf[bufIdx:bufIdx+general_msg_header_list.get("nr_of_symbols")])[0]
+    bufIdx += general_msg_header_list.get("nr_of_symbols")
+    # get qam modulation order
+    qam_mod_order = struct.unpack('B', buf[bufIdx:bufIdx+general_msg_header_list.get("qam_mod_order")])[0]
+    bufIdx += general_msg_header_list.get("qam_mod_order")
+    # get mcs index
+    mcs_index = struct.unpack('B', buf[bufIdx:bufIdx+general_msg_header_list.get("mcs_index")])[0]
+    bufIdx += general_msg_header_list.get("mcs_index")
+    # get mcs table
+    mcs_table = struct.unpack('B', buf[bufIdx:bufIdx+general_msg_header_list.get("mcs_table")])[0]
+    bufIdx += general_msg_header_list.get("mcs_table")
+    # get number of layers
+    nrOfLayers = struct.unpack('B', buf[bufIdx:bufIdx+general_msg_header_list.get("nrOfLayers")])[0]
+    bufIdx += general_msg_header_list.get("nrOfLayers")
+    # get transform precoding
+    transform_precoding = struct.unpack('B', buf[bufIdx:bufIdx+general_msg_header_list.get("transform_precoding")])[0]
+    bufIdx += general_msg_header_list.get("transform_precoding")
+    # get dmrs config type
+    dmrs_config_type = struct.unpack('B', buf[bufIdx:bufIdx+general_msg_header_list.get("dmrs_config_type")])[0]
+    bufIdx += general_msg_header_list.get("dmrs_config_type")
+    # get ul dmrs symb pos
+    ul_dmrs_symb_pos = struct.unpack('<H', buf[bufIdx:bufIdx+general_msg_header_list.get("ul_dmrs_symb_pos")])[0]
+    bufIdx += general_msg_header_list.get("ul_dmrs_symb_pos")
+    # get number dmrs symbols
+    number_dmrs_symbols = struct.unpack('B', buf[bufIdx:bufIdx+general_msg_header_list.get("number_dmrs_symbols")])[0]
+    bufIdx += general_msg_header_list.get("number_dmrs_symbols")
+    # get dmrs port
+    dmrs_port = struct.unpack('<H', buf[bufIdx:bufIdx+general_msg_header_list.get("dmrs_port")])[0]
+    bufIdx += general_msg_header_list.get("dmrs_port")
+    # get dmrs scid
+    dmrs_scid = struct.unpack('<H', buf[bufIdx:bufIdx+general_msg_header_list.get("dmrs_scid")])[0]
+    bufIdx += general_msg_header_list.get("dmrs_scid")
+    # get nb antennas rx for gNB or nb antennas tx for UE
+    nb_antennas = struct.unpack('B', buf[bufIdx:bufIdx+general_msg_header_list.get("nb_antennas")])[0]
+    bufIdx += general_msg_header_list.get("nb_antennas")
+    # get number of bits
+    number_of_bits = struct.unpack('<I', buf[bufIdx:bufIdx+general_msg_header_list.get("number_of_bits")])[0]
+    bufIdx += general_msg_header_list.get("number_of_bits")
+    # get length of bytes
+    length_bytes = struct.unpack('<I', buf[bufIdx:bufIdx+general_msg_header_list.get("length_bytes")])[0]
+    bufIdx += general_msg_header_list.get("length_bytes")
+    # print all captured data
+    if DEBUG_WIRELESS_RECORDED_DATA:
+        print("\n" + "="*80)
+        print(f"UL CONFIG METADATA - {tracer_msgs_identities[msg_id]}")
+        print(f"Unix TS: {unix_capture_ts_sec}.{unix_capture_ts_nsec:09d}  |  MSG ID: {msg_id}")
+        print("-"*80)
+        print(f"Frame: {frame:<5} Slot: {slot:<5} Nid Cell: {Nid_cell:<5} RNTI: {rnti:<5}")
+        print(f"Frame Type: {frame_type}  Freq Range: {freq_range}  |  SCS: {subcarrier_spacing} Hz  Cyclic Prefix: {cyclic_prefix}  Symbols/Slot: {symbols_per_slot}")
+        print("-"*80)
+        print(f"RB: Start={rb_start:<4} Size={rb_size:<4}  |  Symbol Allocation: Start={start_symbol_index:<3} Count={nr_of_symbols:<3}")
+        print(f"MCS: Index={mcs_index:<3} Table={mcs_table}  |  QAM Order: {qam_mod_order}  Layers: {nrOfLayers}  Transform Precoding: {transform_precoding}")
+        print("-"*80)
+        print(f"DMRS: Config Type={dmrs_config_type}  Symb Pos={ul_dmrs_symb_pos} ({number_dmrs_symbols} symbols)")
+        print(f"      Port={dmrs_port}  SCID={dmrs_scid}  |  Antennas: {nb_antennas}")
+        print("-"*80)
+        print(f"Data: {number_of_bits} bits  ({length_bytes} bytes)")
+        print("="*80)
+    # raise exception if time stamp is zero, it means that the data is not recorded yet
+    if unix_capture_ts_sec == 0 and unix_capture_ts_nsec == 0:
+        raise Exception("ERROR: Time stamp is zero, data is not recorded yet or something wrong, check logs!")
+    # get recorded data
+    buf = shm.read(bufIdx + length_bytes)
+    captured_data = {}
+    # If message is bit message, store data in bytes
+    # then the field number_of_bits should be not zero
+    if "_BITS" in tracer_msgs_identities[msg_id]:
+        recorded_data = struct.unpack("<" + int(length_bytes) * 'B', buf[bufIdx:bufIdx + length_bytes])
+        bufIdx += length_bytes
+        # convert data in bytes to bits
+        bits_vector = []
+        for byte in recorded_data:
+            bits_vector.extend([int(bit) for bit in format(int(byte), '08b')])
+        captured_data["sigmf_data_type"] = "ri8_le"
+        # convert to uint8
+        captured_data["recorded_data"] = np.asarray(bits_vector).astype(np.uint8) 
+    else:
+        recorded_data = struct.unpack("<" + int(length_bytes/2) * 'h', buf[bufIdx:bufIdx + length_bytes])
+        bufIdx += length_bytes
+        # Convert real data to complext data
+        # converting list to array
+        recorded_data = np.asarray(recorded_data)
+        recorded_data_complex = common_utils.real_to_complex(recorded_data)
+        captured_data["sigmf_data_type"] = "cf32_le"
+        # convert to complex64
+        captured_data["recorded_data"] = recorded_data_complex.astype(np.complex64)
+    # store data in dictonary
+    captured_data["message_id"] = msg_id
+    captured_data["message_type"] = tracer_msgs_identities[msg_id]
+    captured_data["frame"] = frame
+    captured_data["slot"] = slot
+    captured_data["unix_capture_ts_sec"] = unix_capture_ts_sec
+    captured_data["unix_capture_ts_nsec"] = unix_capture_ts_nsec
+    captured_data["timestamp_seconds"] = timestamp_seconds
+    captured_data["timestamp_nseconds"] = timestamp_nseconds
+    captured_data["frame_type"] = frame_type
+    captured_data["freq_range"] = freq_range
+    captured_data["subcarrier_spacing"] = subcarrier_spacing
+    captured_data["cyclic_prefix"] = cyclic_prefix
+    # captured_data["symbols_per_slot"] = symbols_per_slot  ... not used
+    captured_data["Nid_cell"] = Nid_cell
+    captured_data["rnti"] = rnti
+    captured_data["rb_size"] = rb_size
+    captured_data["rb_start"] = rb_start
+    captured_data["start_symbol_index"] = start_symbol_index
+    captured_data["nr_of_symbols"] = nr_of_symbols
+    captured_data["qam_mod_order"] = qam_mod_order
+    captured_data["mcs_index"] = mcs_index
+    captured_data["mcs_table"] = mcs_table
+    captured_data["nrOfLayers"] = nrOfLayers
+    captured_data["transform_precoding"] = transform_precoding
+    captured_data["dmrs_config_type"] = dmrs_config_type
+    captured_data["ul_dmrs_symb_pos"] = ul_dmrs_symb_pos
+    captured_data["number_dmrs_symbols"] = number_dmrs_symbols
+    captured_data["dmrs_port"] = dmrs_port
+    captured_data["dmrs_scid"] = dmrs_scid
+    captured_data["nb_antennas"] = nb_antennas
+    captured_data["number_of_bits"] = number_of_bits
+    return captured_data, bufIdx
+
+# Read data from Shared memory based Data Conversion Service message structure
+def read_data_from_shm(shm, bufIdx, tracer_msgs_identities):
+    # print buffer index
+    if DEBUG_BUFFER_READING:
+        print("Buffer Index: ", bufIdx)
+    # Get MSG ID by reading Sync Header
+    sync_header_msg_list, sync_header_message_length = \
+        data_recording_messages_def.get_sync_header_msg_list()
+    msg_id_length = sync_header_msg_list.get("msg_id")
+    
+    msg_id = get_msg_id_from_shm(shm, bufIdx, msg_id_length) 
+    
+    if DEBUG_WIRELESS_RECORDED_DATA:
+        print(f"  [T-Tracer] Reading MSG ID: {msg_id} - {tracer_msgs_identities[msg_id]}")
+        # Get MSG data based on Data Conversion Service message structure
+        # UL IQ/bits messages (default UL data format)
+        
+    captured_data, bufIdx = read_msg_data_from_shm(shm, bufIdx, tracer_msgs_identities)
+    
+    return captured_data, bufIdx
+    

common/utils/data_recording/lib/sigmf_interface.py

@@ -6,11 +6,12 @@
 import os
 import sigmf
 from sigmf import SigMFFile
-# from sigmf.utils import get_data_type_str
-import numpy as np
-from datetime import datetime
-import yaml
+from datetime import datetime, timezone
 from sigmf import SigMFCollection
+from .wireless_parameters_mapper import map_waveform_metadata_to_wireless_dic, derive_remaining_5gnr_metadata, STANDARDS
+from .config_interface import get_node_by_type
+
+DEBUG_WIRELESS_RECORDED_DATA = True
 
 """
 SERIALIZATION_SCHEMES = {
@@ -24,21 +25,29 @@ SERIALIZATION_SCHEMES = {
     "tx-payload-bits": ["bits", "subcarriers", "ofdm_symbols"],
 }
 """
-STANDARDS = {"5gnr_oai": "5gnr"}
 
 
-def time_stamp_formating(time_stamp, datetime_offset):
-    # Parse the input string into a datetime object
-    time_stamp_ms_obj = datetime.strptime(time_stamp, "%Y%m%d_%H%M%S%f")
+def time_stamp_formating(unix_capture_ts_sec, unix_capture_ts_nsec, datetime_offset):
+    # Convert Unix epoch (sec, nsec) to a datetime object (UTC).
+    # Python datetime only supports µs, so we format the nanosecond
+    # fractional part manually to preserve the full 9-digit resolution.
+    dt_obj = datetime.fromtimestamp(unix_capture_ts_sec, tz=timezone.utc)
 
-    # Format the datetime object into the desired output format with milliseconds
-    time_stamp_ms_iso = (
-        time_stamp_ms_obj.strftime("%Y_%m_%dT%H:%M:%S.%f")[:-3] + datetime_offset
+    # Format with nanosecond fractional seconds (9 digits)
+    time_stamp_ns_iso = (
+        dt_obj.strftime("%Y_%m_%dT%H:%M:%S")
+        + f".{unix_capture_ts_nsec:09d}"
+        + datetime_offset
     )
-    time_stamp_ms = time_stamp_ms_iso
-    time_stamp_ms_file_name = time_stamp_ms_iso.replace(":", "_").replace(".", "_")
+    time_stamp_ns_file_name = time_stamp_ns_iso.replace(":", "_").replace(".", "_")
 
-    return time_stamp_ms, time_stamp_ms_file_name
+    return time_stamp_ns_iso, time_stamp_ns_file_name
+
+
+def _get_node_meta_data(config_meta_data, role):
+    """Return the meta_data dict for the first node whose type matches the given type string."""
+    node = get_node_by_type(config_meta_data["data_recording_config"]["nodes"], role)
+    return node["meta_data"] if node else {}
 
 
 def create_serialization_metadata(serialization_scheme, data_source: str,
@@ -46,6 +55,10 @@ def create_serialization_metadata(serialization_scheme, data_source: str,
     """Creates dict that specifies the serialization metadata."""
 
     # retrieve parameter from LinkSimulator config
+    # if serialization_scheme empty, return empty dict, It is a meta-data only message
+    if not serialization_scheme:
+        return {}
+
     if data_source == "5gnr_oai":
         num_ofdm_symbol = link_sim_parameters["nr_of_symbols"]
         num_subcarriers = link_sim_parameters["rb_size"] * 12
@@ -72,131 +85,73 @@ def create_serialization_metadata(serialization_scheme, data_source: str,
 
     return serialization_dict
 
-
-def map_metadata_to_sigmf_format(scope, waveform_generator, parameter_map_file, captured_data):
-    """
-    Maps metadata from Waveform creator to API and SigMF format.
-    The used parameters and the mapping pairs are specified in a separate YAML file
-    that must be provided as well.
-    """
-    # read waveform parameter map from yaml file
-    dir_path = os.path.dirname(__file__)
-    src_path = os.path.split(dir_path)[0]
-
-    with open(os.path.join(src_path, parameter_map_file), "r") as file:
-        parameter_map_dic = yaml.load(file, Loader=yaml.Loader)
-    # preallocate target dict
-    sigmf_metadata_dict = {}
-
-    #  get standard and name of generator
-    standard_key = parameter_map_dic["waveform_generator"][waveform_generator]
-    generator = standard_key["generator"]
-
-    if scope == "tx":
-        parameter_map_dic = parameter_map_dic["transmitter"][
-            STANDARDS[waveform_generator]
-        ]
-    elif scope == "channel":
-        parameter_map_dic = parameter_map_dic["channel"]
-    elif scope == "rx":
-        parameter_map_dic = parameter_map_dic["receiver"]
-    else:
-        raise Exception(
-            f"Invalid mapping scope '{scope}'! Only 'tx', 'channel' and 'rx' are valid!"
-        )
-
-    # check if standard key is given
-    if parameter_map_dic is None:
-        raise Exception(
-            "Invalid standard key: "
-            "Name should be corrected or added to wireless_link_parameter_map.yaml, given: "
-            f"{waveform_generator}"
-        )
-
-    for parameter_pair in parameter_map_dic:
-        # check if key for chosen simulator even exists
-        if waveform_generator + "_parameter" in parameter_pair.keys():
-            # only continue with mapping from file if direct equivalent exists
-            if parameter_pair[waveform_generator + "_parameter"]["name"]:
-                # It is not necessary to get all parameters from wireless_link_parameter_map.yaml
-                # in captured_data since some parameters related to DL or UL only
-                if (parameter_pair[waveform_generator + "_parameter"]["name"] in captured_data.keys()):
-                    # extract value from waveform config source
-                    value = captured_data[
-                        parameter_pair[waveform_generator + "_parameter"]["name"]]
-                    # additional mapping if parameter values should come from a discrete set of values
-                    if ("value_map" in parameter_pair[waveform_generator + "_parameter"].keys()):
-                        value = parameter_pair[waveform_generator + "_parameter"]["value_map"][value]
-                    # write to target dictionary for SigMF
-                    sigmf_metadata_dict[parameter_pair["sigmf_parameter_name"]] = value
-            else:
-                raise Exception(
-                    f"Incomplete specification in field '{waveform_generator}_parameter'!")
-        # else:  # fill_non_explicit_fields
-        #     waveform_config[parameter_pair["sigmf_parameter_name"]] = "none"
-    if not sigmf_metadata_dict:
-        raise Exception(
-            """ERROR: Check captured meta-data or provided config and meta data """)
-
-    # check for non-JSON-serializable data types
-    def isfloat(NumberString):
-        try:
-            float(NumberString)
-            return True
-        except ValueError:
-            return False
-
-    for key, value in sigmf_metadata_dict.items():
-        if isinstance(value, np.integer):
-            sigmf_metadata_dict[key] = int(value)
-        elif isinstance(value, (np.float16, np.float32, np.float64)):
-            sigmf_metadata_dict[key] = np.format_float_positional(value, trim="-")
-        elif isinstance(value, int):
-            sigmf_metadata_dict[key] = int(value)
-        elif isinstance(value, float):
-            # store value in decimal and not in scientific notation
-            sigmf_metadata_dict[key] = float(value)
-        elif isinstance(value, str) and key != "standard":
-            # convert string to lower case
-            sigmf_metadata_dict[key] = value.lower()
-            if isfloat(value):
-                if value.isdigit():
-                    sigmf_metadata_dict[key] = int(float(value))
-                elif value.replace(".", "", 1).isdigit() and value.count(".") < 2:
-                    sigmf_metadata_dict[key] = float(value)
-
-    return sigmf_metadata_dict, generator
-
-
-def create_system_components_metadata(waveform_generator, parameter_map_file, captured_data):
+# map OAI config data to Wireless Dictionary Parameter Map - SigMF metadata
+def create_system_components_metadata(captured_data, config_meta_data):
     """Creates system components (TX, channel, RX) metadata that will reside in the annotations."""
-    # map metadata of waveform generator to SigMF format
-    signal_info, generator = map_metadata_to_sigmf_format(
-        "tx", waveform_generator, parameter_map_file, captured_data
-    )
+
+    # map OAI config data to Wireless Dictionary Parameter Map - SigMF metadata
+    waveform_generator = config_meta_data["data_recording_config"]["global_info"][
+        "waveform_generator"]
+    parameter_map_file = config_meta_data["data_recording_config"]["parameter_map_file"]
+
+    # map metadata of waveform generator to Wireless Dictionary Parameter Map - SigMF metadata
+    signal_metadata, generator = map_waveform_metadata_to_wireless_dic(
+        "tx", waveform_generator, parameter_map_file, captured_data)
+
+    # derive remaining Signal metadata
+    signal_metadata = derive_remaining_5gnr_metadata(captured_data, signal_metadata)
+
     tx_metadata = {
         "signal:detail": {
             "standard": STANDARDS[waveform_generator],
             "generator": generator,
-            STANDARDS[waveform_generator]: signal_info,
+            STANDARDS[waveform_generator]: signal_metadata,
         }
     }
 
     channel_metadata = {}   # will be filled later
     rx_metadata = {}        # will be filled later
 
-    return tx_metadata, channel_metadata, rx_metadata
+    # get meta data from config file
+    base_station_meta_data    = _get_node_meta_data(config_meta_data, "gnb")
+    user_equipment_meta_data  = _get_node_meta_data(config_meta_data, "ue")
 
+    # Set number of antennas based on link direction and captured side
+    # Find number of antennas from global info on other side of the link
+    if "UL" in captured_data["message_type"]:
+        # On UE side, It is TX antennas
+        # On gNB side, It is RX antennas
+        if "UE" in captured_data["message_type"]:
+            tx_metadata["signal:detail"][STANDARDS[waveform_generator]][
+                "num_tx_antennas"] = captured_data["nb_antennas"]
+            rx_metadata.update({"num_rx_antennas": base_station_meta_data["num_rx_antennas"]})
+        else:
+            tx_metadata["signal:detail"][STANDARDS[waveform_generator]][
+                "num_tx_antennas"] = user_equipment_meta_data["num_tx_antennas"]
+            rx_metadata.update({"num_rx_antennas": captured_data["nb_antennas"]})
+
+    if "DL" in captured_data["message_type"]:
+        tx_metadata["signal:detail"][STANDARDS[waveform_generator]][
+            "link_direction"] = "downlink"
+        # On UE side, It is RX antennas
+        # On gNB side, It is TX antennasr
+        if "UE" in captured_data["message_type"]:
+            rx_metadata["signal:detail"][STANDARDS[waveform_generator]]["num_rx_antennas"] = \
+                captured_data["nb_antennas"]  ## TO DO check key name: no DL message from UE yet
+            tx_metadata.update({"num_tx_antennas": base_station_meta_data["num_tx_antennas"]})
+        else:
+            tx_metadata["signal:detail"][STANDARDS[waveform_generator]][
+                "num_tx_antennas"] = captured_data["nrOfLayers"]  
+            rx_metadata.update({"num_rx_antennas": user_equipment_meta_data["num_rx_antennas"]})
+    return tx_metadata, channel_metadata, rx_metadata
 
 def write_recorded_data_to_sigmf(captured_data, config_meta_data, global_info, idx):
     """
     Compiles and saves provided data and metadata into SigMF file format.
     """
     # get meta data from config file
-    base_station_meta_data = config_meta_data["data_recording_config"]["base_station"][
-        "meta_data"]
-    user_equipment_meta_data = config_meta_data["data_recording_config"][
-        "user_equipment"]["meta_data"]
+    base_station_meta_data    = _get_node_meta_data(config_meta_data, "gnb")
+    user_equipment_meta_data  = _get_node_meta_data(config_meta_data, "ue")
 
     # Check the receive target path is valid, else create folder
     data_storage_path = config_meta_data["data_recording_config"]["data_storage_path"]
@@ -206,111 +161,34 @@ def write_recorded_data_to_sigmf(captured_data, config_meta_data, global_info, i
 
     # Write recorded data to file
     # Get time stamp
-    time_stamp_ms, time_stamp_ms_file_name = time_stamp_formating(
-        captured_data["time_stamp"], global_info["datetime_offset"])
+    time_stamp, time_stamp_file_name = time_stamp_formating(
+        captured_data["unix_capture_ts_sec"], captured_data["unix_capture_ts_nsec"],
+        global_info["datetime_offset"])
     # Map OAI Message Name to SigMF Message Name
     file_name_prefix = config_meta_data["data_recording_config"][
         "supported_oai_tracer_messages"][captured_data["message_type"]]["file_name_prefix"]
     recorded_data_file_name = (
-        file_name_prefix + "-rec-" + str(idx) + "-" + time_stamp_ms_file_name)
+        file_name_prefix + "-rec-" + str(idx) + "-" + time_stamp_file_name)
 
     dataset_filename = recorded_data_file_name + ".sigmf-data"
     dataset_file_path = os.path.join(data_storage_path, dataset_filename)
-    print(dataset_file_path)
+    if DEBUG_WIRELESS_RECORDED_DATA:
+        print(dataset_file_path)
     captured_data["recorded_data"].tofile(dataset_file_path)
 
-    # map OAI config data to SigMF metadata
+    # ----------------------------------------------------
+    # map OAI config data to Wireless Dictionary Parameter Map - SigMF metadata
     waveform_generator = config_meta_data["data_recording_config"]["global_info"][
         "waveform_generator"]
-    parameter_map_file = config_meta_data["data_recording_config"]["parameter_map_file"]
-    tx_metadata, channel_metadata, rx_metadata = create_system_components_metadata(
-        waveform_generator, parameter_map_file, captured_data)
-
-    # Add other OAI metadata to SigMF metadata
-    # ----------------------------------------------------
-    # Set 1: Parameters needs to be derived from OAI message
-    # ----------------------------------------------------
-    # PUSCH DMRS: OFDM symbol start index within slot
-    get_pusch_dmrs_start_ofdm_symbol = 0
-    tx_metadata["signal:detail"][STANDARDS[waveform_generator]][
-        "pusch_dmrs:ofdm_symbol_idx"] = []
-
-    for symbol in range(captured_data["start_symbol_index"],
-                        captured_data["start_symbol_index"] + captured_data["nr_of_symbols"]):
-
-        dmrs_symbol_flag = (captured_data["ul_dmrs_symb_pos"] >> symbol) & 0x01
-        if dmrs_symbol_flag and not get_pusch_dmrs_start_ofdm_symbol:
-            get_pusch_dmrs_start_ofdm_symbol = 1
-            tx_metadata["signal:detail"][STANDARDS[waveform_generator]][
-                "pusch_dmrs:start_ofdm_symbol"] = symbol
-        if dmrs_symbol_flag:
-            tx_metadata["signal:detail"][STANDARDS[waveform_generator]][
-                "pusch_dmrs:ofdm_symbol_idx"].append(symbol)  # Append symbol to the list
-
-    tx_metadata["signal:detail"][STANDARDS[waveform_generator]][
-        "pusch_dmrs:duration_num_ofdm_symbols"] = 1
-    tx_metadata["signal:detail"][STANDARDS[waveform_generator]][
-        "pusch_dmrs:num_add_positions"] = (captured_data["number_dmrs_symbols"] - 1)
-
-    # If the message is a BIT message, add number of bits to the signal info
-    # "UE_PHY_UL_SCRAMBLED_TX_BITS", "GNB_PHY_UL_PAYLOAD_RX_BITS", "UE_PHY_UL_PAYLOAD_TX_BITS"
-    if "_BITS" in captured_data["message_type"]:
-        # if "BIT" in captured_data["message_type"]:
-        tx_metadata["signal:detail"][STANDARDS[waveform_generator]]["num_bits"] = (
-            captured_data["number_of_bits"])
-
-    # If the message is captured from UE, it is UPLink message,
-    # so number of antennas is num of Tx antennas
-    # number of antennas on Rx side should be read from global_info given by user
-    if "UE" in captured_data["message_type"]:
-        tx_metadata["signal:detail"][STANDARDS[waveform_generator]][
-            "num_tx_antennas"
-        ] = captured_data["nb_antennas"]
-        rx_metadata.update(
-            {"num_rx_antennas": base_station_meta_data["num_rx_antennas"]}
-        )
-    # If the message is captured from gNB, it is UPLink message,
-    # so number of antennas is num of Rx antennas
-    # number of antennas on Tx side should be read from global_info given by user
-    else:
-        tx_metadata["signal:detail"][STANDARDS[waveform_generator]][
-            "num_tx_antennas"] = user_equipment_meta_data["num_tx_antennas"]
-        rx_metadata.update({"num_rx_antennas": captured_data["nb_antennas"]})
-
-    # Check if the message type contains "UL"
-    if "UL" in captured_data["message_type"]:
-        tx_metadata["signal:detail"][STANDARDS[waveform_generator]][
-            "link_direction"] = "uplink"
-    else:
-        tx_metadata["signal:detail"][STANDARDS[waveform_generator]][
-            "link_direction"] = "downlink"
-    # ----------------------------------------------------
-    # Set 2: Parameters that is hardcoded
-    # ----------------------------------------------------
-    tx_metadata["signal:detail"][STANDARDS[waveform_generator]][
-        "pusch:content"] = "compliant"
-    tx_metadata["signal:detail"][STANDARDS[waveform_generator]][
-        "pusch:mapping_type"] = "B"
-    tx_metadata["signal:detail"][STANDARDS[waveform_generator]]["num_slots"] = 1
-    if config_meta_data["test_config"]["test_mode"] == "rf_simulation":
-        tx_metadata["signal:detail"][STANDARDS[waveform_generator]][
-            "pusch:payload_bit_pattern"] = "random"
-    elif config_meta_data["test_config"]["test_mode"] == "rf_real_time":
-        tx_metadata["signal:detail"][STANDARDS[waveform_generator]][
-            "pusch:payload_bit_pattern"] = "zeros"
-    else:
-        raise Exception("ERROR: Test mode is not supported in SigMF formatting")
+    tx_metadata, channel_metadata, rx_metadata = create_system_components_metadata(captured_data, config_meta_data)
+    
     # ----------------------------------------------------
     # Read mean parameters
     freq = config_meta_data["environment_emulation"]["target_link_config"][
         "wireless_channel"]["carrierFreqValueList_hz"][0]
     bandwidth = config_meta_data["data_recording_config"]["common_meta_data"][
         "bandwidth"]
-    # used_signal_bandwidth = (
-    #    pow(10, 6) * config_meta_data["environment_emulation"]["target_link_config"]["ran_config"][
-    #        "uplink"]["ul_used_signal_BW_MHz"])
-    sample_rate = config_meta_data["data_recording_config"]["common_meta_data"][
-        "sample_rate"]
+    sample_rate = config_meta_data["data_recording_config"]["common_meta_data"]["sample_rate"]
 
     # get signal emitter info
     signal_emitter = {
@@ -322,23 +200,14 @@ def write_recorded_data_to_sigmf(captured_data, config_meta_data, global_info, i
         "sample_rate": sample_rate,
         "bandwidth": bandwidth,
         "gain_tx": user_equipment_meta_data["tx_gain"],
-        "clock_reference": config_meta_data["data_recording_config"][
-            "common_meta_data"
-        ]["clock_reference"],
+        "clock_reference": config_meta_data["data_recording_config"]["common_meta_data"]["clock_reference"],
         }
 
     # ----------------------------------------------------
     # get channel metadata
     # ----------------------------------------------------
-    # get downlink channel info
-    # ch_downlink_config = config_meta_data["environment_emulation"]["target_link_config"]["wireless_channel"]["downlink"]
-    # ch_downlink ={"channel_model": ch_downlink_config["type"]}
-
-    # ch_downlink_model = {"channel_model": "AWGN"}
     # get uplink channel info
-    ch_uplink_config = config_meta_data["environment_emulation"]["target_link_config"][
-        "wireless_channel"
-    ]["uplink"]
+    ch_uplink_config = config_meta_data["environment_emulation"]["target_link_config"]["wireless_channel"]["uplink"]
     # Real Time RF Emulation
     if config_meta_data["test_config"]["test_mode"] == "rf_real_time":
         channel_metadata.update({"emulation_mode": "ni rf real time"})
@@ -346,7 +215,7 @@ def write_recorded_data_to_sigmf(captured_data, config_meta_data, global_info, i
             channel_metadata.update(
                 {
                     "channel_model": ch_uplink_config["statistical"]["predef_channel_profile"],
-                    "snr_esn0_db": ch_uplink_config["statistical"]["snr_db"],
+                    "snr_esn0_db": ch_uplink_config["statistical"]["snr_db"], 
                     }
             )
 
@@ -361,6 +230,7 @@ def write_recorded_data_to_sigmf(captured_data, config_meta_data, global_info, i
             )
         else:
             raise Exception("ERROR: channel type is not supported in SigMF formatting")
+        
     # RF Simulation
     elif config_meta_data["test_config"]["test_mode"] == "rf_simulation":
         channel_metadata.update({"emulation_mode": "oai rf simulation"})
@@ -397,16 +267,6 @@ def write_recorded_data_to_sigmf(captured_data, config_meta_data, global_info, i
             ],
         }
     )
-    # ----------------------------------------------------
-    # get test config
-    # ----------------------------------------------------
-    """
-    test_config = {
-        "test_name": config_meta_data["test_config"]["test_name"],
-        "test_mode": config_meta_data["test_config"]["test_mode"],
-        "dut_type": config_meta_data["test_config"]["dut_type"],
-        }
-    """
     # Create sigmf metadata
     # ----------------------
     # Add global parameters to SigMF metadata
@@ -439,10 +299,9 @@ def write_recorded_data_to_sigmf(captured_data, config_meta_data, global_info, i
     # Add capture parameters to SigMF metadata
     # ----------------------
     serialization_scheme = config_meta_data["data_recording_config"][
-        "supported_oai_tracer_messages"
-    ][captured_data["message_type"]]["serialization_scheme"]
+        "supported_oai_tracer_messages"][captured_data["message_type"]]["serialization_scheme"]
     capture_metadata = {
-        sigmf.SigMFFile.DATETIME_KEY: time_stamp_ms,
+        sigmf.SigMFFile.DATETIME_KEY: time_stamp,
         SigMFFile.FREQUENCY_KEY: freq,
         **create_serialization_metadata(
             serialization_scheme, waveform_generator, captured_data
@@ -486,7 +345,8 @@ def write_recorded_data_to_sigmf(captured_data, config_meta_data, global_info, i
     dataset_meta_file_path = os.path.join(data_storage_path, dataset_meta_filename)
     meta.tofile(dataset_meta_file_path)  # extension is optional
 
-    print(dataset_meta_file_path)
+    if DEBUG_WIRELESS_RECORDED_DATA:
+        print(dataset_meta_file_path)
 
     return dataset_meta_file_path
 
@@ -513,8 +373,9 @@ def save_sigmf_collection(streams: list, global_info: dict, description: str, st
 
     # save collection file
     collection.tofile(collection_filepath)
-    print("")
-    print(collection_filepath + ".sigmf-collection")
+    if DEBUG_WIRELESS_RECORDED_DATA:
+        print("")
+        print(collection_filepath + ".sigmf-collection")
 
 
 def load_sigmf(filename: str, storage_path: str, scope: str):
@@ -536,7 +397,7 @@ def load_sigmf(filename: str, storage_path: str, scope: str):
     annotation_length = annotations_metadata[0][sigmf.SigMFFile.LENGTH_INDEX_KEY]
 
     # get capture metadata
-    capture_metadata = signal.get_capture_info(annotation_start_idx)
+    #capture_metadata = signal.get_capture_info(annotation_start_idx)
 
     # from source code: sigmffile.py
     # "autoscale : bool, default True
@@ -551,4 +412,4 @@ def load_sigmf(filename: str, storage_path: str, scope: str):
     data = signal.read_samples(
         annotation_start_idx, annotation_length, autoscale=False, raw_components=True)
 
-    return data, global_metadata, annotations_metadata
+    return data, global_metadata, annotations_metadata

common/utils/data_recording/lib/sync_service.py

@@ -1,12 +1,10 @@
 # SPDX-License-Identifier: LicenseRef-CSSL-1.0
 
 # ---------------------------------------------------------------------
-# brief Sync service between captured Data from 5GNR gNB and UE
+# brief Sync service between captured Data from 5G NR gNB and UE
 
-import struct
 from lib import data_recording_messages_def
-
-DEBUG_POW_MEAS_SYNC = True
+from lib import shared_memory_interface
 
 
 # check if first frame ahead:
@@ -29,34 +27,55 @@ def is_frame_ahead(frame1, frame2, max_frame=1023):
 
 
 # find the frame and slot start
-def find_frame_slot_start(dataset1_start, dataset2_start):
+def find_frame_slot_start(dataset1_start_info, dataset2_start_info, is_dataset2_pow_monitoring):
     """
     Function to find the frame and slot start for data sync
     Args:
     dataset1_start_info: Dictionary containing the start information of dataset1
     dataset2_start_info: Dictionary containing the start information of dataset2
+    is_dataset2_pow_monitoring: Boolean indicating if dataset2 is pow monitoring
     Returns:
     sync_info: Dictionary containing the sync information
         sync_info["frame"] = frame_start
         sync_info["slot"]  = slot_start
+    Note: if is_dataset2_pow_monitoring is true, then the slot start is taken from dataset1
+    Since the pow monitoring is capturing all slots while the gNB and UE are capturing only data slots
 
     """
     sync_info = {}
 
-    if dataset1_start["frame"] == dataset2_start["frame"]:
+    if dataset1_start_info["frame"] == dataset2_start_info["frame"] :
         frame_diff = 0
-        frame_start = dataset1_start["frame"]
-        slot_start = max(dataset1_start["slot"], dataset2_start["slot"])
+        frame_start = dataset1_start_info["frame"] 
+        if is_dataset2_pow_monitoring:
+            if dataset1_start_info["slot"] >= dataset2_start_info["slot"]:
+                slot_start = dataset1_start_info["slot"]
+            elif dataset1_start_info["slot"] < dataset2_start_info["slot"]:
+                # Go to next frame and use data slot as reference
+                frame_start = (frame_start + 1) % 1024
+                # Use the slot from dataset1 as reference
+                slot_start = dataset1_start_info["slot"]
+        else:
+            slot_start = max(dataset1_start_info["slot"], dataset2_start_info["slot"])
 
-    elif is_frame_ahead(dataset1_start["frame"], dataset2_start["frame"]):
-        frame_start = dataset1_start["frame"]
-        slot_start = dataset1_start["slot"]
-        frame_diff = (dataset1_start["frame"]- dataset2_start["frame"] + 1024) % 1024
-    elif is_frame_ahead(dataset2_start["frame"], dataset1_start["frame"]):
-        frame_start = dataset2_start["frame"]
-        slot_start = dataset2_start["slot"]
-        frame_diff = (dataset2_start["frame"] - dataset1_start["frame"] + 1024) % 1024
-    # check first if the delta time between the two datasets is larger than expected offset time.
+    elif is_frame_ahead(dataset1_start_info["frame"], dataset2_start_info["frame"]):
+        frame_start = dataset1_start_info["frame"]
+        slot_start = dataset1_start_info["slot"]
+        frame_diff = (dataset1_start_info["frame"]- dataset2_start_info["frame"] + 1024) % 1024
+    elif is_frame_ahead(dataset2_start_info["frame"], dataset1_start_info["frame"]):
+        frame_start = dataset2_start_info["frame"]
+        if is_dataset2_pow_monitoring:
+            if dataset1_start_info["slot"] >= dataset2_start_info["slot"]:
+                slot_start = dataset1_start_info["slot"]
+            elif dataset1_start_info["slot"] < dataset2_start_info["slot"]:
+                # Go to next frame and use data slot as reference
+                frame_start = (frame_start + 1) % 1024
+                # Use the slot from dataset1 as reference
+                slot_start = dataset1_start_info["slot"]
+        else:
+            slot_start = dataset2_start_info["slot"]
+        frame_diff = (dataset2_start_info["frame"] - dataset1_start_info["frame"] + 1024) % 1024
+    # check first if the delta time between the two datasets is larger than expected offset time. 
     # So, the sync will not be applied
     # check during the calculation of the frame the ramp-around from 1023 to 0
     if abs(frame_diff) > 6:
@@ -70,54 +89,34 @@ def find_frame_slot_start(dataset1_start, dataset2_start):
     sync_info["slot"] = slot_start
     return sync_info
 
+# Sync data across N NR tracer nodes
+def sync_multiple_nr_nodes(node_shm_readings):
+    """Sync frame/slot across N NR tracer nodes.
 
-# Read data from gNB T-tracer Application
-def get_frame_slot_start(shm_reading, bufIdx, general_msg_header_list,
-                         general_message_header_length):
-        buf = shm_reading.read(bufIdx + general_message_header_length)
-        msg_id = struct.unpack("<H", buf[bufIdx: bufIdx + general_msg_header_list.get("msg_id")])[0]
-        bufIdx += general_msg_header_list.get("msg_id")
-        frame = struct.unpack("<H", buf[bufIdx: bufIdx + general_msg_header_list.get("frame")])[0]
-        bufIdx += general_msg_header_list.get("frame")
-        slot = struct.unpack("B", buf[bufIdx: bufIdx + general_msg_header_list.get("slot")])[0]
-        return frame, slot
-
-
-# Sync data between gNB and UE
-def sync_gnb_ue_captured_data(shm_reading_gnb, shm_reading_ue):
-    """
-    Function to get the sync (frame, slot) data between gNB and UE
     Args:
-    shm_reading_gnb: Shared memory for gNB
-    shm_reading_ue: Shared memory for UE
-    Returns:
-    sync_info: Dictionary containing the sync information between gNB and UE
-        sync_info["frame"] = frame_start
-        sync_info["slot"]  = slot_start
-    """
-    # get general message header list
-    general_msg_header_list, general_message_header_length = (
-        data_recording_messages_def.get_general_msg_header_list())
+        node_shm_readings: dict {node_id: shm_reading}
 
-    # Read data from gNB T-tracer Application
-    bufIdx = 0
-    frame_gnb, slot_gnb = get_frame_slot_start(
-        shm_reading_gnb, bufIdx, general_msg_header_list, general_message_header_length)
-    # Read data from UE T-tracer Application
-    bufIdx = 0
-    frame_ue, slot_ue = get_frame_slot_start(
-        shm_reading_ue, bufIdx, general_msg_header_list, general_message_header_length)
-    dataset1_start = {}
-    dataset1_start["frame"] = frame_gnb
-    dataset1_start["slot"] = slot_gnb
-    dataset2_start = {}
-    dataset2_start["frame"] = frame_ue
-    dataset2_start["slot"] = slot_ue
-    # Sync data between gNB and UE
-    # We noticed that the maximum difference between the frame number of gNB and UE is 3 frames
-    # Calculate the frame difference considering the wrap-around from 1023 to 0
-    sync_info = find_frame_slot_start(dataset1_start, dataset2_start)
-    print(" gNB Start info: ", dataset1_start)
-    print(" UE Start info: ", dataset2_start)
-    print(" gNB and UE Sync info: ", sync_info)
+    Returns:
+        sync_info: dict with 'frame' and 'slot' of the common sync point
+    """
+    sync_header_msg, sync_header_msg_length = (
+        data_recording_messages_def.get_sync_header_msg_list())
+
+    # Get frame/slot from each node
+    node_start_infos = {}
+    for node_id, shm_reading in node_shm_readings.items():
+        frame, slot = shared_memory_interface.get_frame_slot_start(
+            shm_reading, 0, sync_header_msg, sync_header_msg_length)
+        node_start_infos[node_id] = {"frame": frame, "slot": slot}
+        print(f"  [NR Node Sync] {node_id} start: frame={frame}, slot={slot}")
+
+    # Find the latest common frame/slot by pairwise comparison
+    node_ids = list(node_start_infos.keys())
+    sync_info = node_start_infos[node_ids[0]]
+    for i in range(1, len(node_ids)):
+        sync_info = find_frame_slot_start(sync_info, node_start_infos[node_ids[i]],
+                                          is_dataset2_pow_monitoring=False)
+
+    print(f"  [NR Node Sync] Multi-node sync point: frame={sync_info['frame']}, slot={sync_info['slot']}")
     return sync_info
+

common/utils/data_recording/lib/wireless_parameters_mapper.py

@@ -0,0 +1,189 @@
+# SPDX-License-Identifier: LicenseRef-CSSL-1.0
+
+# ---------------------------------------------------------------------
+# brief Wireless parameters mapper to map captured parameters from waveform generator to API and SigMF format based on a mapping specification in a YAML file.
+#  It also includes functions to derive other parameters that are not directly captured but can be derived from the captured parameters.
+import os
+import numpy as np
+import yaml
+
+STANDARDS = {"5gnr_oai": "5gnr"}
+
+
+def map_waveform_metadata_to_wireless_dic(scope, waveform_generator, parameter_map_file, captured_data):
+    """
+    Maps metadata from Waveform creator to API and SigMF format.
+    The used parameters and the mapping pairs are specified in a separate YAML file
+    that must be provided as well.
+    """
+    # read waveform parameter map from yaml file
+    dir_path = os.path.dirname(__file__)
+    src_path = os.path.split(dir_path)[0]
+
+    with open(os.path.join(src_path, parameter_map_file), "r") as file:
+        parameter_map_dic = yaml.load(file, Loader=yaml.Loader)
+    # preallocate target dict
+    signal_metadata = {}
+
+    #  get standard and name of generator
+    standard_key = parameter_map_dic["waveform_generator"][waveform_generator]
+    generator = standard_key["generator"]
+
+    if scope == "tx":
+        parameter_map_dic = parameter_map_dic["transmitter"][
+            STANDARDS[waveform_generator]
+        ]
+    elif scope == "channel":
+        parameter_map_dic = parameter_map_dic["channel"]
+    elif scope == "rx":
+        parameter_map_dic = parameter_map_dic["receiver"]
+    else:
+        raise Exception(
+            f"Invalid mapping scope '{scope}'! Only 'tx', 'channel' and 'rx' are valid!"
+        )
+
+    # check if standard key is given
+    if parameter_map_dic is None:
+        raise Exception(
+            "Invalid standard key: "
+            "Name should be corrected or added to wireless_link_parameter_map.yaml, given: "
+            f"{waveform_generator}"
+        )
+
+    for parameter_pair in parameter_map_dic:
+        # check if key for chosen simulator even exists
+        if waveform_generator + "_parameter" in parameter_pair.keys():
+            # only continue with mapping from file if direct equivalent exists
+            if parameter_pair[waveform_generator + "_parameter"]["name"]:
+                # It is not necessary to get all parameters from wireless_link_parameter_map.yaml 
+                # in captured_data since some parameters related to DL or UL only
+                if (parameter_pair[waveform_generator + "_parameter"]["name"] in captured_data.keys()):
+                    # extract value from waveform config source
+                    value = captured_data[
+                        parameter_pair[waveform_generator + "_parameter"]["name"]]
+                    # additional mapping if parameter values should come from a discrete set of values
+                    if ("value_map" in parameter_pair[waveform_generator + "_parameter"].keys()):
+                        value = parameter_pair[waveform_generator + "_parameter"]["value_map"][value]
+                    # write to target dictionary for SigMF
+                    signal_metadata[parameter_pair["sigmf_parameter_name"]] = value
+            else:
+                raise Exception(
+                    f"Incomplete specification in field '{waveform_generator}_parameter'!")
+    if not signal_metadata:
+        raise Exception(
+            """ERROR: Check captured meta-data or provided config and meta data """)
+
+    # check for non-JSON-serializable data types
+    def isfloat(NumberString):
+        try:
+            float(NumberString)
+            return True
+        except ValueError:
+            return False
+
+    for key, value in signal_metadata.items():
+        if isinstance(value, np.integer):
+            signal_metadata[key] = int(value)
+        elif isinstance(value, (np.float16, np.float32, np.float64)):
+            signal_metadata[key] = np.format_float_positional(value, trim="-")
+        elif isinstance(value, int):
+            signal_metadata[key] = int(value)
+        elif isinstance(value, float):
+            # store value in decimal and not in scientific notation
+            signal_metadata[key] = float(value)
+        elif isinstance(value, str) and key != "standard":
+            # convert string to lower case
+            signal_metadata[key] = value.lower()
+            if isfloat(value):
+                if value.isdigit():
+                    signal_metadata[key] = int(float(value))
+                elif value.replace(".", "", 1).isdigit() and value.count(".") < 2:
+                    signal_metadata[key] = float(value)
+
+    return signal_metadata, generator
+
+
+# Derive other Signal metadata from captured metadata
+def derive_remaining_5gnr_metadata(captured_data, signal_metadata):
+    # Add other OAI metadata to Wireless Dictionary Parameter Map - SigMF metadata
+    # ----------------------------------------------------
+    # Set 1: Parameters needs to be derived from OAI message
+    # ----------------------------------------------------
+    # DMRS Info: 
+    # Check if the message type contains "UL"
+    if "UL" in captured_data["message_type"]:
+        # PUSCH DMRS: OFDM symbol start index within slot
+        get_pusch_dmrs_start_ofdm_symbol = 0
+        signal_metadata["pusch_dmrs:ofdm_symbol_idx"] = []
+
+        for symbol in range(captured_data["start_symbol_index"], 
+                            captured_data["start_symbol_index"] + captured_data["nr_of_symbols"]):
+
+            dmrs_symbol_flag = (captured_data["ul_dmrs_symb_pos"] >> symbol) & 0x01
+            if dmrs_symbol_flag and not get_pusch_dmrs_start_ofdm_symbol:
+                get_pusch_dmrs_start_ofdm_symbol = 1
+                signal_metadata["pusch_dmrs:start_ofdm_symbol"] = symbol
+            if dmrs_symbol_flag:
+                signal_metadata["pusch_dmrs:ofdm_symbol_idx"].append(symbol)  # Append symbol to the list
+
+        signal_metadata["pusch_dmrs:duration_num_ofdm_symbols"] = 1
+        signal_metadata["pusch_dmrs:num_add_positions"] = (captured_data["number_dmrs_symbols"] - 1)
+
+    if "DL" in captured_data["message_type"]:
+        # PDSCH DMRS: OFDM symbol start index within slot
+        get_pdsch_dmrs_start_ofdm_symbol = 0
+        signal_metadata["pdsch_dmrs:ofdm_symbol_idx"] = []
+        for symbol in range(captured_data["StartSymbolIndex"], 
+                            captured_data["StartSymbolIndex"] + captured_data["NrOfSymbols"]):
+            dmrs_symbol_flag = captured_data["dlDmrsSymbPos"] & (1 << symbol)
+            if dmrs_symbol_flag:
+                if dmrs_symbol_flag and not get_pdsch_dmrs_start_ofdm_symbol:
+                    get_pdsch_dmrs_start_ofdm_symbol = 1
+                    signal_metadata["pdsch_dmrs:start_ofdm_symbol"] = symbol
+            if dmrs_symbol_flag:
+                signal_metadata["pdsch_dmrs:ofdm_symbol_idx"].append(symbol)  # Append symbol to the list
+
+        signal_metadata["pdsch_dmrs:duration_num_ofdm_symbols"] = 1
+        signal_metadata["pdsch_dmrs:num_add_positions"] = (captured_data["numDmrsSymbols"] - 1)
+
+    # If the message is a BIT message, add number of bits to the signal info
+    # "UE_PHY_UL_SCRAMBLED_TX_BITS", "GNB_PHY_UL_PAYLOAD_RX_BITS", "UE_PHY_UL_PAYLOAD_TX_BITS"
+    if "_BITS" in captured_data["message_type"]:
+        # if "BIT" in captured_data["message_type"]:
+        signal_metadata["num_bits"] = (captured_data["number_of_bits"])
+
+    # Set Link direction and number of antennas based on link direction and captured side
+    if "UL" in captured_data["message_type"]:
+        signal_metadata["link_direction"] = "uplink"
+        # On UE side, It is TX antennas
+        # On gNB side, It is RX antennas
+        if "UE" in captured_data["message_type"]:
+            signal_metadata["num_tx_antennas"] = captured_data["nb_antennas"]
+        else:
+            signal_metadata["num_rx_antennas"] = captured_data["nb_antennas"]
+
+    if "DL" in captured_data["message_type"]:
+        signal_metadata["link_direction"] = "downlink"
+        # On UE side, It is RX antennas
+        # On gNB side, It is TX antennas
+        if "UE" in captured_data["message_type"]:
+            signal_metadata["num_rx_antennas"] = captured_data["nb_antennas"]  ## TO DO check key name: no DL message from UE yet
+        else:
+            signal_metadata["num_tx_antennas"] = captured_data["nrOfLayers"]  
+    # ----------------------------------------------------
+    # Set 2: Parameters that is hardcoded
+    # ----------------------------------------------------
+    # Note: The mapping type is hardcoded. It is fixed in OAI to mapping type A for DL and B for UL.
+    # Look to: openair2/RRC/NR/nr_rrc_config.c
+    if "UL" in captured_data["message_type"]:
+        signal_metadata["pusch:mapping_type"] = "B"
+        signal_metadata["pusch:content"] = "compliant"
+        signal_metadata["pusch:payload_bit_pattern"] = "random"
+    if "DL" in captured_data["message_type"]:
+        signal_metadata["pdsch:mapping_type"] = "A"
+        signal_metadata["pdsch:content"] = "compliant"
+        signal_metadata["pdsch:payload_bit_pattern"] = "random"
+        
+    signal_metadata["num_slots"] = 1
+
+    return signal_metadata

common/utils/data_recording/sync_validation_demo.py

@@ -118,7 +118,7 @@ def calculate_stats(main_config_file: str):
 
 if __name__ == "__main__":
 
-    main_config = "/home/user/workarea/oai_recorded_data/config_data_recording_app.json"
+    main_config = "/home/user/workarea/oai_recorded_data/config_data_recording_app_extended.json"
 
     # calculate BER stats
     calculate_stats(main_config)