Before starting a new iperf3 server instance in CI tests, explicitly kill any existing
iperf3 processes bound to the same port. This prevents conflicts or failures caused by
leftover iperf3 servers from previous runs that did not terminate properly or on time.
The cleanup is performed using `pkill` before launching the new server.
Handle GTP receiver errors, fix memory leaks, add CU-UP test
- Fix use-after-free bugs in GTP by properly stopping the receive thread(s)
- Fix a bug from !3519 (merged): QFI 0 is a valid QFI
- Fix multiple memory leaks in CU-UP, E1, CU-UP load tester, SCTP
- Add a (conservative) functional CU-UP test that verifies that the load tester
and CU-UP work. Test only a low throughput, as it will run in parallel with
other tests.
Use the CU-UP load tester to test the functionality of the CU-UP and
its loader tester. The requirements on this load test are intentionally
low, as it will be run with other unit tests in parallel, and should
still pass.
It uses an existing configuration file for the CU-UP. The test script
runs the CU-UP, runs the load tester, stops the CU-UP, and returns the
load tester return value as test result (which will be 0=success if all
packets have been received).
Note that this strdup has been intentioally created for debugging
purposes in core files.
See also: 1ab3b8dd8e ("add version signature in core files...")
Direct leak of 8 byte(s) in 1 object(s) allocated from:
#0 0x7f7bc7ee68a3 in calloc (/lib64/libasan.so.8+0xe68a3) (BuildId: 10b8ccd49f75c21babf1d7abe51bb63589d8471f)
#1 0x0000004abfa6 in calloc_or_fail /home/richie/oai/common/utils/utils.h:74
#2 0x0000004abfa6 in decode_e1ap_cuup_setup_request /home/richie/oai/openair2/E1AP/lib/e1ap_interface_management.c:219
#3 0x00000046a9f5 in e1apCUCP_handle_SETUP_REQUEST /home/richie/oai/openair2/E1AP/e1ap.c:187
#4 0x000000470b35 in e1ap_handle_message /home/richie/oai/openair2/E1AP/e1ap.c:109
#5 0x000000470b35 in e1_task_handle_sctp_data_ind /home/richie/oai/openair2/E1AP/e1ap.c:120
#6 0x00000047717b in E1AP_CUCP_task /home/richie/oai/openair2/E1AP/e1ap.c:832
#7 0x7f7bc7e28ee5 in asan_thread_start(void*) (/lib64/libasan.so.8+0x28ee5) (BuildId: 10b8ccd49f75c21babf1d7abe51bb63589d8471f)
Direct leak of 8 byte(s) in 1 object(s) allocated from:
#0 0x7f7bc7ee68a3 in calloc (/lib64/libasan.so.8+0xe68a3) (BuildId: 10b8ccd49f75c21babf1d7abe51bb63589d8471f)
#1 0x0000004ac30c in calloc_or_fail /home/richie/oai/common/utils/utils.h:74
#2 0x0000004ac30c in decode_e1ap_cuup_setup_request /home/richie/oai/openair2/E1AP/lib/e1ap_interface_management.c:194
#3 0x00000046a9f5 in e1apCUCP_handle_SETUP_REQUEST /home/richie/oai/openair2/E1AP/e1ap.c:187
#4 0x000000470b35 in e1ap_handle_message /home/richie/oai/openair2/E1AP/e1ap.c:109
#5 0x000000470b35 in e1_task_handle_sctp_data_ind /home/richie/oai/openair2/E1AP/e1ap.c:120
#6 0x00000047717b in E1AP_CUCP_task /home/richie/oai/openair2/E1AP/e1ap.c:832
#7 0x7f7bc7e28ee5 in asan_thread_start(void*) (/lib64/libasan.so.8+0x28ee5) (BuildId: 10b8ccd49f75c21babf1d7abe51bb63589d8471f)
Direct leak of 100 byte(s) in 1 object(s) allocated from:
#0 0x7f6585ee6f2b in malloc (/lib64/libasan.so.8+0xe6f2b) (BuildId: 10b8ccd49f75c21babf1d7abe51bb63589d8471f)
#1 0x000000414a0d in malloc_or_fail /home/richie/oai/common/utils/utils.h:86
#2 0x0000004061c9 in main /home/richie/oai/tests/nr-cuup/nr-cuup-load-test.c:544
#3 0x7f6585211574 in __libc_start_call_main (/lib64/libc.so.6+0x3574) (BuildId: 48c4b9b1efb1df15da8e787f489128bf31893317)
#4 0x7f6585211627 in __libc_start_main@GLIBC_2.2.5 (/lib64/libc.so.6+0x3627) (BuildId: 48c4b9b1efb1df15da8e787f489128bf31893317)
#5 0x000000413034 in _start (/home/richie/oai/build/tests/nr-cuup/nr-cuup-load-test+0x413034) (BuildId: 8af0132792b03fa12ba95b5623865c9a8a5625a3)
Direct leak of 100 byte(s) in 1 object(s) allocated from:
#0 0x7f6585ee6f2b in malloc (/lib64/libasan.so.8+0xe6f2b) (BuildId: 10b8ccd49f75c21babf1d7abe51bb63589d8471f)
#1 0x000000414a0d in malloc_or_fail /home/richie/oai/common/utils/utils.h:86
#2 0x0000004061ff in main /home/richie/oai/tests/nr-cuup/nr-cuup-load-test.c:546
#3 0x7f6585211574 in __libc_start_call_main (/lib64/libc.so.6+0x3574) (BuildId: 48c4b9b1efb1df15da8e787f489128bf31893317)
#4 0x7f6585211627 in __libc_start_main@GLIBC_2.2.5 (/lib64/libc.so.6+0x3627) (BuildId: 48c4b9b1efb1df15da8e787f489128bf31893317)
#5 0x000000413034 in _start (/home/richie/oai/build/tests/nr-cuup/nr-cuup-load-test+0x413034) (BuildId: 8af0132792b03fa12ba95b5623865c9a8a5625a3)
Free memory including the "base pointer". Fix two places in which it was
on the stack to harmonize. The only stack variable is in reception of
new E1 messages (e1ap_handle_message()), which uses
ASN_STRUCT_FREE_CONTENTS_ONLY().
This fixes bugs similar to these logs:
Direct leak of 40 byte(s) in 1 object(s) allocated from:
#0 0x7f418f4e68a3 in calloc (/lib64/libasan.so.8+0xe68a3) (BuildId: 10b8ccd49f75c21babf1d7abe51bb63589d8471f)
#1 0x00000049e367 in calloc_or_fail /home/richie/oai/common/utils/utils.h:74
#2 0x00000049e367 in encode_e1_bearer_context_release_command /home/richie/oai/openair2/E1AP/lib/e1ap_bearer_context_management.c:1235
#3 0x000000482951 in e1apCUCP_send_BEARER_CONTEXT_RELEASE_COMMAND /home/richie/oai/openair2/E1AP/e1ap.c:546
#4 0x000000482951 in E1AP_CUCP_task /home/richie/oai/openair2/E1AP/e1ap.c:856
#5 0x7f418f428ee5 in asan_thread_start(void*) (/lib64/libasan.so.8+0x28ee5) (BuildId: 10b8ccd49f75c21babf1d7abe51bb63589d8471f)
Direct leak of 40 byte(s) in 1 object(s) allocated from:
#0 0x7f418f4e68a3 in calloc (/lib64/libasan.so.8+0xe68a3) (BuildId: 10b8ccd49f75c21babf1d7abe51bb63589d8471f)
#1 0x00000049380a in calloc_or_fail /home/richie/oai/common/utils/utils.h:74
#2 0x00000049380a in encode_E1_bearer_context_setup_request /home/richie/oai/openair2/E1AP/lib/e1ap_bearer_context_management.c:578
#3 0x00000047f6a2 in e1apCUCP_send_BEARER_CONTEXT_SETUP_REQUEST /home/richie/oai/openair2/E1AP/e1ap.c:378
#4 0x0000004829f8 in E1AP_CUCP_task /home/richie/oai/openair2/E1AP/e1ap.c:846
#5 0x7f418f428ee5 in asan_thread_start(void*) (/lib64/libasan.so.8+0x28ee5) (BuildId: 10b8ccd49f75c21babf1d7abe51bb63589d8471f)
Direct leak of 40 byte(s) in 1 object(s) allocated from:
#0 0x7f418f4e68a3 in calloc (/lib64/libasan.so.8+0xe68a3) (BuildId: 10b8ccd49f75c21babf1d7abe51bb63589d8471f)
#1 0x00000046ede7 in calloc_or_fail /home/richie/oai/common/utils/utils.h:74
#2 0x00000046ede7 in encode_e1ap_cuup_setup_response /home/richie/oai/openair2/E1AP/lib/e1ap_interface_management.c:305
#3 0x00000047e412 in e1ap_send_SETUP_RESPONSE /home/richie/oai/openair2/E1AP/e1ap.c:167
#4 0x000000482a68 in E1AP_CUCP_task /home/richie/oai/openair2/E1AP/e1ap.c:836
#5 0x7f418f428ee5 in asan_thread_start(void*) (/lib64/libasan.so.8+0x28ee5) (BuildId: 10b8ccd49f75c21babf1d7abe51bb63589d8471f)
QFI 0 is a valid QFI: don't check for qfi <= 0, as uint8 cannot be
negative. Also, correctly use SDAP_MAP_RULE_EMPTY for a DRB ID (which
need to be larger than 0).
The gNB will stop the periodical logging of UEs in MAC Info level after
a configurable number of UEs have been printed, by default 8.
Set it to 17 in some CI tests to ensure we still get all UEs for the
time being, as the CI might depend on this.
improvements in formatting and LOGs in DMRS common functions at MAC
DMRS LOGs at MAC were misleading. This MR tried to improve them (and also
adds some formatting fix).
The call to snprintf() as was used is wrong: we use the return value to
advance output, but the man page says
> The functions snprintf() and vsnprintf() do not write more than size
> bytes (including the terminating null byte ('\0')). If the output was
> truncated due to this limit, then the return value is the number of
> characters (excluding the terminating null byte) which would have been
> written to the final string if enough space had been available.
> Thus, a return value of size or more means that the output was
> truncated.
Thus, output could go beyond end, and we get a stack overflow. Instead,
encapsulate the call to snprintf() checking this condition, and only
advancing output as intended, limiting to 0 if we are at the end of the
buffer. This avoids this error:
==964825==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7be799f51988 at pc 0x7fe7d66b0d39 bp 0x7be79b825ef0 sp 0x7be79b8256c0
WRITE of size 3 at 0x7be799f51988 thread T24
#0 0x7fe7d66b0d38 in vsnprintf (/lib64/libasan.so.8+0xb0d38) (BuildId: 10b8ccd49f75c21babf1d7abe51bb63589d8471f)
#1 0x7fe7d66b2d44 in snprintf (/lib64/libasan.so.8+0xb2d44) (BuildId: 10b8ccd49f75c21babf1d7abe51bb63589d8471f)
#2 0x0000008fe7c6 in dump_mac_stats /home/richie/w/refactor-dlsch/openair2/LAYER2/NR_MAC_gNB/main.c:183
#3 0x00000092071a in gNB_dlsch_ulsch_scheduler /home/richie/w/refactor-dlsch/openair2/LAYER2/NR_MAC_gNB/gNB_scheduler.c:200
#4 0x0000008f8f78 in run_scheduler_monolithic /home/richie/w/refactor-dlsch/openair2/NR_PHY_INTERFACE/NR_IF_Module.c:399
There is the (global) sched_lock that prevents concurrent access. A
dedicated UE lock does not make sense in that case. Worse, this lock
does not prevent all concurrent UE accesses (e.g., there are many loops
over UEs that are not protected), so it's also wrong.
truncate resets the file size to 0. Without this, e.g., when
disconnecting a UE, the old UE remains at the end and is not
overwritten, which is confusing.
- Corrected AMF Set ID data type in ngap_gNB_nnsf_select_amf_by_amf_setid() from uint8_t to uint16_t.
- AMF Set ID is a 10-bit field as per 3GPP TS 38.413 §9.3.3.12.
- The incorrect data type caused truncation for AMF Set IDs >255, leading to lookup failures during RRC reestablishment.
- Verified successful reestablishment with large AMF Set IDs after the fix.
Reason for fix: AMF lookup using AMF Set ID was failing because the 10-bit AMF Set ID was truncated to 8 bits.
Closes#1012
Stop the receive thread before closing the socket, which avoids an error
message when stopping the GTP softmodem:
[GTPU] [91] Recvfrom failed (Bad file descriptor)
[GTPU] exiting thread
On error, exit the GTP thread after printing a diagnostic message. This
avoids also an error flagged by address sanitizer for use-after-free.
[GTPU] [91] Recvfrom failed (Bad file descriptor)
=================================================================
==285377==ERROR: AddressSanitizer: heap-use-after-free on address 0x7cd7b7fe5590 at pc 0x0000006a9e18 bp 0x7b97ad4e1d50 sp 0x7b97ad4e1d48
READ of size 4 at 0x7cd7b7fe5590 thread T8
=================================================================
==285377==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 160 byte(s) in 1 object(s) allocated from:
#0 0x0000006a9e17 in gtpv1uReceiver /home/richie/oai/openair3/ocp-gtpu/gtp_itf.cpp:1346
#1 0x7f97b9a28ee5 in asan_thread_start(void*) (/lib64/libasan.so.8+0x28ee5) (BuildId: 10b8ccd49f75c21babf1d7abe51bb63589d8471f)
#2 0x7f97b8e7ff53 in start_thread (/lib64/libc.so.6+0x71f53) (BuildId: 48c4b9b1efb1df15da8e787f489128bf31893317)
#3 0x7f97b8f0332b in __clone3 (/lib64/libc.so.6+0xf532b) (BuildId: 48c4b9b1efb1df15da8e787f489128bf31893317)
#0 0x7f97b9ae60cb in memalign (/lib64/libasan.so.8+0xe60cb) (BuildId: 10b8ccd49f75c21babf1d7abe51bb63589d8471f)
#1 0x000000471039 in newNotifiedFIFO_elt /home/richie/oai/common/utils/threadPool/notified_fifo.h:75
#2 0x000000471039 in tx_func /home/richie/oai/executables/nr-gnb.c:107
#3 0x000000471039 in L1_tx_thread /home/richie/oai/executables/nr-gnb.c:167
#4 0x7f97b9a28ee5 in asan_thread_start(void*) (/lib64/libasan.so.8+0x28ee5) (BuildId: 10b8ccd49f75c21babf1d7abe51bb63589d8471f)
0x7cd7b7fe5590 is located 336 bytes inside of 400-byte region [0x7cd7b7fe5440,0x7cd7b7fe55d0)
freed by thread T0 here:
SUMMARY: AddressSanitizer: 160 byte(s) leaked in 1 allocation(s).
Integration: `2025.w43`
* !3605 Support beam index in OAI 7.2 Fronthaul Interface
* !3709 Fixes for T-Tracer to work with NR-UE
* !3712 Restore UE Capability IE after RRCReestablishment
* !3716 Remove unused ULSCH measurements at gNB
* !3603 UE symbol based PDCCH receiver
* !3708 Add monolithic N2 handover test pipeline with B210 and Quectel
* !3719 Remove Benetel-specific radio drivers
* !3710 [CI] Update the Push to DockerHub Stage
* !3711 CI: Multiple adjustments
* !3717 Initialize NCC=0 on NGAP Initial Context Setup (3GPP TS 33.501 §6.9.2.1.1)
* !3720 fix direct bugs of not used RU_RX_SLOT_DEPTH defined constant
* !3714 Improve handling of selection of RSRP/SINR report via configuration file
* !3519 Refactor SDAP adaptation work for QoS
* !3698 Other CSI measurement fixes
* !3699 Updates for Feature Set
Closes#1008
See merge request oai/openairinterface5g!3713
Other CSI measurement fixes
Errors fixed in this MR:
- UE currently sends empty CSI measurement report resulting in invalid
cqi_idx 0, default to MCS 9 at gNB because mismanagement of new L1
measurement structure at MAC UE
- nrofReportedRS not updated after UE capability reception at gNB
- multiple RSRP reports multiplexed / de-multiplexed in wrong order
Refactor SDAP adaptation work for QoS
This MR is an adaptation of the code introduced in !2703.
The goal is a cleanup and functional refactor of the SDAP/PDCP
integration, that is:
- centralize SDAP configuration logic
- simplify entity creation
- improve robustness of role-based SDAP handling
- prepare the codebase for clean support of DRB reconfiguration and
teardown flows
More concretely:
1. New sdap_config_t and get_sdap_Config()
- Introduced an internal struct sdap_config_t to abstract SDAP config
input
- Centralized SDAP config parsing into new get_sdap_Config() function
that translates ASN.1 NR_SDAP_Config into this internal
representation. This is called during RRC triggered
(re)configuration procedures.
- Enable mapping of number of QoS flows > 1
2. Unified role handling with bitmasks
- Added a bitmask-based entity_role field to track SDAP directions:
SDAP_UL_RX, SDAP_UL_TX, SDAP_DL_RX, SDAP_DL_TX: header presence was
determined by simple direction (UE vs gNB) and individual
is_sdap_rx/tx() checks, however nr_sdap_rx_entity behaves
differently depending on the direction of the RX entity (DL/UL),
therefore this change can help enable a more accurate SDAP header
handling across directions and node types.
- Replaces old helpers is_sdap_rx() / is_sdap_tx() with internal
checks against the role bitfield
- QFI-to-DRB mapping is now annotated with direction for both data and
control PDUs
3. Simplified DRB and SDAP setup
- add_drb() in PDCP API now delegates SDAP config processing cleanly
- new_nr_sdap_entity() is used consistently to register a fresh SDAP
context for the UE and PDU session.
- Also, re-use and reconfigure existing SDAP entity if already
exists.
- logging has been added to trace mapped QFIs from RRC and resulting
table entries
4. Improved QFI-to-DRB management
- Refactored nr_sdap_qfi2drb_map_update() to register QFIs based on
parsed config
- Ensures control PDUs are correctly routed even after reconfiguration
- Extended nr_sdap_ue_qfi2drb_config: perform UL QoS flow to DRB
mapping configuration for a SDAP entity has already been established
according to TS 37.324, 5.3 QoS flow to DRB Mapping, 5.3.1
Configuration Procedures. Handle both UL QoS Flows mapping rules to
add and to remove. QFIs will now be properly unmapped if specified
via mappedQoS_FlowsToRelease, this might prevent stale QFI mappings
from being used incorrectly.
- nr_sdap_qfi2drb() and sdap_map_ctrl_pdu() now strictly fallback to
default_drb only if a QFI mapping is missing. Missing default DRB
now triggers LOG_E() (previously silent or misrouted to DRB 0).
5. Other cleanup
- Remove dependencies from legacy rb_id_t
Laurent THOMAS