CI: Add FHI7.2 phytest pipeline
This MR consists of multiple changes related to FHI7.2 setups in CI.
- remove unused fhi72 configurations in CI
- introduce dpdk-init service in fhi72 deployments to setup sriov on the machine
- update phytest fhi72 configurations for running on bulbul
- add phytest fhi72 to RAN-SA-FHI72-CN5G pipeline
- Introduce dpdk-init service for SR-IOV initialization within the container
- Execute script to enable low-latency RT performance (restored after tests)
- Change gNB RRC enable_sdap default to true
- Update RRC and RUNMODEM docs to reflect new default behavior
- Clean up CI gNB configs by removing redundant enable_sdap=1
- Keep explicit SDAP disable config for NFAPI gNB VNF
The CI test proved not as stable as expected (because the UE triggers A3
events which we don't want), so revert this functionality while we work
on a new pipeline.
This reverts commit bc62e8abca.
[FHI72] [xran F] Handle fragmented (un)compressed packets
In both E and F, the radio-transport fragmentation is not supported, and
all the "fragmented" (not seen as fragments in the xran, only as
sections) packets contain "E-bit" equal to 1. Per spec, the value 1 for
"E-bit" signifies the last fragment of one symbol. For more info, please
see "ecpriSeqid" section in the spec.
In E:
- nRBStart and nRBSize is updated according to the sent/received packet;
- max sections = 1;
- when a packet is fragmented, the xran uses memcpy(pos, iq_data_start,
size) function, and nothing is stored in start_prbu , num_prbu ,...
In F:
- nRBStart and nRBSize is only used for C-plane messages;
- nPrbElm represents the number of sections for DL only;
- based on the nPrbElm, each section contains UP_nRBStart and UP_nRBSize
parameters which represent the start and number of PRBs in one section
or one fragment;
- when a packet is fragmented, the xran uses nSecDesc and sec_desc to
extract the packet section content, and the data is stored in
start_prbu , num_prbu ,...
Previous work on the fragmentation in F:
- #884
- !3394 - only increases the XRAN_MAX_FRAGMENT from 4 to 6, since it was
only tested with BFP 8, but not end-to-end unfortunately. But this MR
fixes all the issues.
Tested with VVDN RU, 2x2, 16 bits, 100MHz, 30kHz, MTU <= 9216 B, and
Quectel UE RM520N-GL Revision: RM520NGLAAR03A03M4G.
Implement TS 38.473 transparency for CellGroupConfig forwarding and handle re-establishment on a different DU
This MR implements the following fixes that are relevant in RRC
procedures (e.g. re-establishment, reconfiguration):
1. Transparent CellGroupConfig forwarding
This MR implements TS 38.473 transparency requirements by ensuring that
the CellGroupConfig IE is transparently forwarded from the gNB-DU to the
UE without any re-encoding at the gNB-CU. The CU stores and forwards the
encoded bytes transparently, while the DU is responsible for all
encoding and modifications (e.g., reestablishRLC flags). Motivation
Motivation: The 3GPP TS 38.473 specification mandates that the
CellGroupConfig IE shall be transparently signaled to the UE.
Previously, the CU was re-encoding CellGroupConfig depending on the
scenario, instead of retrieving it from the DU, violating the
transparency requirement. CU may decode for internal use but must not
re-encode for forwarding. Solution
Solution: Core Principle: The CU stores the encoded CellGroupConfig
bytes received from the DU and forwards them directly to the UE without
any re-encoding. The DU handles all encoding and modifications.
Key Changes:
1. Transparent Forwarding in RRC Reconfiguration
- Modified build_RRCReconfiguration_IEs() to use pre-encoded bytes
directly
- Changed parameter from cell_group_config to cgc (encoded byte
array with buffer and size)
2. Encoded Bytes Storage
- Added byte_array_t mcg field to RRC UE structure to store encoded
CellGroupConfig
- Added store_cgc() helper function to save encoded cell group in
the RRC UE Context at gNB (from F1AP messages)
3. DU Encoding Responsibilities
- Add gNB-DU Configuration Query IE support to trigger cgc encoding
to request CellGroupConfig from DU
- DU encodes CellGroupConfig with proper reestablishRLC flags
during re-establishment
- DU removes SRB1 from rlc-BearerToAddModList during
re-establishment (SRB1 re-establishment is done before
RRCReconfiguration)
4. New functions
- Add rrc_detect_reestablishment() function to detect
re-establishment during UE Context Modification
- Extracted rrc_gNB_first_reconfiguration_after_reestablishment()
function to handle first reconfiguration after re-establishment
- Added dump_cgc() and dump_mcg() utility functions for debugging
purposes
5. Re-establishment Flow
- rrc_gNB_process_RRCReestablishmentComplete() requests
CellGroupConfig from DU via gNB_DU_Configuration_Query in UE
Context Modification Request
- DU detects re-establishment and add reestablishRLC IEs
- CU stores encoded CGC from the UE Context Modification Response
and forwards it transparently in subsequent RRC Reconfiguration
Testing
Manual Testing:
- RRC re-establishment procedure completes successfully
- RRC Reconfiguration after re-establishment includes all DU-provided information
Issues fixed
Closes issue #837
2. Trigger UE Context Setup fallback upon reestablishment on DU
different from the original one
As per 38.401 §8.7: if the UE accesses a DU different from the original
one during re-establishment, the CU must trigger UE Context Setup toward
the new DU. This refers to steps 9-10 and steps 9’-10’. Either one or
the other can be executed during re-establishment.
e.g. if this happens during handover, it shall trigger a UE Context
Setup fallback on the CU.
This MR implements the required behavior by:
- Adding a new helper function rrc_handle_f1_ho_reestab_on_target() to
handle F1 handover reestablishment on target DU. The function triggers
UE Context Setup procedure when the UE reestablishes on the target DU
during an ongoing F1 handover.
- Calling this function when detecting that the UE is reestablishing on
the target DU during handover, i.e.: (1) from
handle_rrcReestablishmentComplete(), as per 38.401 §8.7, NOTE on steps
9-10: CU gets UL RRC MESSAGE TRANSFER from a DU different than the
original (2) from rrc_CU_process_ue_modification_required, as per
38.401 §8.7, NOTE on steps 9'-10': CU gets a UE CONTEXT MODIFICATION
REQUIRED from a DU different from the original one.
Also:
- add helper rrc_fill_f1_ue_context_setup and adopt it in handover
- move DRB list filler fill_drb_to_be_setup into rrc_gNB.c and rename it
rrc_fill_f1_drb_to_setup for consistency
See also TS 38.401 Figure 8.7-1 "RRC connection reestablishment
procedure" for reference
3. DU: Handle missing old UE context in DL RRC Message Transfer
During F1 handover reestablishment, the CU may forward a DL RRC Message
Transfer to the target DU with old_gNB_DU_ue_id included, even though
the target DU does not have the old UE context yet.
The code previously used AssertFatal when the old UE context was not
found, causing a crash: "CU claims we should know UE %04x, but we don't"
According to TS 38.473, the gNB-DU shall release the old gNB-DU UE F1AP ID only:
> if the gNB-DU identifies the UE-associated logical F1-connection If
> there is no matching connection, there is nothing to release.
Fix:
- Check if old UE context exists before accessing it
- If old UE does not exist, log error, clean up any stale F1 UE data,
and return early (do not deliver RRC message since reestablishment
can't be completed)
The goal is to address a recurrent issue in the F1 pipeline, see
jenkins-oai.eurecom.fr/job/RAN-SA-Handover-CN5G/514:
Assertion (oldUE) failed!
In dl_rrc_message_transfer() /oai-ran/openair2/LAYER2/NR_MAC_gNB/mac_rrc_dl_handler.c:956
CU claims we should know UE 35ce, but we don't
Even though the jumbo frame has 9600 bytes by the spec, the max MTU is limited to the MTU of a switch, if any used.
However, the value can be < max MTU of a switch.
This is very important for fragmentation purposes. If not properly set, the PDSCH packets will be missing.
- increase requirements for Event A3 trigger to avoid unwanted handovers
and allow testing reestablishment on different DUs
- increase ping_packetloss_threshold from 5 to 30 because we are forcing
a reestablishment, thus, packet loss will inevitably happen
- further adjustments to avoid A3 Events
Add Ethernet PDU session support and make PDU session request configurable
This MR adds support for Ethernet PDU sessions, and makes PDU sessions
configurable: per-PDU session ID, type (IPv4/IPv4v6/IPv6/Ethernet [1]),
DNN, NSSAI.
Concretely, it adds a new configuration file section pdu_sessions below
uicc0 that looks like this if everything is specified (default values
are shown):
uicc0:
[...]
pdu_sessions:
- id: 1
type: "IPv4"
nssai_sst: 1
nssai_sd: 0xffffff
dnn: "oai"
It is possible to list multiple PDU sessions (add new line - id: 2 to
request a second PDU session). It is backwards-compatible to the current
PDU session configuration (uicc0.dnn/nssai_sst/etc), the idea being that
either legacy or new configuration could be used, but (1) if
pdu_sessions exist, it will overwrite the legacy configuration, and (2)
multiple PDU sessions, and type, can only be configured using the new
config. The old parameters --extra-pdu-id and --default-pdu-id are not
accepted anymore. Documentation has been added to the UE tutorial.
It is possible to test an Ethernet PDU session [1]. It is possible to
test the PDU session inside the UE by verifying it has an TAP interface
oaitap_ue1, and ARPing the data network (arping has been added to
oai-nr-ue container for that purpose):
ip addr add 192.168.72.140/26 dev oaitap_ue1 # same subnet as data network!
ip link set dev oaitap_ue1 up
arping -c 20 -I oaitap_ue1 192.168.72.135 # will send ARP request in Ethernet frame to get IP address of data network
[1] See description MR !3769 for more information. On the UE side, only
the type has to be changed to "Ethernet".
Use the new PDU session configuration functionality of the parent
commit. It is now possible to request multiple PDU sessions, and to
request specific PDU session types. While the new uicc0.pdu_sessions is
preferred, it is still possible to use the "legacy"
uicc0.dnn/nssai_sst/nssai_sd.
The old parameters --default-pdu-id and --extra-pdu-id are not allowed
anymore, as it is possible to manually define this in the UE
configuration. Correspondingly, fix the use of those options, where
applicable.
Note: since the default_pdu_id is changed to -1, all interface names
will have pX to interface names appended (oaitun_ue1pX). The next commit
will correct this.
This feature shall allow in future to simulate an arbitratry number of UEs connected to an other arbitrary number of cells.
Therefore each RF-frontend (also called RU) needs to be configured for one specific cell.
In case of handover of a UE from one cell to another cell, it needs to switch RUs accordingly (TODO).
There has to be a 1-to-1 mapping between RUs and cells.
Each UE will be assigned to one of the defined cells.
Initially all UEs are assigned to cell 0.
SSB position in burst is set to 2 so only SSB index 1 is transmitted.
prach_msg1_FDM is set to 1 so there are two occasions per time slot.
All the UE connecting to gNB will receive SSB index 1 and trigger PRACH
in the second FDM prach occasion.
vrtsim: Optimize channel modelling
Optimization pass for channel modelling on transmission in vrtsim. This
comes from a requirement of O-RU to perform channel modelling with minimum
latency. The latency measured in O-RU 4TX 2RX on the UE is lowered from
~400 uS to ~100uS.
- Reduced MAX_CHANNEL_LENGTH to 200 and introduced SAVED_SAMPLES_LEN
- Refactored channel modelling actor allocation to parallelize work better
- Improved batching and task creation for channel modelling
- Replaced static global saved_samples with local static buffer in vrtsim_write_with_chanmod
Create a new CI pipeline for testing with vrtsim on Gracehopper:
https://jenkins-oai.eurecom.fr/job/RAN-VRT-Sim-Test-5G/
CFRA: mark RA complete when sending Msg2
The current contention-free random access forces a "Msg3" (which does
not exist in CFRA) to be received by a UE. Sometimes, this "Msg3" is not
received (for whatever reason), and the MAC declare that RA failed.
This is problematic, as the spec says that RA is complete once Msg2 is
received by the UE. Since the gNB cannot know when this is, simply mark
it as complete once we sent Msg2.
We still send Msg3 in CFRA, which is explained further in the commits.
Also, improve logging and stabilize an RFsim test by reducing sl_ahead.
Refactor DLSCH scheduler
Refactor the DLSCH post processor similarly to what has been done in
!3521 with the goal of saving an additional iteration through all UEs,
hopefully making the DLSCH scheduler faster.
It further includes changes to make working with and debugging multi-UE
operation easier:
- add some measurements in L1 (for total TX/RX time)
- a new option MACRLCs.[0].stats_max_ue to limit the number of UEs in
the periodic output (default: 8, 0=disable)
- avoid stack overflow in the MAC
- fix the writing of stats to nrMAC_stats.log and nrL1_stats.log when
the written output reduces by truncating the files
- avoid to schedule retransmission with 0 RBs.
Refactor PRACH handling at the gNB
- Remove race conditions by correctly setting mutexes
- Fix a bug when having multiple RACH occasions
- Reduce memory footprint by removing global memory, which should also
fix potential memory data races in FHI 7.2
- Additional fixes, see commit messages
a bit difficult to understand: sl_ahead=6 => we systematically miss the
rach detection in the scheduler because the scheduler runs more than 6
slots ahead the rx slot processing (here > 11 slots because 6 (DL and UL
slots) + 4 (RU_RX_SLOT_DEPTH, so UL only))
so, reducing sl_ahead works better than large sl_ahead because the idea
behind is: the scheduler of slot+sl_ahead will never create work to do
in a UL slot before slot+sl_ahead+X (as minimum k2 is X, in the 24PRB CI
test: 6)
Jaroslava Fiedlerova