PDU Session Release
This MR implements end-to-end PDU Session Release functionality
according to 3GPP TS 38.413 and TS 38.331 specifications.
Reason for Change Previous implementation had incomplete and wrongly
places tunnel cleanup during PDU session release. UP cleanup shall be
done at CUUP, RRCReconfiguration shall be sent after retrieval of
updated CellGroupConfig from the DU. This implies F1 and E1 exchanges
during PDU session release procedure.
Major Changes:
1. RRC PDU Session Release: DRBs and PDU session teardown in NGAP PDU
Session Release Response callback
- Helper Functions: Introduced rm_pduSession,
find_drb_by_pdusession_id and rm_drbs() helpers to locate and clean
up DRBs linked to a specific PDU session
- Targeted Cleanup: RRC PDU Session and per-session DRB removal in
rrc_gNB_send_NGAP_PDUSESSION_RELEASE_RESPONSE()
- Status Cleanup: Removed unused PDU_SESSION_STATUS_RELEASED status
2. RRC PDU Session Release: Refactor
rrc_gNB_process_NGAP_PDUSESSION_RELEASE_COMMAND
- Any PDU session in the pdusession_release_params list that is
present in the RRC list can be set to release, regardless of its
status
- Function Signature Update: Updated function signature and other
minor changes
- Response Handling: rrc_gNB_send_NGAP_PDUSESSION_RELEASE_RESPONSE is
sent upon completion of RRCReconfiguration
3. RRC PDU Session Release: add missing procedures E1AP Integration:
- Populates E1AP Bearer Modification for each to be released session
- PDCP Enhancement: Added nr_pdcp_release_drbs() to remove all DRBs
linked to a given PDU session in CUUP
- GTP-U Cleanup: GTP tunnel teardown handled in CUUP
e1_bearer_context_modif() using correct F1 or N3 GTP instance
4. NR PDCP/GTP:
- Enhanced Tunnel Deletion Logic: Separate handling for N3 tunnels (1
per PDU session) and F1-U tunnels (1 per DRB)
- New PDCP API Function: Added nr_pdcp_get_drb_ids_for_pdusession()
function to retrieve all DRB IDs associated with a specific PDU
session
5. RRC PDU Session Release: F1AP-based UE Context Modification:
- Replaced direct RRC message transfer with F1AP UE Context
Modification Request for proper retrieval from DU
- Cell Group Configuration: handling of updated CellGroupConfig from
DU ensures DRB release at CU
6. Docs and testing:
- Added PDU session add/remove test cases to 25 PRB RFSIM pipeline
- Documentation: Added PDU Session Release sequence diagram
- Telnet Commands: Added ci command to trigger NGAP PDU Session
Release and ciUE command to trigger establishment of a new PDU
Session
7. Additional fixes:
- UE RLC Management: ensure release of RLC entities
- Unify NG delay control before calling the RRC NGAP handler, for both
Setup and Release
8. Code Cleanup: Cleaned up unused GTP teardown functions.
Closes#867
- Enable telnet server on VNF (gNB) and UE in docker-compose.yaml
- Add new test cases for PDU session management:
* Add PDU session via UE telnet
* Release PDU session via telnet
* Verify connectivity after PDU session release
- Integrate telnet commands into existing container_5g_rfsim_u0_25prb.xml pipeline
- Test cases added before undeploy step in proper execution order
Remove schedule response and update L1 threading documentation
This merge requests removes the somewhat superfluous "schedule response"
mechanism. "schedule response" was run after running the scheduler
itself and copied various FAPI messages into a global msgDataTx
structure from which the various TX/RX jobs were triggered. It is not
necessary, though, given that the scheduler is run inside the tx_func()
(running the TX jobs of the L1), and so the scheduler output FAPI
messages can directly be read into a variable of the existing
NR_Sched_Rsp_t type. This allows to delete a couple of hundred lines of
code (while retaining functionality), and simplifies the msgDataTx type
(which is still there to trigger TX job (including the scheduler) from
the ru thread).
In short, prior, there was this call chain:
- tx_func() calls run_scheduler_monolithic() (through fptr
NR_slot_indication())
- get some memory through allocate_sched_response() which has enough
space for NR_Sched_Rsp_t type variable
- call the scheduler and fill the variable
- call nr_schedule_response() (through fptr NR_schedule_response())
which copies into L1: to msgDataTx for TX, to separate structures
for RX
this is changed to
- tx_func() runs one message at a time, hence we can use memory in the
data segment for NR_Sched_Rsp_t. It calls run_scheduler_monolithic()
through the fptr
- pointers into NR_Sched_Rsp_t are passed to the scheduler, which
fills as before
- we return, memory is in its right place and does not need an extra
copy.
Additionally, this merge request updates documentation on threading and
cleans up some unused code along the way.
[MAC] Add UL total/used aggregated MAC stats; [E2 agent] Properly calculate
used UE PRBs in KPM; [CI] Simplify and improve the Undeploy_Object() and
implement Stop_Object()
- Create common dlul_mac_stats_t struct for both DL and UL total/used aggregated
PRBs
- Use correct unit [%] for RRU.PrbTotDl and RRU.PrbTotUl , instead of [PRBs]
- Use correct unit [Mb] for DRB.PdcpSduVolumeDL and DRB.PdcpSduVolumeUL [kb]
- Implement Stop_Object() (docker compose stop = SIGTERM)
- Simplify the Undeploy_Object() function
- Improve the logging for Deploy/Undeploy_Object() functions
- OAI-FlexRIC CI: properly stop all the containers (one by one or a subgroup of
running containers), and rename oai-flexric to nearRT-RIC container
1. Rename oai-flexric to nearRT-RIC. FlexRIC is a project combining nearRT-RIC, xApps, and E2 agent emulators.
2. Instead of undeploying all the containers at the same time, and not checking the logs for xApps and nearRT-RIC, do the following:
- Stop containers (docker compose stop) - one or subgroup of running containers in the following order:
a) xapp-kpm-moni
b) xapp-kpm-rc
c) xapp-gtp-mac-rlc-pdcp-moni
d) oai-nr-ue oai-nr-ue2
e) oai-gnb
f) nearRT-RIC
- Undeploy containers (docker compose down), collect and analyze all the logs.
For some test cases, we might want to stop a service earlier. This
functionality is done by StopObject().
Test log collection is still done by UndeployObject(), which will
collect the logs of all services (at that point, stopped or not).
GetRunningServices() was returning all services, running or stopped.
Hence, rename to reflect what this function does, and rewrite slightly
to better capture what is actually happening. Also, don't return the
container ID, as we don't use it.
Call stop on all services, as we will down the deployment right after.
The list of services is still used to retrieve logs.
Remove a stray "return False", which would sometimes prevent us from
seeing all error messages.
CI: Add unified Jenkinsfile for all test pipelines
This MR introduces a single Jenkinsfile to cover all current CI test pipelines
Improvements:
- correct reporting of the start timestamp in the HTML report
- harmonize log collection - always collect as test_logs_${env.BUILD_ID}.zip
PRS bug fix and CI Integration
- Fix the bug in PRS channel estimation while copying memory to compute the
impulse response
- Integrate PRS testing to CI
This commit inputs several distances at once and obtain the summary of
successful and failed tests. We look for the number of successful tests
that are geater than 0. We do that here since we sometimes dont receive
a response for a telnet command from the rfsim although the command
is applied. This is related to the queing telnet commands in the rfsim.
The script for the distances 50, 100 and 150 are called as follows,
./set-and-verify-distance-prs.sh 50 100 150
The summary can be viewed as follows,
==================== SUMMARY ====================
Total tests run : 3
Successful tests: 3
Failed tests : 0
=================================================
Set RFsim (DL) distance and use PRS to estimate the new distance. This
is verified by comparing the distance set in RFsim with the distance
reported through ToA estimation at the UE.
Note that since RFsim telnet output is asynchronous (it uses a queue
internally), we use grep --max-count 1 to wait for the matching line,
together with ncat --idle 0.3 to keep ncat open for some time. The
--max-count option will terminate grep after the first occurrence. For
this to work reliably, we need to use grep <(echo | ncat) as opposod to
simply echo | ncat | grep, as the latter does not seem to reliably
make grep exit after the occurrence.
Co-authored-by: Robert Schmidt <robert.schmidt@openairinterface.org>
Handle SIGINT and mark any subsequent steps as failed. Receiving SIGINT
a second time will abort the script immediately (as is the case before
this commit).
Note that Python seems to relay the signal to the "controlled" process
(e.g., Ping), which would return immediately with an error. So some
steps like Ping will abort immediately, others (e.g., IdleSleep) will
finish their step, and then the present logic will mark subsequent steps
as failed.
Some CI tasks/commands like Iperf(), Ping() get multiple nodes (via XML
parameter "nodes") to potentially run UEs on different nodes (hosts) at
the same time. However, I argue that this is not good:
- we don't actually use this -- where we specify multiple UEs, it's
always "localhost localhost..."
- it is inconsistent, as we typically have a single "node", not nodes,
and there is a possibility to harmonize (see also next commit)
- if we needed it, there would be better ways to achieve the same.
First, for hardware-based UEs, the ci_infra.yaml can specify different
UEs. For simulated UEs, it would be feasible to have multiple XML
steps run in parallel, e.g., in the XML <parallel> tag, which would
clarify that multiple UEs on different hosts run in parallel
- it reduces code.
Hence, make a single <node> for these XML steps.
Created Jenkinsfiles:
- Jenkinsfile: for stadard test pipelines
- Jenkinsfile-oc: for deployments with OC
These 2 Jenkinsfiles covers all current test pipelines.
Change registry name env var content
Currently setting REGISTRY="" in a .env file results in docker failing to find
/image-name:tag. This corrects that, so it looks for local images named
image-name:tag.
Following the change to use:
image: ${REGISTRY-oaisoftwarealliance/}${GNB_IMG:-oai-gnb}:${TAG:-develop}
the REGISTRY variable must now include the trailing slash.
Refactor DLSCH scheduler
Refactor the DLSCH post processor similarly to what has been done in
!3521 with the goal of saving an additional iteration through all UEs,
hopefully making the DLSCH scheduler faster.
It further includes changes to make working with and debugging multi-UE
operation easier:
- add some measurements in L1 (for total TX/RX time)
- a new option MACRLCs.[0].stats_max_ue to limit the number of UEs in
the periodic output (default: 8, 0=disable)
- avoid stack overflow in the MAC
- fix the writing of stats to nrMAC_stats.log and nrL1_stats.log when
the written output reduces by truncating the files
- avoid to schedule retransmission with 0 RBs.
Refactor PRACH handling at the gNB
- Remove race conditions by correctly setting mutexes
- Fix a bug when having multiple RACH occasions
- Reduce memory footprint by removing global memory, which should also
fix potential memory data races in FHI 7.2
- Additional fixes, see commit messages
a bit difficult to understand: sl_ahead=6 => we systematically miss the
rach detection in the scheduler because the scheduler runs more than 6
slots ahead the rx slot processing (here > 11 slots because 6 (DL and UL
slots) + 4 (RU_RX_SLOT_DEPTH, so UL only))
so, reducing sl_ahead works better than large sl_ahead because the idea
behind is: the scheduler of slot+sl_ahead will never create work to do
in a UL slot before slot+sl_ahead+X (as minimum k2 is X, in the 24PRB CI
test: 6)
CI: ensure clean iperf3 server startup
- Resolves issue with iperf3 test failure in RFSim5G multiue test
- Minor fix of iperf3 test descriptions in RFSim5G pipeline
Fix for scenario with no CSI report configured for CSI-RS
After changes in !3714 (merged), if we select to measure RSRP on SSB and not
CSI-RS (default option) and there is a single antenna port (so no MIMO report
either) we won't schedule any CSI-RS measurements for the UE. In that case
ideally we shouldn't set do_CSIRS = 1 in configuration file but still the
option is possible. So the fix is to not perform any measurements at UE on
that CSI-RS but not asserting.
Closes#1015
Before starting a new iperf3 server instance in CI tests, explicitly kill any existing
iperf3 processes bound to the same port. This prevents conflicts or failures caused by
leftover iperf3 servers from previous runs that did not terminate properly or on time.
The cleanup is performed using `pkill` before launching the new server.
The gNB will stop the periodical logging of UEs in MAC Info level after
a configurable number of UEs have been printed, by default 8.
Set it to 17 in some CI tests to ensure we still get all UEs for the
time being, as the CI might depend on this.
Improve handling of selection of RSRP/SINR report via configuration file
In current develop, the selection of SSB or CRI RSRP depends on the
do_CSIRS flag being activated. That's not wanted. CRI report for now it
is never handled, while we need SSB RSRP report for beam switching
regardless of CSI-RS configuration.
[CI] Update the Push to DockerHub Stage
This MR addresses two cases:
1. Error Handling: Ensures subsequent Image Test Processes stages
continue to run even if the preceding Push to DockerHub stage fails
(applies when the condition is PUSH event and doBuild is true).
2. Introduces conditional logic to correctly handle multi-architecture
images: Images built for both AMD64 and Jetson ARMv8 (oai-gnb,
oai-nr-ue, oai-nr-cuup) use docker buildx imagetools for push to
DockerHub.
All other AMD64 images are pushed using standard Docker commands.
Related MR !3691
- Push both AMD64 & ARMv8 Jetson images together for images built on both using `docker buildx imagetools`
- Use standard 'docker pull and push' for images built only on AMD64.
- Add XML pipeline for N2 handover testing
- Add Docker compose for two monolithic gNBs
- Features:
* Two monolithic gNBs (gNB1, gNB2) with proper CU+DU parameters
* Gradual attenuation control using RC4DAT attenuator (5s duration)
* N2 handover sequence: gNB1 → gNB2 → gNB1
* Proper channel mapping: ch1-2 for gNB1, ch3-4 for gNB2
* Telnet server configuration for CI commands
* Handover-specific neighbor configuration
* Clean test state management with attenuator reset
Update based on !3645
Note: For some test configurations we still need to provide
ra_ResponseWindow, for more details, check commit message of
41d14b571c
By configuring the Jenkins pythonExecutor to match the test machine, we can use
localhost as the node for running individual test cases. This change makes it
easy to move tests across machines, as the only adjustment needed is updating
the pythonExecutor in Jenkins.
Robert Schmidt