CFRA: mark RA complete when sending Msg2
The current contention-free random access forces a "Msg3" (which does
not exist in CFRA) to be received by a UE. Sometimes, this "Msg3" is not
received (for whatever reason), and the MAC declare that RA failed.
This is problematic, as the spec says that RA is complete once Msg2 is
received by the UE. Since the gNB cannot know when this is, simply mark
it as complete once we sent Msg2.
We still send Msg3 in CFRA, which is explained further in the commits.
Also, improve logging and stabilize an RFsim test by reducing sl_ahead.
CI: Reduce the tries to attach the UE in CI runs
The goal of this MR is to save time in CI runs by reducing the UE attach tries
to 3 instead of 4 and UE timeout to 20 seconds, which will be incremented after
each fail, instead of 60 seconds.
CI build analysis: print log file errors
The generated HTML from CI builds so far only shows the number of errors, but
not the errors themselves. This is inconvenient, as this means that developers
have to go to the logs.
This commit modifies AnalyzeBuildLogs(), which already iterates the entire file,
to store the lines containing "error:" (as it would be emitted by a compiler).
The callers of AnalyzeBuildLogs() now also print the analysis outcome in a more
concise manner.
Also, change the return parameter of AnalyzeBuildLogs() to a tuple to make the
variable handling easier, and remove the unused warnings list.
Fix DRB integrity failures during handover and re-establishment
This MR fixes DRB integrity protection failures that occur during N2 handover
and connection re-establishment scenarios. The issues were caused by incorrect
security key handling and ciphering configuration during these procedures with
DRB integity enabled.
See merge request !3767 for more details.
This change updates the NSA-B200 CI pipeline to guarantee that the EPC is
properly terminated regardless of pipeline success or failure. This avoids
leaving EPC processes running after failed test.
The generated HTML from CI builds so far only shows the number of
errors, but not the errors themselves. This is inconvenient, as this
means that developers have to go to the logs.
This commit modifies AnalyzeBuildLogs(), which already iterates the
entire file, to store the lines containing "error:" (as it would be
emitted by a compiler). The callers of AnalyzeBuildLogs() now also print
the analysis outcome in a more concise manner.
Also, change the return parameter of AnalyzeBuildLogs() to a tuple to make
the variable handling easier, and remove the unused warnings list.
PDU Session Release
This MR implements end-to-end PDU Session Release functionality
according to 3GPP TS 38.413 and TS 38.331 specifications.
Reason for Change Previous implementation had incomplete and wrongly
places tunnel cleanup during PDU session release. UP cleanup shall be
done at CUUP, RRCReconfiguration shall be sent after retrieval of
updated CellGroupConfig from the DU. This implies F1 and E1 exchanges
during PDU session release procedure.
Major Changes:
1. RRC PDU Session Release: DRBs and PDU session teardown in NGAP PDU
Session Release Response callback
- Helper Functions: Introduced rm_pduSession,
find_drb_by_pdusession_id and rm_drbs() helpers to locate and clean
up DRBs linked to a specific PDU session
- Targeted Cleanup: RRC PDU Session and per-session DRB removal in
rrc_gNB_send_NGAP_PDUSESSION_RELEASE_RESPONSE()
- Status Cleanup: Removed unused PDU_SESSION_STATUS_RELEASED status
2. RRC PDU Session Release: Refactor
rrc_gNB_process_NGAP_PDUSESSION_RELEASE_COMMAND
- Any PDU session in the pdusession_release_params list that is
present in the RRC list can be set to release, regardless of its
status
- Function Signature Update: Updated function signature and other
minor changes
- Response Handling: rrc_gNB_send_NGAP_PDUSESSION_RELEASE_RESPONSE is
sent upon completion of RRCReconfiguration
3. RRC PDU Session Release: add missing procedures E1AP Integration:
- Populates E1AP Bearer Modification for each to be released session
- PDCP Enhancement: Added nr_pdcp_release_drbs() to remove all DRBs
linked to a given PDU session in CUUP
- GTP-U Cleanup: GTP tunnel teardown handled in CUUP
e1_bearer_context_modif() using correct F1 or N3 GTP instance
4. NR PDCP/GTP:
- Enhanced Tunnel Deletion Logic: Separate handling for N3 tunnels (1
per PDU session) and F1-U tunnels (1 per DRB)
- New PDCP API Function: Added nr_pdcp_get_drb_ids_for_pdusession()
function to retrieve all DRB IDs associated with a specific PDU
session
5. RRC PDU Session Release: F1AP-based UE Context Modification:
- Replaced direct RRC message transfer with F1AP UE Context
Modification Request for proper retrieval from DU
- Cell Group Configuration: handling of updated CellGroupConfig from
DU ensures DRB release at CU
6. Docs and testing:
- Added PDU session add/remove test cases to 25 PRB RFSIM pipeline
- Documentation: Added PDU Session Release sequence diagram
- Telnet Commands: Added ci command to trigger NGAP PDU Session
Release and ciUE command to trigger establishment of a new PDU
Session
7. Additional fixes:
- UE RLC Management: ensure release of RLC entities
- Unify NG delay control before calling the RRC NGAP handler, for both
Setup and Release
8. Code Cleanup: Cleaned up unused GTP teardown functions.
Closes#867
- Enable telnet server on VNF (gNB) and UE in docker-compose.yaml
- Add new test cases for PDU session management:
* Add PDU session via UE telnet
* Release PDU session via telnet
* Verify connectivity after PDU session release
- Integrate telnet commands into existing container_5g_rfsim_u0_25prb.xml pipeline
- Test cases added before undeploy step in proper execution order
Remove schedule response and update L1 threading documentation
This merge requests removes the somewhat superfluous "schedule response"
mechanism. "schedule response" was run after running the scheduler
itself and copied various FAPI messages into a global msgDataTx
structure from which the various TX/RX jobs were triggered. It is not
necessary, though, given that the scheduler is run inside the tx_func()
(running the TX jobs of the L1), and so the scheduler output FAPI
messages can directly be read into a variable of the existing
NR_Sched_Rsp_t type. This allows to delete a couple of hundred lines of
code (while retaining functionality), and simplifies the msgDataTx type
(which is still there to trigger TX job (including the scheduler) from
the ru thread).
In short, prior, there was this call chain:
- tx_func() calls run_scheduler_monolithic() (through fptr
NR_slot_indication())
- get some memory through allocate_sched_response() which has enough
space for NR_Sched_Rsp_t type variable
- call the scheduler and fill the variable
- call nr_schedule_response() (through fptr NR_schedule_response())
which copies into L1: to msgDataTx for TX, to separate structures
for RX
this is changed to
- tx_func() runs one message at a time, hence we can use memory in the
data segment for NR_Sched_Rsp_t. It calls run_scheduler_monolithic()
through the fptr
- pointers into NR_Sched_Rsp_t are passed to the scheduler, which
fills as before
- we return, memory is in its right place and does not need an extra
copy.
Additionally, this merge request updates documentation on threading and
cleans up some unused code along the way.
[MAC] Add UL total/used aggregated MAC stats; [E2 agent] Properly calculate
used UE PRBs in KPM; [CI] Simplify and improve the Undeploy_Object() and
implement Stop_Object()
- Create common dlul_mac_stats_t struct for both DL and UL total/used aggregated
PRBs
- Use correct unit [%] for RRU.PrbTotDl and RRU.PrbTotUl , instead of [PRBs]
- Use correct unit [Mb] for DRB.PdcpSduVolumeDL and DRB.PdcpSduVolumeUL [kb]
- Implement Stop_Object() (docker compose stop = SIGTERM)
- Simplify the Undeploy_Object() function
- Improve the logging for Deploy/Undeploy_Object() functions
- OAI-FlexRIC CI: properly stop all the containers (one by one or a subgroup of
running containers), and rename oai-flexric to nearRT-RIC container
1. Rename oai-flexric to nearRT-RIC. FlexRIC is a project combining nearRT-RIC, xApps, and E2 agent emulators.
2. Instead of undeploying all the containers at the same time, and not checking the logs for xApps and nearRT-RIC, do the following:
- Stop containers (docker compose stop) - one or subgroup of running containers in the following order:
a) xapp-kpm-moni
b) xapp-kpm-rc
c) xapp-gtp-mac-rlc-pdcp-moni
d) oai-nr-ue oai-nr-ue2
e) oai-gnb
f) nearRT-RIC
- Undeploy containers (docker compose down), collect and analyze all the logs.
For some test cases, we might want to stop a service earlier. This
functionality is done by StopObject().
Test log collection is still done by UndeployObject(), which will
collect the logs of all services (at that point, stopped or not).
GetRunningServices() was returning all services, running or stopped.
Hence, rename to reflect what this function does, and rewrite slightly
to better capture what is actually happening. Also, don't return the
container ID, as we don't use it.
Call stop on all services, as we will down the deployment right after.
The list of services is still used to retrieve logs.
Remove a stray "return False", which would sometimes prevent us from
seeing all error messages.
CI: Add unified Jenkinsfile for all test pipelines
This MR introduces a single Jenkinsfile to cover all current CI test pipelines
Improvements:
- correct reporting of the start timestamp in the HTML report
- harmonize log collection - always collect as test_logs_${env.BUILD_ID}.zip
PRS bug fix and CI Integration
- Fix the bug in PRS channel estimation while copying memory to compute the
impulse response
- Integrate PRS testing to CI
This commit inputs several distances at once and obtain the summary of
successful and failed tests. We look for the number of successful tests
that are geater than 0. We do that here since we sometimes dont receive
a response for a telnet command from the rfsim although the command
is applied. This is related to the queing telnet commands in the rfsim.
The script for the distances 50, 100 and 150 are called as follows,
./set-and-verify-distance-prs.sh 50 100 150
The summary can be viewed as follows,
==================== SUMMARY ====================
Total tests run : 3
Successful tests: 3
Failed tests : 0
=================================================
Set RFsim (DL) distance and use PRS to estimate the new distance. This
is verified by comparing the distance set in RFsim with the distance
reported through ToA estimation at the UE.
Note that since RFsim telnet output is asynchronous (it uses a queue
internally), we use grep --max-count 1 to wait for the matching line,
together with ncat --idle 0.3 to keep ncat open for some time. The
--max-count option will terminate grep after the first occurrence. For
this to work reliably, we need to use grep <(echo | ncat) as opposod to
simply echo | ncat | grep, as the latter does not seem to reliably
make grep exit after the occurrence.
Co-authored-by: Robert Schmidt <robert.schmidt@openairinterface.org>
Handle SIGINT and mark any subsequent steps as failed. Receiving SIGINT
a second time will abort the script immediately (as is the case before
this commit).
Note that Python seems to relay the signal to the "controlled" process
(e.g., Ping), which would return immediately with an error. So some
steps like Ping will abort immediately, others (e.g., IdleSleep) will
finish their step, and then the present logic will mark subsequent steps
as failed.
Some CI tasks/commands like Iperf(), Ping() get multiple nodes (via XML
parameter "nodes") to potentially run UEs on different nodes (hosts) at
the same time. However, I argue that this is not good:
- we don't actually use this -- where we specify multiple UEs, it's
always "localhost localhost..."
- it is inconsistent, as we typically have a single "node", not nodes,
and there is a possibility to harmonize (see also next commit)
- if we needed it, there would be better ways to achieve the same.
First, for hardware-based UEs, the ci_infra.yaml can specify different
UEs. For simulated UEs, it would be feasible to have multiple XML
steps run in parallel, e.g., in the XML <parallel> tag, which would
clarify that multiple UEs on different hosts run in parallel
- it reduces code.
Hence, make a single <node> for these XML steps.
Created Jenkinsfiles:
- Jenkinsfile: for stadard test pipelines
- Jenkinsfile-oc: for deployments with OC
These 2 Jenkinsfiles covers all current test pipelines.
Change registry name env var content
Currently setting REGISTRY="" in a .env file results in docker failing to find
/image-name:tag. This corrects that, so it looks for local images named
image-name:tag.
Following the change to use:
image: ${REGISTRY-oaisoftwarealliance/}${GNB_IMG:-oai-gnb}:${TAG:-develop}
the REGISTRY variable must now include the trailing slash.
maki
Robert Schmidt