- Trigger handover
- IdleSleep 2s
- Ping ext-dn from NR-UE
- Verify UE(1) connected to DU-PCI1
Validate absence of heap-use-after-free ASAN error found in !3600.
Closes#940
- On PUSH events, if image build stages succeed, trigger DockerHub push stage
- Build the images, then trigger the Push to DockerHub stage
- Publish Jetson (ARM64/v8) and AMD64 images under the same tag on DockerHub
The next commit introduces an "UL-heavy" configuration to be tested with
Aerial. To avoid waiting for the CN, start&stop it once at the beginning
of the CI run.
Reduce throughput requirement, as this fails once two DUs are connected
to a UE, which creates more load on the machine and frequently fails
with low throughput.
CI: Enable Physim threshold check for tests on Caracal and GH
This MR adds support for configurable timing threshold files in PhySim
tests.
- Introduces a new CMake cache variable PHYSIM_CHECK_FILES to specify
one or more timing threshold files (semicolon-separated).
- Enables optional enforcement of timing thresholds check in PhySim
tests without changing existing test definitions.
- Defines timing thresholds for LDPC encoding/decoding in nr_ulsim,
nr_dlsim and ldpctest on Caracal and GH machines.
Also, don't use "set -x" debugging for oc login command:
ci-scripts/cls_cmd.py:242 sets BASH_XTRACEFD=1, which makes that all
"set -x" output is in stdout, leading to a redirect to the file into
which we redirect. This would lead to the following error with
unsuccessfull oc login:
error: You are not a member of project "****-core-for-ci-ran".
Error from server (Forbidden): imagestreams.image.openshift.io "oai-physim"
is forbidden: User "system:anonymous" cannot get resource "imagestreams" in
API group "image.openshift.io" in the namespace "****-ran"
This fixes basically two problems. First, remove "set -e", which leads
to the script exiting on error. For instance, there was the sleep which
should account for the time of the pod being in "ContainerCreating", but
might not be enough. In that case, the script would exit, leaving the
pod deployed.
Second, a fixed time is not good, as it can be too short and unduly
prolong waiting time. Replace with a loop.
Add OAIUE longrun pipeline
Add a new XML file for OAIUE longrun.
- Test description:
* 40 MHz BW, n78
* gNB with USRP N310, UE with USRP N310
* CN deployed on OC
- Test duration: 1 hour
- Test scenario: 3x DL 10 min test (20 Mbps) + 3x UL 10 min test (5
Mbps)
pods deployed through helm occasionally take more time, leading to
errors such as
Error from server (BadRequest): container "physim" in pod "oai-physims-4g-sczvl" is waiting to start: ContainerCreating
Give a bit more time before checking the logs. We cannot wait too much,
as the container will just end on FINISHED in running state, so the logs
will always be there.
Previously, the script attempted to directly merge ${merge}, which could f
ail if the branch was not already available locally. This update explicitly
fetches the branch from origin and then merges FETCH_HEAD with --ff, ensuring
the merge works even when the branch is missing locally.
- Remove extensive traffic testing before HO (iperf tests 030001, 030002)
- Do 1 ping test after attachment
- Reduce ping packet count from 100 to 50 packets for remaining connectivity test
Time savings: ~2-3 minutes by removing traffic validation overhead
`def` creates a variable local to the script object, not automatically visible inside helper functions.
This fixes the following exception in Jenkins:
Found unhandled groovy.lang.MissingPropertyException exception:
No such property: OAI_Registry for class: groovy.lang.Binding
Lower the RX gain used by the Amarisoft UE in CI tests to prevent DL signal
saturation. This resolves an issue with low DL throughput observed in CI.
RX/TX signal levels on the Amarisoft UE can be checked with "t spl" command
in lteue; MAX values should not approach 0.
Extend t300 in multi_ue rfsim testcase
Extend t300 to 1000 to increase the time the gNB has to handle MSG3. Hopefully
this prevents the UEs from timing out after MSG3. Add UE id to timer timeout
LOGs so the UE behavior can be traced through the stack.
Code cleanup (SRS, config files and more)
- Remove deprecated elements from some configuration files (also closes#997)
- Improve SRS code with a better separation between common code and gNB/UE code
- Other minor compilation improvements
N2 handover
This MR introduces full support for N2 handover, covering handover
decision, preparation, execution, and notification between source and
target NG-RAN nodes.
Key features:
1. Handover Decision & Preparation:
- Implement N2 handover decision based on UE measurement reports.
- Generate the Handover Preparation Information message, including
RRCReconfiguration.
- Send Handover Required message from the source NG-RAN to the AMF.
2. Handover Execution:
- Decode and handle the Handover Request on the target NG-RAN.
- Process the Handover Command on the source gNB, triggering
RRCReconfiguration.
- Initiate Handover Notify after successful completion.
3 NGAP & RRC Enhancements:
- Implement NGAP encoding/decoding functions for Handover Required,
Handover Request, Handover Command, Handover Failure, and Handover
Notify (3GPP TS 38.413).
- Adds NG-RAN Status Transfer support (UL/DL) for PDCP COUNT
preservation during mobility:
- support for NGAP Uplink and Downlink RAN Status Transfer messages
(TS 38.413 §9.2.3.13-14) to enable proper PDCP COUNT value
transfer during handover or recovery: NGAP encoding/decoding for
RAN Status Transfer, PDCP helpers to extract and apply COUNT (HFN
+ SN) per DRB. This aligns with TS 38.300 §9.2.3.2.1 and TS 23.502
§4.9.1.3.3, ensuring PDCP status preservation in AM.
4. Testing:
- Introduce a Telnet command to manually trigger N2 handover using UE
ID and neighbor PCI.
- Add configuration file for testing N2 handover with a target gNB.
CI: build images on Jetson (armv8)
Add a new build pipeline for NVIDIA Jetson (armv8). Images have tag
prefix armv8_ to distinguish from gracehopper-built images prefixed arm_
(should have maybe been better with armv9_?)
The pipeline is using the already existing conf files
for the source/target gNB CUs/DUs.
For the sake of simplicity, neighbour-config.conf has
been rewritten and simplified in this commit, then
adopted in this pipeline. The file was not used before.
For clarity, rename to native_armv9 for clarity. The check inside Python
checked for native_arm, and if we introduce native_armv8 (for jetson),
it would also match that.
Add nr-cuup to run_locally.sh
E1 deploy wasn't enable in the run_locally.sh script to run the CI on a local
machine. This MR is adding it. It was validated locally.
CI: Generalize PhySim tests, run PhySims on Gracehopper
This changeset generalizes/harmonizes the CI infrastructure for physim
testing, and adds a new pipeline for physim tests on Gracehopper
machines.
Specifically:
- Refactor the existing CI code for running and analyzing physical
simulators. The existing code assumed those tests are deployed using
OpenShift. Instead, OpenShift/Helm-specificities (e.g., running a
"release") have been removed, and instead of hardcoding the deployment
of the physical simulators using a script for Helm, the function takes
any script that can deploy using Helm/docker/from source. The overall
appearance is the same.
- Add a new pipeline RAN-PhySim-GraceHopper-5G that executes physical
simulators on gracehopper. Reporting is done in the same way as for
existing physical simulators (e.g., like in the RAN-PhySim-Cluster-5G
pipeline), but deployed in docker instead of helm.
-> some tests don't work reliably on ARM yet and are skipped
- the RAN-gNB-N300-Timing-Phytest-LDPC pipeline has been reworked to use
the same code for testing T2 offload, with ctest, and the same
reporting.
- the CUDA LDPC test XML has been removed, as these tests don't work
reliably.
- add a new macro add_timed_physim_test() that allows to define
thresholds to be checked, so that ctest will check for them. see the
documentation for more information
CI: Creation of F1 Handover pipeline
In this MR we are modifying/adding all the required files (.xml, .conf,
.yml) in order to create a pipeline on Jenkins that allows us to test
OTA F1 handover.
Remove the runtime, which is not interesting, and add instead
- test name and if the test failed
- the threshold checks, if any, and
- possible additional information about test failures.
Prior to this change, each test would have, as label, it's description.
This leads to many labels, which are normally reserved for grouping
tests. Move the description into a custom property instead.
Since we rely on the JSON output for the generation of HTML reports, we
need to also extract the description from the generated JSON. However,
custom properties are only logged starting from cmake 3.30. For
back-wards compatibility, write an additional, separate environment
variable TEST_DESCRIPTION from which the description can be read as a
fallback.
For the add_timed_physim_test(), this has been already done in a parent
commit.
Robert Schmidt