The next commit introduces an "UL-heavy" configuration to be tested with
Aerial. To avoid waiting for the CN, start&stop it once at the beginning
of the CI run.
Reduce throughput requirement, as this fails once two DUs are connected
to a UE, which creates more load on the machine and frequently fails
with low throughput.
CI: Enable Physim threshold check for tests on Caracal and GH
This MR adds support for configurable timing threshold files in PhySim
tests.
- Introduces a new CMake cache variable PHYSIM_CHECK_FILES to specify
one or more timing threshold files (semicolon-separated).
- Enables optional enforcement of timing thresholds check in PhySim
tests without changing existing test definitions.
- Defines timing thresholds for LDPC encoding/decoding in nr_ulsim,
nr_dlsim and ldpctest on Caracal and GH machines.
Add OAIUE longrun pipeline
Add a new XML file for OAIUE longrun.
- Test description:
* 40 MHz BW, n78
* gNB with USRP N310, UE with USRP N310
* CN deployed on OC
- Test duration: 1 hour
- Test scenario: 3x DL 10 min test (20 Mbps) + 3x UL 10 min test (5
Mbps)
- Remove extensive traffic testing before HO (iperf tests 030001, 030002)
- Do 1 ping test after attachment
- Reduce ping packet count from 100 to 50 packets for remaining connectivity test
Time savings: ~2-3 minutes by removing traffic validation overhead
N2 handover
This MR introduces full support for N2 handover, covering handover
decision, preparation, execution, and notification between source and
target NG-RAN nodes.
Key features:
1. Handover Decision & Preparation:
- Implement N2 handover decision based on UE measurement reports.
- Generate the Handover Preparation Information message, including
RRCReconfiguration.
- Send Handover Required message from the source NG-RAN to the AMF.
2. Handover Execution:
- Decode and handle the Handover Request on the target NG-RAN.
- Process the Handover Command on the source gNB, triggering
RRCReconfiguration.
- Initiate Handover Notify after successful completion.
3 NGAP & RRC Enhancements:
- Implement NGAP encoding/decoding functions for Handover Required,
Handover Request, Handover Command, Handover Failure, and Handover
Notify (3GPP TS 38.413).
- Adds NG-RAN Status Transfer support (UL/DL) for PDCP COUNT
preservation during mobility:
- support for NGAP Uplink and Downlink RAN Status Transfer messages
(TS 38.413 §9.2.3.13-14) to enable proper PDCP COUNT value
transfer during handover or recovery: NGAP encoding/decoding for
RAN Status Transfer, PDCP helpers to extract and apply COUNT (HFN
+ SN) per DRB. This aligns with TS 38.300 §9.2.3.2.1 and TS 23.502
§4.9.1.3.3, ensuring PDCP status preservation in AM.
4. Testing:
- Introduce a Telnet command to manually trigger N2 handover using UE
ID and neighbor PCI.
- Add configuration file for testing N2 handover with a target gNB.
The pipeline is using the already existing conf files
for the source/target gNB CUs/DUs.
For the sake of simplicity, neighbour-config.conf has
been rewritten and simplified in this commit, then
adopted in this pipeline. The file was not used before.
For clarity, rename to native_armv9 for clarity. The check inside Python
checked for native_arm, and if we introduce native_armv8 (for jetson),
it would also match that.
CI: Generalize PhySim tests, run PhySims on Gracehopper
This changeset generalizes/harmonizes the CI infrastructure for physim
testing, and adds a new pipeline for physim tests on Gracehopper
machines.
Specifically:
- Refactor the existing CI code for running and analyzing physical
simulators. The existing code assumed those tests are deployed using
OpenShift. Instead, OpenShift/Helm-specificities (e.g., running a
"release") have been removed, and instead of hardcoding the deployment
of the physical simulators using a script for Helm, the function takes
any script that can deploy using Helm/docker/from source. The overall
appearance is the same.
- Add a new pipeline RAN-PhySim-GraceHopper-5G that executes physical
simulators on gracehopper. Reporting is done in the same way as for
existing physical simulators (e.g., like in the RAN-PhySim-Cluster-5G
pipeline), but deployed in docker instead of helm.
-> some tests don't work reliably on ARM yet and are skipped
- the RAN-gNB-N300-Timing-Phytest-LDPC pipeline has been reworked to use
the same code for testing T2 offload, with ctest, and the same
reporting.
- the CUDA LDPC test XML has been removed, as these tests don't work
reliably.
- add a new macro add_timed_physim_test() that allows to define
thresholds to be checked, so that ctest will check for them. see the
documentation for more information
CI: Creation of F1 Handover pipeline
In this MR we are modifying/adding all the required files (.xml, .conf,
.yml) in order to create a pipeline on Jenkins that allows us to test
OTA F1 handover.
RF emulator and improvements of the timers
RF emulator:
Add an RF library emulating a real device without the need of a real
device. This is thought to enable using phy-test without requiring a
radio device or channel simulation. The library synchronizes on the
real time clock, drops TX and generate noise on RX. Noise generation
can be disabled and the noise level configured.
Timers improvements:
- Fix standard deviation for merged timers
- Add display of ULSCH decoding timers
- Add decoding timer in AAL
CI: speed up and simplify build, remove unnecessary Python code, remove --eNB* parameter list
This cleans up the CI framework for running end-to-end tests, and
improves the integration with OAI code. Specifically
- cleanup build_oai and helpers to output all text, which is captured
better by the CI
- build some images in the Cluster-Build in parallel to increase speed
- remove all --eNB{,1,2}{IPAddress,UserName,Password} variables. Those
were indexed through "instance" and "server ID" and are replaced with
the actual node name. I consider further cleanup to introduce a
"global" XML file node name so we don't need this in each step and
instead define this once.
- individual cleanup
It is superseded by the CreateWorkspace() function and task step (which
removes and recreates a workspace), and this option was not actually
used in any function except for the build of unit tests, but no XML
step actually specified this option.
Reimplement these functions by
- using cls_cmd
- retain only the necessary steps from the timing-phytest function
- pass in ctx and node information, instead of server ID
- remove all superfluous variables that don't seem to be used
- make some lists (corresponding indices are "server ID") into normal
variables.
Pass node directly from XML to these functions
- BuildClusterImage()
- BuildImage()
- BuildProxy()
- BuildRunTests()
instead of reading the "server ID" and looking up from
eNB_IPAddress/eNB1_IPAddress/eNB2_IPAddress.
Improve the proxy build unit test by properly passing ctx information.
Make Amarisoft UE sim a RedCap UE
Change Amarisoft UE configuration and gNB of the AW2S pipeline to
be/handle RedCap UEs.
The UE should only connect if the gNB handles RedCap UEs. In the logs,
we should see
[MAC] UE with RNTI 3725 is RedCap
which we check for the first two UEs (one normal UE/absence of this
message, one RedCap UE/presence of this message).
Since the parent commit, UE 1 is normal (no RedCap), UE 2 is RedCap.
Verify that the gNB agrees with that and detects the second UE as RedCap
by analyzing some logs.
The first check is a bit elaborate, but basically is
[ $(docker logs | grep XYZ | wc -l) -eq 0 ]
^ ^ ^ ^ ^
1 2 3 4 5
(1) is the if statement, that compares
(4) the number of occurrences
(3) of "RedCap" in the
(2) docker logs
(5) to being equal to
Note for all UEs which number, and whether it is RedCap (so this need to
be kept in sync with the Amarisoft UE sim configuration file in use!).
CI: Refactor log collection, collect also (COTS) UE logs
Refactor log handling in CI: collect according to currently used XML files. This
makes it homogeneous in the pipelines, and avoids accidental overwriting (as
happens in phytest-timing), as well as harmonizing the log placement
(e.g., no YAML lookup in Ping/Iperf).
Allow to trace and archive UE logs (e.g., Quectel via QLog, Amarisoft UE logs).
By default, no logs will be collected as of this MR, as many files are big.
Various cleanup in Jenkinsfiles, some bugfixes, add unit tests.
Sometimes the connection fails because of the reboot. Without may_fail,
it would stop the build, which will then run the wrong code, possibly.
Instead, continue; if there is a problem with USRP, it will be visible
when starting nr-softmodem.
It is not necessary (the final undeploys at the end are "always_exec"),
and makes the CI potentially fail since there is nothing to copy at this
point.
Robert Schmidt