CI: Add unified Jenkinsfile for all test pipelines
This MR introduces a single Jenkinsfile to cover all current CI test pipelines
Improvements:
- correct reporting of the start timestamp in the HTML report
- harmonize log collection - always collect as test_logs_${env.BUILD_ID}.zip
PRS bug fix and CI Integration
- Fix the bug in PRS channel estimation while copying memory to compute the
impulse response
- Integrate PRS testing to CI
This commit inputs several distances at once and obtain the summary of
successful and failed tests. We look for the number of successful tests
that are geater than 0. We do that here since we sometimes dont receive
a response for a telnet command from the rfsim although the command
is applied. This is related to the queing telnet commands in the rfsim.
The script for the distances 50, 100 and 150 are called as follows,
./set-and-verify-distance-prs.sh 50 100 150
The summary can be viewed as follows,
==================== SUMMARY ====================
Total tests run : 3
Successful tests: 3
Failed tests : 0
=================================================
Set RFsim (DL) distance and use PRS to estimate the new distance. This
is verified by comparing the distance set in RFsim with the distance
reported through ToA estimation at the UE.
Note that since RFsim telnet output is asynchronous (it uses a queue
internally), we use grep --max-count 1 to wait for the matching line,
together with ncat --idle 0.3 to keep ncat open for some time. The
--max-count option will terminate grep after the first occurrence. For
this to work reliably, we need to use grep <(echo | ncat) as opposod to
simply echo | ncat | grep, as the latter does not seem to reliably
make grep exit after the occurrence.
Co-authored-by: Robert Schmidt <robert.schmidt@openairinterface.org>
Handle SIGINT and mark any subsequent steps as failed. Receiving SIGINT
a second time will abort the script immediately (as is the case before
this commit).
Note that Python seems to relay the signal to the "controlled" process
(e.g., Ping), which would return immediately with an error. So some
steps like Ping will abort immediately, others (e.g., IdleSleep) will
finish their step, and then the present logic will mark subsequent steps
as failed.
Some CI tasks/commands like Iperf(), Ping() get multiple nodes (via XML
parameter "nodes") to potentially run UEs on different nodes (hosts) at
the same time. However, I argue that this is not good:
- we don't actually use this -- where we specify multiple UEs, it's
always "localhost localhost..."
- it is inconsistent, as we typically have a single "node", not nodes,
and there is a possibility to harmonize (see also next commit)
- if we needed it, there would be better ways to achieve the same.
First, for hardware-based UEs, the ci_infra.yaml can specify different
UEs. For simulated UEs, it would be feasible to have multiple XML
steps run in parallel, e.g., in the XML <parallel> tag, which would
clarify that multiple UEs on different hosts run in parallel
- it reduces code.
Hence, make a single <node> for these XML steps.
Created Jenkinsfiles:
- Jenkinsfile: for stadard test pipelines
- Jenkinsfile-oc: for deployments with OC
These 2 Jenkinsfiles covers all current test pipelines.
Change registry name env var content
Currently setting REGISTRY="" in a .env file results in docker failing to find
/image-name:tag. This corrects that, so it looks for local images named
image-name:tag.
Following the change to use:
image: ${REGISTRY-oaisoftwarealliance/}${GNB_IMG:-oai-gnb}:${TAG:-develop}
the REGISTRY variable must now include the trailing slash.
Refactor DLSCH scheduler
Refactor the DLSCH post processor similarly to what has been done in
!3521 with the goal of saving an additional iteration through all UEs,
hopefully making the DLSCH scheduler faster.
It further includes changes to make working with and debugging multi-UE
operation easier:
- add some measurements in L1 (for total TX/RX time)
- a new option MACRLCs.[0].stats_max_ue to limit the number of UEs in
the periodic output (default: 8, 0=disable)
- avoid stack overflow in the MAC
- fix the writing of stats to nrMAC_stats.log and nrL1_stats.log when
the written output reduces by truncating the files
- avoid to schedule retransmission with 0 RBs.
Refactor PRACH handling at the gNB
- Remove race conditions by correctly setting mutexes
- Fix a bug when having multiple RACH occasions
- Reduce memory footprint by removing global memory, which should also
fix potential memory data races in FHI 7.2
- Additional fixes, see commit messages
a bit difficult to understand: sl_ahead=6 => we systematically miss the
rach detection in the scheduler because the scheduler runs more than 6
slots ahead the rx slot processing (here > 11 slots because 6 (DL and UL
slots) + 4 (RU_RX_SLOT_DEPTH, so UL only))
so, reducing sl_ahead works better than large sl_ahead because the idea
behind is: the scheduler of slot+sl_ahead will never create work to do
in a UL slot before slot+sl_ahead+X (as minimum k2 is X, in the 24PRB CI
test: 6)
CI: ensure clean iperf3 server startup
- Resolves issue with iperf3 test failure in RFSim5G multiue test
- Minor fix of iperf3 test descriptions in RFSim5G pipeline
Fix for scenario with no CSI report configured for CSI-RS
After changes in !3714 (merged), if we select to measure RSRP on SSB and not
CSI-RS (default option) and there is a single antenna port (so no MIMO report
either) we won't schedule any CSI-RS measurements for the UE. In that case
ideally we shouldn't set do_CSIRS = 1 in configuration file but still the
option is possible. So the fix is to not perform any measurements at UE on
that CSI-RS but not asserting.
Closes#1015
Before starting a new iperf3 server instance in CI tests, explicitly kill any existing
iperf3 processes bound to the same port. This prevents conflicts or failures caused by
leftover iperf3 servers from previous runs that did not terminate properly or on time.
The cleanup is performed using `pkill` before launching the new server.
The gNB will stop the periodical logging of UEs in MAC Info level after
a configurable number of UEs have been printed, by default 8.
Set it to 17 in some CI tests to ensure we still get all UEs for the
time being, as the CI might depend on this.
Improve handling of selection of RSRP/SINR report via configuration file
In current develop, the selection of SSB or CRI RSRP depends on the
do_CSIRS flag being activated. That's not wanted. CRI report for now it
is never handled, while we need SSB RSRP report for beam switching
regardless of CSI-RS configuration.
[CI] Update the Push to DockerHub Stage
This MR addresses two cases:
1. Error Handling: Ensures subsequent Image Test Processes stages
continue to run even if the preceding Push to DockerHub stage fails
(applies when the condition is PUSH event and doBuild is true).
2. Introduces conditional logic to correctly handle multi-architecture
images: Images built for both AMD64 and Jetson ARMv8 (oai-gnb,
oai-nr-ue, oai-nr-cuup) use docker buildx imagetools for push to
DockerHub.
All other AMD64 images are pushed using standard Docker commands.
Related MR !3691
- Push both AMD64 & ARMv8 Jetson images together for images built on both using `docker buildx imagetools`
- Use standard 'docker pull and push' for images built only on AMD64.
- Add XML pipeline for N2 handover testing
- Add Docker compose for two monolithic gNBs
- Features:
* Two monolithic gNBs (gNB1, gNB2) with proper CU+DU parameters
* Gradual attenuation control using RC4DAT attenuator (5s duration)
* N2 handover sequence: gNB1 → gNB2 → gNB1
* Proper channel mapping: ch1-2 for gNB1, ch3-4 for gNB2
* Telnet server configuration for CI commands
* Handover-specific neighbor configuration
* Clean test state management with attenuator reset
Update based on !3645
Note: For some test configurations we still need to provide
ra_ResponseWindow, for more details, check commit message of
41d14b571c
By configuring the Jenkins pythonExecutor to match the test machine, we can use
localhost as the node for running individual test cases. This change makes it
easy to move tests across machines, as the only adjustment needed is updating
the pythonExecutor in Jenkins.
By configuring the Jenkins pythonExecutor to match the test machine, we can use
localhost as the node for running individual test cases. This change makes it
easy to move tests across machines, as the only adjustment needed is updating
the pythonExecutor in Jenkins.
By configuring the Jenkins pythonExecutor to match the test machine, we can use
localhost as the node for running individual test cases. This change makes it
easy to move tests across machines, as the only adjustment needed is updating
the pythonExecutor in Jenkins.
CI: Upgrade Aerial setup to 25-2.1
The purpose of this MR is to upgrade the cuBB image in the Aerial setup to
release 25-2.1 and update the doc/Aerial_FAPI_Split_Tutorial.md accordingly.
CI: Upgrade HO setup to use an attenuator
The goal of this MR is to upgrade HO CI setup to include a digital attenuator
that will allow us to trigger handover by attenuating the signals on the DUs.
Robert Schmidt