Speed up docker deployment in CI
- Using docker compose up --wait instead of a custom healthcheck script:
from: https://docs.docker.com/reference/cli/docker/compose/up/
--wait Wait for services to be running|healthy. Implies detached mode.
--wait-timeout Maximum duration in seconds to wait for the project to be running|healthy
- Speeding up the healthchecks by reducing the interval from 1-10s to
0.5s during start period
start_period creates a special period after the start of the container
where the healthcheck is executed every start_interval . During this
time the failures do not count towards the retries counter, but
successes change the status to healthy. This allows container to reach
healthy status much sooner.
- Using init: true to speed up docker compose down/stop
`init: true` initializes the docker container with tini or
alternative: tini is used to collect all the processes and correctly
pass the termination signal to them. This allows the docker container
to be stopped without waiting 10 seconds to kill the container
- Smallest example of this is:
services:
test-init:
image: ubuntu:jammy
command: sleep 10d
This modification aims to improve health status detection by using PRACH IO as
an indicator of the gNB's status (container is marked as "healthy" when the
PRACH IO value in gNB nrL1_stats.log exceeds 0.0 dB.). The new condition accounts
for scenarios where the container may not exit or produce error codes (e.g.,
“sleep” logs or "error code overflow" messages).
- `init: true` initializes the docker container with tini or alternative - it is
used to collect all the processes and correctly pass the termination signal to
them, this allows the docker container to be stopped without waiting 10 seconds
to kill the container
- speeding up the healthchecks by reducing the interval from 1-10s to 0.5s during
start period
Co-authored-by: Bartosz Podrygajlo <bartosz.podrygajlo@openairinterface.org>
CI: Upgrade Aerial version to 25-1
- Update the Nvidia L1 (cuBB) from version 24-3 to 25-1
- Decrease the number of cores dedicated to the OAI gNB; testing showed
it does not need that many
CI: Add "may_fail" step to execute flaky steps
Add a generic may_fail CI XML step option that indicates the CI step may
fail, but the CI would still be marked as succeeded (instead of skipping
all steps and mark as failed). The cause of this failure depends on the
specific step, e.g., Iperf failing because of low throughput. This
allows to clean up some code and e.g. remove the command_fail option of
Custom_Command, which is then superfluous (and less generic). An
inconvenience is that those may_fail steps will be marked as "failed" in
the HTML. Later cleanup can correct this.
The may_fail is used for executing FHI7.2 testing to push 700 Mbps in DL
(close to the maximum).
Fix some bugs and add some clean up, see commit messages.
Enabling usage of ulsyncvalidityDuration timer in SIB19
ntn-UlSyncValidityDuration in SIB19 from Spec 38.331 v17.12 A validity
duration configured by the network for assistance information which
indicates the maximum time duration (from epochTime) during which the UE
can apply assistance information without having acquired new assistance
information. The unit of ntn-UlSyncValidityDuration is second. Value s5
corresponds to 5 s, value s10 indicate 10 s and so on.
ntn-UlSyncValidityDuration-r17 ENUMERATED{ s5, s10, s15, s20, s25, s30, s35,s40, s45, s50, s55, s60, s120, s180, s240, s900}
Implementation of epochtime is under discussion.
It can be configured in configuration file according to the above
enumeration. For GEO - 240s, MEO - 20s, LEO-5s can be used.
- This is to avoid the Docker image builder from being confused in case we have multiple nvIPC files in `/opt/nvidia-ipc`
- This occurs when we have an MR to upgrade the cuBB image
"multi-ue profile" was a specific iperf option to do iperf across
multiple UEs. However, this is not necessary anymore, the Iperf
handling multiple UEs "implicitly" (based on the number of UEs).
As of the parent commit, there is a generic "may_fail" option through
which we could still ignore an error. Thus, command_fail is superfluous,
and avoids some extra code.
Further, in all these cases, we should better be informed about any
error, so let's not use may_fail for these tests.
There might be various occasions where a CI step should be attempted to
be executed, despite a chance of failing. For instance, we might want
to try executing an instable Iperf test, and still mark the CI as
passing if it fails.
This commit introduces a generic "may_fail" option that will mark any CI
test as succeeded, despite having failed.
Ideally, we would print a warning in the HTML code, but this is not
possible because each step emits its own HTML code, instead of the main
loop handling this. We can improve this in the future.
Step 004000 appears double in the XML description. Remove one to avoid
that it is executed twice (because the Python code goes through the list
of test descriptions and executes all that match the current ID =>
double if appears twice).
[E2 agent] E2AP README update and OAI-FlexRIC CI pipeline improvements
- E2AP README - add OSC nearRT-RIC section; update with proper FlexRIC
links, and explanations on prerequisites, as well as for xApps
- OAI-FlexRIC CI - use separate containers for xApps
- add reference files for CU-UP/CU-CP
- add logging for KPM NSSAI condition
Instead of running xApps from oai-flexric container,
create dedicated xApp containers.
In addition, modify the flexric.conf: DB_PATH doesn't exist;
DB_DIR is the correct name.
Following a power cycle of the CI machines in the NSA-B200 setup, the gNB
machine encountered a networking issue. Although packets were observed leaving
and returning to the host of gNB, they failed to reach the container.
Debugging steps taken:
- upgraded the gNB machine
- reinstalled Docker
- reviewed and verified IP rules and routing settings
Setting the container to use host network mode resolved the issue.
Add CI test to force RRC IDLE and new connection setup with 5G-S-TMSI
This MR is a follow up to !2725 (merged)..
The primary goal is to introduce a CI test that enforces the following behavior:
- Forces the UE to enter RRC_IDLE state.
- Validates the UE's re-connection procedure using the 5G-S-TMSI as the UE
identity.
Add physim tests into ctest framework
Convert physim testcases listed in
cmake_targets/autotests/test_case_list.xml into ctest. Note: Tests for
dlsim_tm4 are excluded, due to compilation issues.
CTest evaluates success based on return code: '0' = pass, non-zero =
fail. For some simulators, the return code is non-zero even when the
test is successful (as indicated by success messages in the test logs).
For some simulators, the return code is always '0'. This inconsistency
has been address in the first two commits of this MR.
To build and execute these tests:
- using cmake
cd openairinterface5g
mkdir build && cd build # you can also do that in cmake_targets/ran_build/build
cmake .. -GNinja -DENABLE_PHYSIM_TESTS=ON && ninja tests
ctest
- using build_oai: physim-related ctests are activated when PHY
simulators are built (use build_oai with '-P' or '--phy_simulators'
option)
cd openairinterface5g/cmake_targets
./build_oai --ninja --phy_simulators
cd ran_build/build
ctest
Link to newly created 4g and 5g RAN-Physim-Cluster pipelines:
- RAN-Physim-Cluster-4G: https://jenkins-oai.eurecom.fr/job/RAN-Physim-Cluster-4G/
- RAN-Physim-Cluster-5G: https://jenkins-oai.eurecom.fr/job/RAN-Physim-Cluster-5G/
The cls_physim1.py is no longer needed. Function for physim deployment has
been migrated to the Cluster class. Log analysis functionality has been
moved to the Analysis class.
Robert Schmidt