CI: speed up and simplify build, remove unnecessary Python code, remove --eNB* parameter list
This cleans up the CI framework for running end-to-end tests, and
improves the integration with OAI code. Specifically
- cleanup build_oai and helpers to output all text, which is captured
better by the CI
- build some images in the Cluster-Build in parallel to increase speed
- remove all --eNB{,1,2}{IPAddress,UserName,Password} variables. Those
were indexed through "instance" and "server ID" and are replaced with
the actual node name. I consider further cleanup to introduce a
"global" XML file node name so we don't need this in each step and
instead define this once.
- individual cleanup
A previous commit changed to use archiveArtifact() to recover CI logs,
and then attempted to "guess" the L1/MAC stats file name through a
regex. However, the format "enb-[0-9]+" is not correct, as
archiveArtifact() puts the task ID ("[0-9]+") BEFORE the file name
(here, "enb"). So, make it more generic and match the final "-enb.log"
("$" means "end of string").
Fixes: 244ea8fc22 ("Use archiveArtifact() for TerminateeNB()")
Many of these tests exit with an exit code != 0. Thus, the CI
automatically logs the result. Rather, write everything including stderr
to the file for analysis.
Also, rewrite the file with ">" instead of appending through ">>". While
appending works because the CI removes files after archiving, it seems
"cleaner" to remove files.
It is superseded by the CreateWorkspace() function and task step (which
removes and recreates a workspace), and this option was not actually
used in any function except for the build of unit tests, but no XML
step actually specified this option.
After having transitioned to direct node names in the XML, these
parameters are not relevant anymore.
The best way to view this patch is with
git show <SHA> --ignore-all-space
Remove this class. It is not safe to use (no return codes, can get out
of sync), hardcodes the use of passwords, and it's API is cumbersone.
More explanations can also be found in !1867. It is superseded by
cls_cmd.py
Since 28ce0dc444 ("Main: pass in ctx"), the CI always collects logs
into cmake_targets/log/. Hence, LogCollecteNB() is never called. We can
remove it.
Further, change the cmake_targets/log/ to ci-log/, which is shorter, and
append a / on the zip path to make sure that only contents is zipped.
Reimplement these functions by
- using cls_cmd
- retain only the necessary steps from the timing-phytest function
- pass in ctx and node information, instead of server ID
- remove all superfluous variables that don't seem to be used
- make some lists (corresponding indices are "server ID") into normal
variables.
Pass node directly from XML to these functions
- BuildClusterImage()
- BuildImage()
- BuildProxy()
- BuildRunTests()
instead of reading the "server ID" and looking up from
eNB_IPAddress/eNB1_IPAddress/eNB2_IPAddress.
Improve the proxy build unit test by properly passing ctx information.
Refactor ULSCH scheduler towards more TDD pattern flexibility
Refactors ULSCH scheduler, notably to support "UL-heavy" TDD patterns,
e.g., DSUUU. Also allows to run TDD pattern2 with UL slots, which was a
limitation in the previous code.
Increase the number of aggregration level candidates. Since we have many
UEs with concurrent traffic in that pipeline, we should ensure that we
can allocate many DCIs to not waste resources (e.g., not all RBs used
because could not find DCIs for specific UEs).
Make Amarisoft UE sim a RedCap UE
Change Amarisoft UE configuration and gNB of the AW2S pipeline to
be/handle RedCap UEs.
The UE should only connect if the gNB handles RedCap UEs. In the logs,
we should see
[MAC] UE with RNTI 3725 is RedCap
which we check for the first two UEs (one normal UE/absence of this
message, one RedCap UE/presence of this message).
Since the parent commit, UE 1 is normal (no RedCap), UE 2 is RedCap.
Verify that the gNB agrees with that and detects the second UE as RedCap
by analyzing some logs.
The first check is a bit elaborate, but basically is
[ $(docker logs | grep XYZ | wc -l) -eq 0 ]
^ ^ ^ ^ ^
1 2 3 4 5
(1) is the if statement, that compares
(4) the number of occurrences
(3) of "RedCap" in the
(2) docker logs
(5) to being equal to
Note for all UEs which number, and whether it is RedCap (so this need to
be kept in sync with the Amarisoft UE sim configuration file in use!).
Change the parameters to be redcap (release: 17, redcap: "redcap") on
some UEs, and modify the path to the updated configuration.
Specifically, make UE 1 normal (no RedCap), UE 2 RedCap, UEs 3&4 normal,
UEs 5&6 RedCap, etc.
RedCap UEs should now only be able to connect if the gNB is handling Redcap
UEs, which is done in the parent commit.
As DEBIAN_FRONTEND has been set as environment variable, there is no need to use DEBIAN_FRONTEND=noninteractive inside RUN lines, it is redundant, hence it has been removed.
Libboost went from 1.74.0 to 1.83.0 alongside Ubuntu update.
GCC version is set as default to be more practical.
In Ubuntu22, apt sources were in .list files, in Ubuntu24 they are in .sources files.
Add --break-system-packages for pip installation to use same logic as before, see the pip update 23.0.1 (2023-02-17).
With Ubuntu24 liboai_device requires libpython3.12
Redirecting to a file has the problem that it is difficult in the CI to
recover these files. Furthermore, at least with ninja, when running
build_oai interactively, the build output will only take some lines
(because ninja rewrites lines), so the change should not be that big to
users.
One CI function would print the entire output from build_oai to the
console. Since that output might be long now, do not do this. The file
is archived, anyway.
CI: Refactor log collection, collect also (COTS) UE logs
Refactor log handling in CI: collect according to currently used XML files. This
makes it homogeneous in the pipelines, and avoids accidental overwriting (as
happens in phytest-timing), as well as harmonizing the log placement
(e.g., no YAML lookup in Ping/Iperf).
Allow to trace and archive UE logs (e.g., Quectel via QLog, Amarisoft UE logs).
By default, no logs will be collected as of this MR, as many files are big.
Various cleanup in Jenkinsfiles, some bugfixes, add unit tests.
Sometimes the connection fails because of the reboot. Without may_fail,
it would stop the build, which will then run the wrong code, possibly.
Instead, continue; if there is a problem with USRP, it will be visible
when starting nr-softmodem.
Robert Schmidt