Remove the runtime, which is not interesting, and add instead
- test name and if the test failed
- the threshold checks, if any, and
- possible additional information about test failures.
Prior to this change, each test would have, as label, it's description.
This leads to many labels, which are normally reserved for grouping
tests. Move the description into a custom property instead.
Since we rely on the JSON output for the generation of HTML reports, we
need to also extract the description from the generated JSON. However,
custom properties are only logged starting from cmake 3.30. For
back-wards compatibility, write an additional, separate environment
variable TEST_DESCRIPTION from which the description can be read as a
fallback.
For the add_timed_physim_test(), this has been already done in a parent
commit.
A future commit will enable to deploy physical simulators via docker.
Hence:
- Move the script to run out of deploy_oc_physim(), to make it more
generic (later commits will introduce other scripts)
- pass the working directory (the function will be moved into
cls_oaicitest, as it fits neither cls_containerize nor cls_cluster
when deploying the tests from source)
- not using ci-scripts: don't hardcode a specific path, just use the
working directory
- avoid helm-specific pods summary, and put it into the general "physim
logs"
- remove helm-specific "release", and use a generic name for results
- always log pod logs and status to debug, even if deployment was not
successful or tests did not finish.
Instead of making the function a class member, make it an instance
member because (1) it is more aligned with other functions, notably,
run(), and (2) especially for RemoteCmd() it can be faster, as
successive exec_script() calls do not require a new SSH connection.
RFSim test for 10 UEs/process
This MR adds a testcase and changes required to run 10 UEs in a single
process with RFSim
- Added deregistration for all connected UEs
- Refactored MAC/RRC instance handling
- made map_current_symbol thread safe
- made opeanir0_cfg thread safe and local
- Added 10UE/process testcase
Update RFsim CN
- Update RFsim CNs to latest version
- update the MBIM stop script to remove IP addresses (we don't use this
script directly, so it does not have an impact)
The CI detects whether a UE connected (and has a PDU session) by trying
to read the IP address. It therefore assumes that as long as the UE is
not connected, no IP would be present.
With the existing stop_quectel_mbim.sh script, this was not the case; it
would simply leave the IP address. Modify this script to flush all IP
addresses on disconnect to remedy this.
RF emulator and improvements of the timers
RF emulator:
Add an RF library emulating a real device without the need of a real
device. This is thought to enable using phy-test without requiring a
radio device or channel simulation. The library synchronizes on the
real time clock, drops TX and generate noise on RX. Noise generation
can be disabled and the noise level configured.
Timers improvements:
- Fix standard deviation for merged timers
- Add display of ULSCH decoding timers
- Add decoding timer in AAL
CI: speed up and simplify build, remove unnecessary Python code, remove --eNB* parameter list
This cleans up the CI framework for running end-to-end tests, and
improves the integration with OAI code. Specifically
- cleanup build_oai and helpers to output all text, which is captured
better by the CI
- build some images in the Cluster-Build in parallel to increase speed
- remove all --eNB{,1,2}{IPAddress,UserName,Password} variables. Those
were indexed through "instance" and "server ID" and are replaced with
the actual node name. I consider further cleanup to introduce a
"global" XML file node name so we don't need this in each step and
instead define this once.
- individual cleanup
A previous commit changed to use archiveArtifact() to recover CI logs,
and then attempted to "guess" the L1/MAC stats file name through a
regex. However, the format "enb-[0-9]+" is not correct, as
archiveArtifact() puts the task ID ("[0-9]+") BEFORE the file name
(here, "enb"). So, make it more generic and match the final "-enb.log"
("$" means "end of string").
Fixes: 244ea8fc22 ("Use archiveArtifact() for TerminateeNB()")
Many of these tests exit with an exit code != 0. Thus, the CI
automatically logs the result. Rather, write everything including stderr
to the file for analysis.
Also, rewrite the file with ">" instead of appending through ">>". While
appending works because the CI removes files after archiving, it seems
"cleaner" to remove files.
It is superseded by the CreateWorkspace() function and task step (which
removes and recreates a workspace), and this option was not actually
used in any function except for the build of unit tests, but no XML
step actually specified this option.
After having transitioned to direct node names in the XML, these
parameters are not relevant anymore.
The best way to view this patch is with
git show <SHA> --ignore-all-space
Remove this class. It is not safe to use (no return codes, can get out
of sync), hardcodes the use of passwords, and it's API is cumbersone.
More explanations can also be found in !1867. It is superseded by
cls_cmd.py
Since 28ce0dc444 ("Main: pass in ctx"), the CI always collects logs
into cmake_targets/log/. Hence, LogCollecteNB() is never called. We can
remove it.
Further, change the cmake_targets/log/ to ci-log/, which is shorter, and
append a / on the zip path to make sure that only contents is zipped.
Reimplement these functions by
- using cls_cmd
- retain only the necessary steps from the timing-phytest function
- pass in ctx and node information, instead of server ID
- remove all superfluous variables that don't seem to be used
- make some lists (corresponding indices are "server ID") into normal
variables.
Pass node directly from XML to these functions
- BuildClusterImage()
- BuildImage()
- BuildProxy()
- BuildRunTests()
instead of reading the "server ID" and looking up from
eNB_IPAddress/eNB1_IPAddress/eNB2_IPAddress.
Improve the proxy build unit test by properly passing ctx information.
Robert Schmidt