Commit Graph
27233 Commits
Author SHA1 Message Date
Ghislain Bourgeois 53d31eb26e Use getaddrinfo() when connecting to rfsimulator
This commits uses getaddrinfo(3) when connecting as client to the
rfsimulator. This enables name lookups, through DNS or a hosts file.

This change makes it easier to use the simulator in a dynamic
environment like Docker or Kubernetes by only targeting the name of
the container or service.
2024-10-02 09:34:24 +02:00
Robert Schmidt fc0cbc3147 Create_Workspace: exit 0 at end
the create_workspace.sh script checks if a branch should be merged into
what we are trying to checkount (i.e., typically develop into
feature-branch). The problem is that it uses a short-hand for an if
branch.

    [ -n "${merge}"] && git merge...

If we don't request a merge, [ -n XX ] will evalute to 1, and not do the
git merge. Since it's the last command in the script, this makes the
script return 1 (failure), when it actually succeeded.

Add exit 0 to ensure we return 0 after successful end of the script. If
there is a failure, the script will terminate early. It can be tested by
e.g., giving a wrong commit ID.

Reviewed-by: Jaroslava Fiedlerova <Jaroslava.Fiedlerova@openairinterface.org>
2024-10-01 09:19:43 +02:00
Robert Schmidt 54f927b2cb Merge branch 'integration_2024_w39' into 'develop'
Integration: `2024.w39`

Closes #846

See merge request oai/openairinterface5g!2990

* !2718 Added Jenkins job for Colosseum testing integration
* !2901 Fix log2_approx
* !2983 fix(docker): making tini download platform specific to support ARM64 native builds
* !2919 Add E2 SM in the target dockerfiles
* !2981 NR gNB fix PMI report evaluation
* !2979 NR-UE: Merging different UE NR SLOT FEP procedures into one.
* !2899 SIB19 initialization and scheduling on gNB
* !2953 Remove unnecessary c includes
* !2926 Fix SRS FAPI at gNB
* !2987 fix for variable power in PDSCH REs in case of 25 PRBs Bandwidth
* !2993 remove DRX from UE capabilities: not supported yet
* !2711 NR UE UL time alignment handling improvements
* !2995 some small fixes for 5G NR
* !2992 cleaner faster nr_pdcch_demapping_deinterleaving() inner loop
* !2986 NR PDCP ticks for F1 split
* !2982 CI framework cleanup
2024-10-01 06:24:27 +00:00
Jaroslava Fiedlerova 66b34d1ef1 Merge remote-tracking branch 'origin/ci-framework-cleanup' into integration_2024_w39 2024-09-30 22:21:59 +02:00
Robert Schmidt 0e23b795b7 CI: use Custom_Script to execute SR-IOV/setup scripts for FHI72 pipeline 2024-09-30 16:08:58 +02:00
Robert Schmidt 5ee9e20f7d SAbox: consistently pull/clean test images in each step 2024-09-30 15:31:41 +02:00
Robert Schmidt d3b4a6b788 Fix: clear HTML global's internal state 2024-09-30 15:25:10 +02:00
Jaroslava Fiedlerova 6984c05aad Merge remote-tracking branch 'origin/fix_nr_pdcp_tick' into integration_2024_w39 2024-09-30 13:14:06 +02:00
Robert Schmidt 17c148b7a5 Merge remote-tracking branch 'origin/nr_pdcch_demapping_deinterleaving-improve' into integration_2024_w39 2024-09-29 22:10:38 +02:00
Robert Schmidt 51bb3d25cf Merge remote-tracking branch 'origin/small_fixes_for_5G_NR' into integration_2024_w39 2024-09-29 22:09:26 +02:00
Xin Zhe Khooi 163d058d4a introduce dedicated nr pdcp tick thread for F1 split
Currently, NR PDCP ticks are active only with monolithic gNB, triggered by the MAC scheduler in gNB_scheduler.c.
In F1 split mode, as there is no MAC, there’s no source to advance the NR PDCP timer via "nr_pdcp_wakeup_timer_thread()".
This commit introduces a dedicated thread for NR PDCP ticks in gNB-CU and CU-UP mode.
Without it, packet losses at the F1 uplink would cause indefinite buffering, as t-Reordering won't expire due to the lack of NR PDCP timer ticks.
2024-09-29 15:10:54 +00:00
Robert Schmidt 2b67b0e1de CI PullImage(): HTML: indicate pulled images 2024-09-28 12:58:22 +02:00
Robert Schmidt 904534f0da Reboot USRP: use infinitive to harmonize with other steps 2024-09-28 12:58:22 +02:00
Robert Schmidt 60b9649e00 Implement CreateWorkspace() through script, print dir in HTML
Use a script, which can be tested independently of CI code (by running
it on the host on which it will be used), and print the directory that
has been prepared in the HTML.  Also, in the description, use the
infinitive/imperative(?) of the verb to harmonize with other steps.
2024-09-28 12:58:06 +02:00
Robert Schmidt e541aee49e Implement runScript in cls_cmd for LocalCmd/RemoteCmd, add test
A lot of CI code is python mixed with bash, e.g.,

    ssh = getConnection(host)
    ssh.run('ls')
    ssh.run('echo')

At least some of this CI code would benefit if it was written in a
simple bash script, returning error codes and potentially other
information either through stdout/stderr or files, to the calling Python
code:

    ssh = runScript(host, script)
    # script does: ls; echo

This commit introduces the possibility to run entire scripts. The idea
is that the executor has a script (on localhost), which is either
executed locally or on a remote host. For the remote host, the script is
not copied but piped into a remotely executed bash. In both cases,
output is either returned like the Cmd.run() function with returncode
and mixed stdout/stderr, or optionally redirected into a file on the
(remote) host, which can be treated further by the Python code in later
steps.
2024-09-28 12:52:59 +02:00
Robert Schmidt fc0bc2357c Modify all YAML files to work with .env files from DeployObject() 2024-09-28 12:46:04 +02:00
Robert Schmidt 51bf5c0fc9 Introduce CI deployment tests based on OAI images 2024-09-28 12:46:04 +02:00
Robert Schmidt 5201a5c37a Reimplement DeployObject()/UndeployObject()
This reimplements the CI deployment functions, and makes them a bit more
robust.  Concretely:

- Introduce a generic "deployment tag" option that can be used to
  override the tag to use for specific images. By default, it is the
  current "branch-commitID[0:8]" tag name (requires the change from
  function ImageTagToUse() to CreateTag(), impacting also pull/push
  image functions)
- Avoid sed for image names, and use an .env file that docker-compose
  picks up automatically; the deployment analyzes a potentially existing
  .env file and updates instead of simply overriding. For instance, some
  pipelines might use -asan images for the gNB and "normal" (non-asan)
  images for UE, and a simple overwriting might make the -asan image
  name tag disappear for the gNB, resulting in deployment failures).
  Finally, undeployment removes the .env file, so that there are no
  modifications in the repository after undeployment.
- Redo the previous behavior of autodetecting asan, and use that (the
  current function always uses asan, no matter what)
- Remove deployKind/displayedNewTags globals, as they are not necessary
- Make the usedImage output in HTML slimmer
- On undeployment, print service names next to undeploy analysis, and
  return success/failure.
- Make the functions generally less verbose and easier to read

Note that as of and only in this commit, deployment does not work, as
all the YAML files have not been updated to work with this updated
version.  The next commit adds tests for the new deployment, and updates
one YAML file (also being used in the tests). The follow-up commit then
modifies all YAML files.
2024-09-28 12:46:04 +02:00
Robert Schmidt d81abdaf7b Reimplement Native Build (CI step Build_eNB)
The BuildeNB steps either does things that are already implemented
(created workspace), or things that we don't use (separate build_wait),
and is generally quite convoluted when all it does is calling build_oai.
Reimplement by reusing Create_Workspace, and make build_oai in a single
step. To clean up, make a new module in which this function does not
need a class instantiation. Instead, we give all necessary parameters
directly.
2024-09-28 12:45:34 +02:00
Robert Schmidt 3feaa93ba0 cls_cmd: print the host on which a command is executed 2024-09-28 12:45:34 +02:00
Robert Schmidt 0d63861bf1 cls_cmd.py: local copy: don't copy if src/tgt identical 2024-09-28 12:45:34 +02:00
Robert Schmidt f21b5d2ef4 CI: Add basic unittest for cls_cmd 2024-09-28 12:45:33 +02:00
Robert Schmidt 606e2c749a cls_cmd: execute commands through bash
The next commit will use "echo -e". The problem with that is that bash
and sh differ:

- in bash: this results in "a\nb" (a+newline+b)
- in sh: this results in -e a\nb (-e a+newline+b)

The problem is that by default, commands are executed through sh, which
nobody expects. Change to bash, which is likely more aligned with what
people want to use.
2024-09-28 12:45:33 +02:00
Robert Schmidt 5d5ade1697 Remove superfluous cleanup/terminate/teardown XMLs
After the previous commit, the CI will always do an undeployment step
even if the CI failed. Hence, these XMLs are not necessary anymore. Some
XMLs are updated to do what their "cleanup XML" counterparts did, such
as removing additional images.
2024-09-28 12:45:07 +02:00
Robert Schmidt 8bdc399b27 Remove AutoTerminateUEeNB() functions, use auto_exec to make CI always undeploy 2024-09-28 12:45:07 +02:00
Robert Schmidt 041b2b2d7a Remove prematureExit/exitStatus, and implement in task main loop
Use the previous commit on returning success from each task execution
function, and implement handling of errors in the main loop. Add a
script with which this is visible using Custom_Commands.
2024-09-28 12:45:07 +02:00
Robert Schmidt 7b89548867 Consistently return success true/false from task steps 2024-09-28 12:45:07 +02:00
Robert Schmidt e969ddb897 IdleSleep(): no need for class members 2024-09-28 12:45:07 +02:00
Robert Schmidt 71807db4ab Custom_Command(): no need for class members 2024-09-28 12:45:07 +02:00
Robert Schmidt 61ea76e7af Remove CI repeatCount XML property
The repeatCount property is useless: it would be enough to specify the
same XML multiple times to achieve the same thing.
2024-09-28 12:45:07 +02:00
Robert Schmidt 77a8f5fa99 Basic test runner script 2024-09-28 12:45:07 +02:00
Robert Schmidt d108648312 Handle parameter reading and test case execution together
For reasons I don't understand, the XML option reading and actual test
case execution are at different places in the code. This is problematic,
because what this led to is that options are not passed as function
arguments, but indirectly through class members. This led to a lot of
"indirect code"; it's a total mess

To improve this, move them together, so that parameters can be passed
directly as function arguments later on.

Because we reference the globals, mark them inside
ExecuteActionWithParam(). That is a Python antipattern, but necessary at
this moment. I believe that we can remove the following globals over
time:

- SSH (with new cls_cmd, which we don't need to pass)
- EPC (replaced through a class similar to ue_module, i.e., refer to
  core networks on specific hosts through an ID, instead of a global)
- HTML (should use templating, where individual functions give back a
  dictionary of what they want to print, and a final pass assembles the
  entire HTML file)
- RAN/SCA/PHYSIM/CLUSTER/ldpc (can be replaced with stateless scripts
  build/run eNB/gNB on a specific node)
2024-09-28 12:45:07 +02:00
Jaroslava Fiedlerova e9f9718f63 CI: add more unit tests for UDP iperf3 analysis function 2024-09-28 12:45:07 +02:00
Jaroslava Fiedlerova 5fd5ed43c7 CI: Adjust regex patterns for Iperf3 UDP log parsing
Update the regex for both sender and receiver data to allow optional
bitrate unit prefixes - accepts also no prefix. Modify the patterns for
lost and total packets to accept negative values (`-?\d+`).
2024-09-28 12:45:07 +02:00
Jaroslava Fiedlerova 9d92c0e477 CI: add unit test for UDP iperf3 analysis function 2024-09-28 12:45:07 +02:00
Jaroslava Fiedlerova 1ba5677056 CI: get UDP iperf3 results even if packet count is negative 2024-09-28 12:45:07 +02:00
Robert Schmidt 0da2f27fa7 CI Ping/Iperf: add tests, print without lock, reduce code lines
Add some basic unit tests to verify that Iperf/Ping work. Harmonize the
logging by only print in the "main" iperf/ping function, to avoid that
if we test with multiple UEs, the output intermixes. Also, do one common
mkdir before starting multiple pings/iperfs. Finally, harmonize the
paths so that Iperf and Ping log their files in the same place.
2024-09-28 12:45:07 +02:00
Robert Schmidt 02d5f953a8 Add simple unit test cases for Deployment, including instructions 2024-09-27 19:59:54 +02:00
Robert Schmidt 25f7f61d73 Simplify GetContainerHealth(): we never use unhealthy count 2024-09-27 19:59:54 +02:00
Robert Schmidt 4bf7c45be1 SSHConnection class: don't exit, raise Exception to have main loop handle 2024-09-27 19:59:54 +02:00
Robert Schmidt 1620994f0a DeployObject: EPC not needed 2024-09-27 19:59:54 +02:00
Robert Schmidt 7c1920aea9 Fixup: no HTML in create_workspace, so can't print to HTML 2024-09-27 19:59:54 +02:00
Robert Schmidt 6f868b038f Remove more unused CI files 2024-09-27 19:59:54 +02:00
Robert Schmidt cded721189 Remove unused CI XML and Python code 2024-09-27 19:50:41 +02:00
Jaroslava Fiedlerova 6d09b2179e Merge remote-tracking branch 'origin/NR_UE_UL_time_alignment' into integration_2024_w39 2024-09-27 17:40:15 +02:00
Jaroslava Fiedlerova ed4af19590 Merge remote-tracking branch 'origin/remove-DRX-UE-capabilities' into integration_2024_w39 2024-09-27 17:33:20 +02:00
Jaroslava Fiedlerova 16d8281418 Merge remote-tracking branch 'origin/NR_GNB_PDSCH_FIX' into integration_2024_w39 2024-09-27 17:13:18 +02:00
Jaroslava Fiedlerova 36ade4a153 Merge remote-tracking branch 'origin/fix_fapi_SRS_gNB' into integration_2024_w39 2024-09-27 17:12:52 +02:00
Jaroslava Fiedlerova 2ba7d59dfe Merge remote-tracking branch 'origin/remove-includes' into integration_2024_w39 2024-09-27 17:12:39 +02:00
Thomas Schlichter 60c2b3fe00 nrscope: fix label on gNB scope - it is NR not LTE ;-) 2024-09-27 17:07:20 +02:00