Use a script, which can be tested independently of CI code (by running
it on the host on which it will be used), and print the directory that
has been prepared in the HTML. Also, in the description, use the
infinitive/imperative(?) of the verb to harmonize with other steps.
A lot of CI code is python mixed with bash, e.g.,
ssh = getConnection(host)
ssh.run('ls')
ssh.run('echo')
At least some of this CI code would benefit if it was written in a
simple bash script, returning error codes and potentially other
information either through stdout/stderr or files, to the calling Python
code:
ssh = runScript(host, script)
# script does: ls; echo
This commit introduces the possibility to run entire scripts. The idea
is that the executor has a script (on localhost), which is either
executed locally or on a remote host. For the remote host, the script is
not copied but piped into a remotely executed bash. In both cases,
output is either returned like the Cmd.run() function with returncode
and mixed stdout/stderr, or optionally redirected into a file on the
(remote) host, which can be treated further by the Python code in later
steps.
This reimplements the CI deployment functions, and makes them a bit more
robust. Concretely:
- Introduce a generic "deployment tag" option that can be used to
override the tag to use for specific images. By default, it is the
current "branch-commitID[0:8]" tag name (requires the change from
function ImageTagToUse() to CreateTag(), impacting also pull/push
image functions)
- Avoid sed for image names, and use an .env file that docker-compose
picks up automatically; the deployment analyzes a potentially existing
.env file and updates instead of simply overriding. For instance, some
pipelines might use -asan images for the gNB and "normal" (non-asan)
images for UE, and a simple overwriting might make the -asan image
name tag disappear for the gNB, resulting in deployment failures).
Finally, undeployment removes the .env file, so that there are no
modifications in the repository after undeployment.
- Redo the previous behavior of autodetecting asan, and use that (the
current function always uses asan, no matter what)
- Remove deployKind/displayedNewTags globals, as they are not necessary
- Make the usedImage output in HTML slimmer
- On undeployment, print service names next to undeploy analysis, and
return success/failure.
- Make the functions generally less verbose and easier to read
Note that as of and only in this commit, deployment does not work, as
all the YAML files have not been updated to work with this updated
version. The next commit adds tests for the new deployment, and updates
one YAML file (also being used in the tests). The follow-up commit then
modifies all YAML files.
The BuildeNB steps either does things that are already implemented
(created workspace), or things that we don't use (separate build_wait),
and is generally quite convoluted when all it does is calling build_oai.
Reimplement by reusing Create_Workspace, and make build_oai in a single
step. To clean up, make a new module in which this function does not
need a class instantiation. Instead, we give all necessary parameters
directly.
The next commit will use "echo -e". The problem with that is that bash
and sh differ:
- in bash: this results in "a\nb" (a+newline+b)
- in sh: this results in -e a\nb (-e a+newline+b)
The problem is that by default, commands are executed through sh, which
nobody expects. Change to bash, which is likely more aligned with what
people want to use.
After the previous commit, the CI will always do an undeployment step
even if the CI failed. Hence, these XMLs are not necessary anymore. Some
XMLs are updated to do what their "cleanup XML" counterparts did, such
as removing additional images.
Use the previous commit on returning success from each task execution
function, and implement handling of errors in the main loop. Add a
script with which this is visible using Custom_Commands.
For reasons I don't understand, the XML option reading and actual test
case execution are at different places in the code. This is problematic,
because what this led to is that options are not passed as function
arguments, but indirectly through class members. This led to a lot of
"indirect code"; it's a total mess
To improve this, move them together, so that parameters can be passed
directly as function arguments later on.
Because we reference the globals, mark them inside
ExecuteActionWithParam(). That is a Python antipattern, but necessary at
this moment. I believe that we can remove the following globals over
time:
- SSH (with new cls_cmd, which we don't need to pass)
- EPC (replaced through a class similar to ue_module, i.e., refer to
core networks on specific hosts through an ID, instead of a global)
- HTML (should use templating, where individual functions give back a
dictionary of what they want to print, and a final pass assembles the
entire HTML file)
- RAN/SCA/PHYSIM/CLUSTER/ldpc (can be replaced with stateless scripts
build/run eNB/gNB on a specific node)
Add some basic unit tests to verify that Iperf/Ping work. Harmonize the
logging by only print in the "main" iperf/ping function, to avoid that
if we test with multiple UEs, the output intermixes. Also, do one common
mkdir before starting multiple pings/iperfs. Finally, harmonize the
paths so that Iperf and Ping log their files in the same place.
See e.g., egrep(1) [https://linux.die.net/man/1/egrep]:
> two variant programs egrep and fgrep are available. egrep is the same
> as grep -E. fgrep is the same as grep -F. Direct invocation as either
> egrep or fgrep is deprecated
See also the rationale of the POSIX description of grep
(https://pubs.opengroup.org/onlinepubs/9699919799/utilities/grep.html):
> This grep has been enhanced in an upwards-compatible way to provide
> the exact functionality of the historical egrep and fgrep commands as
> well. It was the clear intention of the standard developers to
> consolidate the three greps into a single command.
Closes: #839
As of now, these seem to be the Jenkinsfiles used on Jenkins
ci-scripts/Jenkinsfile-cluster
ci-scripts/Jenkinsfile-colosseum
ci-scripts/Jenkinsfile-GitLab-Container
ci-scripts/Jenkinsfile-GitLab-COTS-UE-Test
ci-scripts/Jenkinsfile-nrue
ci-scripts/Jenkinsfile-physim-deploy
ci-scripts/Jenkinsfile-push-registry
ci-scripts/Jenkinsfile-tmp-full-ran
ci-scripts/Jenkinsfile-tmp-multi-enb
ci-scripts/Jenkinsfile-tmp-multi-enb-nsa
ci-scripts/Jenkinsfile-tmp-ran
(gather all files in a file, then cat <file> | sort | uniq)
Remove all others.
The default values set by xran for DPDK socket memory/limit is 8192. It
is changed when the variable for the memory size, given down to xran, is
non-zero (cf also previous patch). Since the last patch, if nothing is
given in OAI configuration, OAI will pass down a size of 8192. However,
a configuration value of 0 is a valid choice (it is a pre-allocation,
DPDK will allocate dynamically), so modify the default in xran to be 0
so that all sizes, including 0, can be given to DPDK.
Note that the currently used implementation of xran will always try to
allocate this memory on NUMA node 0. If you want to use another one,
this field has to be set to 0 (no pre-allocation), and DPDK will
allocate it dynamically.
Co-authored-by: Sagar Arora <sagar.arora@openairinterface.org>
xran has a mechanism to request DPDK to allocate a configurable amount
of bytes for huge pages. We set it to 0 until now; expose this variable
through configuration, and provide a meaningful default 8192 MiB.
Provide documentation.
Build on ARM fails in the previous version, while the current master
(1a09d3bc) works. So let's update to a newer version to avoid problems
on ARM, and assuming x86 works.
The F1AP UE context release procedure can happen in at least two cases:
1. core requested a release of a UE
2. CU requests DU a release of a UE without a core release (e.g.,
handover)
In case 1., the CU has to fully release the UE in the CU, then ack the
NGAP message from the core, whilst in 2., nothing of that should happen
after the DU acknowledges the F1AP UE context release cmd.
cuup_notify_reestablishment() used
&ue_p->established_drbs[drb_id]
to look up a bearer, which is wrong, because it should be drb_id - 1
as in the top of the loop. Also, it used the dedicated get_drb()
function; harmonize to use only this function.
A follow-up commit refactors the filling of information of DRBs to set
up in a separate function (to be reused for handover later). Hence, we
cannot store the DRB TEID/address in the same functions, because it is
a logically separate step, and might not always be available.
In e1_send_bearer_updates(), we send the DL (DU) TEID and address to the
CU-UP. Therefore, correct the variable names to reflect that the F1-U
information pertains to the DU.
Also, move the tunnel info update function f1u_dl_gtp_update() to a more
prominent place, and take the pointer to the struct containing the
"interesting" information.
A maximum RRC buffer size of 1024 might be very small. We should
increase it, but then the 5G RRC_BUF_SIZE does conflict with other
definitions. To make it unique, also rename to NR_RRC_BUF_SIZE.
Pass consistently the NR_timer_t object through a pointer. Since we have
to modify all occurrences of is_nr_timer_active(), we use that occasion
and rename that function to nr_timer_is_active() to make it consistent
with all other functions, starting with nr_timer.
Reorganize the sequence of checks done in reestablishment to make it
more logical:
- check conditions of request (correct C-RNTI)
- check conditions of DU (has MIB/SIB1, MTC)
- check the UE, and from which DU it comes
This will also help to extend it towards reestablishment coming from
different DUs in the case of handover.
Before this bug fix, when using F1, we get this warning:
Constraint validation failed: LCID: constraint failed (/home/richie/openairinterface5g/cmake_targets/ran_build/build/openair2/F1AP/MESSAGES/F1AP_LCID.c:30)
This is because (1) we don't actually store the LCID, then (2) we did
not put it into the F1 ASN.1 structures. Fix both, to make this warning
disappear.
To actually print the full error message, increase the error buffer
size. Also, for the future, to avoid guessing, print ASN.1 XML when we
encounter a failed constraint validation.
It can happen that SCTP communication is simply lost, e.g.
[SCTP] SCTP_ASSOC_CHANGE to SCTP_COMM_LOST
[SCTP] sctp_recvmsg (fd 104, len -1 ): Connection reset by peer:104
The previous code only handled the SHUTDOWN event, but in abnormal
situations, we should also handle the connection state change and signal
to RRC that the endpoint is dead, which is now implemented.
User of the SCTP module use the assoc_id for connection identification.
The previous code set the assoc_id to -1, so there is no way of telling
which connection broke. Use the assoc_id instead
The failure mode observed here was that the CU sent a release command
for a UE that the DU does not know. Given, that the outcome of the
release command (no UE) is equal to the current state (no UE), from
the DU point of view, we can safely ack the command.
According to the ThreadPoolExecutor documentation [1], the default
number of maximum workers min(32, os.cpu_count() + 4). The problem is
that we execute many pipelines from nano, on which os.cpu_count()
reports 4, so we can only have up to 8 concurrent workers by default.
This leads to the fact that on pipelines with many UEs (e.g.,
AW2S-AmariUE), we don't ping/iperf on all UEs at the same time.
Fix this by increasing the number of workers to 64. This should be fine,
as there is almost no real work being done in the Workers (only opening
a shell and execute programs such as ping/iperf).
[1] https://docs.python.org/3/library/concurrent.futures.html#threadpoolexecutor
At least in Eurecom, there are no RHEL 7/CentOS 7+8 servers. Existing
installations will work, but we don't want to encourage people to use
these old distributions, so remove it. Also, they reached EOL
Add CentOS stream as an alternative.
In the past, it seemed that we had to add a trailing slash (/) after the
OpenShift Registry URI. Now, at least on the CI machine matix, this
makes it fail:
INFO: oc whoami -t | docker login -u oaicicd --password-stdin default-route-openshift-image-registry.apps.oai.cs.eurecom.fr/
INFO: docker pull default-route-openshift-image-registry.apps.oai.cs.eurecom.fr/oaicicd-ran/oai-gnb:develop-46a1d2a6
WARNING: command "docker pull default-route-openshift-image-registry.apps.oai.cs.eurecom.fr/oaicicd-ran/oai-gnb:develop-46a1d2a6" returned non-zero returncode 1: output:
Error response from daemon: Head "https://default-route-openshift-image-registry.apps.oai.cs.eurecom.fr/v2/oaicicd-ran/oai-gnb/manifests/develop-46a1d2a6": unauthorized: authentication required
Remove the trailing slash makes it work, like so:
INFO: oc whoami -t | docker login -u oaicicd --password-stdin default-route-openshift-image-registry.apps.oai.cs.eurecom.fr
INFO: docker pull default-route-openshift-image-registry.apps.oai.cs.eurecom.fr/oaicicd-ran/oai-gnb:develop-46a1d2a6
The docker pull succeeds.
Before this commit, the following will trigger a complete rebuild:
- create another git name (git checkout -b test)
- run cmake again manually (cmake ..)
- run ninja again
The top-level CMakeLists.txt uses add_definitions() to add git
information to all targets; if some git information changes, this will
mark all command lines as dirty, triggering the build.
There is an easy solution: instead of passing this information on the
command line, pass it in a header. This commit uses configure_file() to
generate a file oai_version.h in the build directory with the same
information. Various sources files, e.g., main executables, use this to
output the same information as before. It is then also not necessary to
use header guards, as cmake takes care to create this file.
If cmake is re-run, it will update the header, and trigger only files
including this header will be rebuilt (which can be shown with the above
stpes). The information passed is the same as before, and updates the
information when the old method would have updated it.
Robert Schmidt