Merge remote-tracking branch 'origin/softmodem-caps' into integration_2024_w22
This commit is contained in:
Robert Schmidt
@@ -163,6 +163,8 @@ docker exec -it oai-ext-dn iperf -u -t 86400 -i 1 -fk -B 192.168.70.135 -b 100M
|
||||
|
||||
# 6. Advanced configurations (optional)
|
||||
|
||||
See also the [dedicated document on performance tuning](./tuning_and_security.md).
|
||||
|
||||
## 6.1 USRP N300 and X300 Ethernet Tuning
|
||||
|
||||
Please also refer to the official [USRP Host Performance Tuning Tips and Tricks](https://kb.ettus.com/USRP_Host_Performance_Tuning_Tips_and_Tricks) tuning guide.
|
||||
|
||||
@@ -24,6 +24,7 @@
|
||||
- [clang-format.md](./clang-format.md): how to format the code
|
||||
- [sanitizers.md](./dev_tools/sanitizers.md): how to run with ASan/UBSan/MemSAN/TSan
|
||||
- [environment-variables.md](./environment-variables.md): the environment variables used by OAI
|
||||
- [tuning_and_security.md](./tuning_and_security.md): performance and security considerations
|
||||
|
||||
There is some general information in the [OpenAirInterface Gitlab Wiki](https://gitlab.eurecom.fr/oai/openairinterface5g/-/wikis/home)
|
||||
|
||||
@@ -50,6 +51,7 @@ There is some general information in the [OpenAirInterface Gitlab Wiki](https://
|
||||
- [How to use the positioning reference signal (PRS)](./RUN_NR_PRS.md)
|
||||
- [How to use device-to-device communication (D2D, 4G)](./d2d_emulator_setup.txt)
|
||||
- [How to run with E2 agent](../openair2/E2AP/README.md)
|
||||
- [How to run the physical simulators](./physical-simulators.md)
|
||||
|
||||
Legacy unmaintained files:
|
||||
- [`L2NFAPI_NOS1.md`](./L2NFAPI_NOS1.md), [`L2NFAPI_S1.md`](./L2NFAPI_S1.md):
|
||||
|
||||
@@ -169,6 +169,7 @@ information on how the images are built.
|
||||
~4G-LTE ~5G-NR
|
||||
- cluster (`Asterix-OC-oaicicd-session` resource), tests in OpenShift Cluster
|
||||
- unitary simulators (`nr_dlsim`, etc.)
|
||||
- see [`./physical-simulators.md`](./physical-simulators.md) for an overview
|
||||
- [RAN-RF-Sim-Test-4G](https://jenkins-oai.eurecom.fr/job/RAN-RF-Sim-Test-4G/)
|
||||
~4G-LTE
|
||||
- obelix (eNB, lteUE, OAI EPC)
|
||||
|
||||
@@ -0,0 +1,135 @@
|
||||
This document describes the physical simulators, also called "unitary
|
||||
simulators", or short "physims".
|
||||
|
||||
[[_TOC_]]
|
||||
|
||||
# Introduction
|
||||
|
||||
There are several unitary simulations for the physical layer, testing
|
||||
individual L1 transport/control channels using Monte-Carlo simulations, for
|
||||
instance:
|
||||
|
||||
- `nr_pucchsim` for 5G PUCCH,
|
||||
- `nr_dlsim` for 5G DLSCH/PDSCH,
|
||||
- `nr_prachsim` for 5G PRACH,
|
||||
- `dlsim` for 4G DLSCH/PDSCH, etc.
|
||||
|
||||
These simulators constitute the starting point for testing any new code in the
|
||||
PHY layer, and are required tests to be performed before committing any new
|
||||
code to the repository, in the sense that all of these should compile and
|
||||
execute correctly. They provide estimates of the transport block error rates
|
||||
and HARQ thoughput, DCI error rates, etc. The simulators reside in
|
||||
`openair1/SIMULATION/LTE_PHY` and `openair1/SIMULATION/NR_PHY`. For instance,
|
||||
`nr_dlsim` is built from `openair1/SIMULATION/NR_PHY/dlsim.c`.
|
||||
|
||||
Further unitary simulation of the coding subsystem components also exists in
|
||||
`openair1/PHY/CODING/TESTBENCH`, e.g., `ldpctest` for encoding/decoding of
|
||||
LDPC.
|
||||
|
||||
These are the simulators known to work properly and tested in CI:
|
||||
- 4G: `dlsim`, `ulsim`
|
||||
- 5G: `nr_dlsim`, `nr_ulsim`, `nr_dlschsim`, `nr_ulschsim`, `ldpctest`,
|
||||
`nr_pbchsim`, `nr_prachsim`, `nr_pucchsim`, `polartest`, `smallblocktest`,
|
||||
`nr_psbchsim` (sidelink)
|
||||
|
||||
# How to run
|
||||
|
||||
You first have to build simulators:
|
||||
|
||||
```bash
|
||||
cd openairinterface5g/
|
||||
. oaienv # important for the run_exec_autotests.bash script
|
||||
cd cmake_targets/
|
||||
./build_oai --ninja -P
|
||||
```
|
||||
|
||||
This builds the simulators in the default build directory `ran_build/build`
|
||||
(all paths given here are relative to `cmake_targets/` in the root of
|
||||
`openairinterface5g/`). To run all simulators in this default build directory
|
||||
of above `build_oai` invocation, run the `run_exec_autotests.bash` script:
|
||||
```bash
|
||||
autotests/run_exec_autotests.bash
|
||||
```
|
||||
|
||||
This runs all simulations as defined in `autotests/test_case_list.xml`.
|
||||
|
||||
To run a group of tests, you can specify the group using `-g`. You can also
|
||||
specify the build directory, after building the simulators, which, for the
|
||||
default directory, is:
|
||||
|
||||
```bash
|
||||
autotests/run_exec_autotests.bash -g "nr_dlsim.basic" -d ran_build/build/
|
||||
```
|
||||
|
||||
This will run the `nr_dlsim.basic` group, as listed in
|
||||
`autotests/test_case_list.xml`.
|
||||
|
||||
## How to interpret the output
|
||||
|
||||
From the output, you can see the invoked tests, its parameters, and the log files.
|
||||
Example:
|
||||
|
||||
```
|
||||
[user@oai cmake_targets]$ autotests/run_exec_autotests.bash -g "nr_dlsim.basic" -d ran_build/build/
|
||||
[...]
|
||||
Test case nr_dlsim.basic match found in group
|
||||
name = nr_dlsim.basic
|
||||
Description = nr_dlsim Test cases. (Test1: 106 PRB),
|
||||
(Test2: 217 PRB),
|
||||
(Test3: 273 PRB),
|
||||
(Test4: HARQ test 25% TP 4 rounds),
|
||||
(Test5: HARQ test 33% TP 3 rounds),
|
||||
(Test6: HARQ test 50% TP 2 rounds),
|
||||
(Test7: 25 PRBs, 15 kHz SCS)
|
||||
main_exec = ran_build/build//nr_dlsim
|
||||
main_exec_args = -n100 -R106 -b106 -s5
|
||||
-n100 -R217 -b217 -s5
|
||||
-n100 -R273 -b273 -s5
|
||||
-n100 -s1 -S2 -t25
|
||||
-n100 -s1 -S2 -t33
|
||||
-n100 -s5 -S7 -t50
|
||||
-n100 -m0 -e0 -R25 -b25 -i 2 1 0
|
||||
search_expr_true = PDSCH test OK
|
||||
search_expr_false = segmentation fault|assertion|exiting|fatal
|
||||
tags = test1 test2 test3 test4 test5 test6 test7
|
||||
nruns = 3
|
||||
i = PDSCH test OK
|
||||
Test1: 106 PRB
|
||||
Executing test case nr_dlsim.basic.test1, Run Index = 1, Execution Log file = /home/oai/openairinterface5g/cmake_targets/autotests/log/nr_dlsim.basic/test.nr_dlsim.basic.test1.run_1
|
||||
Executing test case nr_dlsim.basic.test1, Run Index = 2, Execution Log file = /home/oai/openairinterface5g/cmake_targets/autotests/log/nr_dlsim.basic/test.nr_dlsim.basic.test1.run_2
|
||||
Executing test case nr_dlsim.basic.test1, Run Index = 3, Execution Log file = /home/oai/openairinterface5g/cmake_targets/autotests/log/nr_dlsim.basic/test.nr_dlsim.basic.test1.run_3
|
||||
[...]
|
||||
nr_dlsim.basic.test1 RUN = 1 Result = PASS
|
||||
nr_dlsim.basic.test1 RUN = 2 Result = PASS
|
||||
nr_dlsim.basic.test1 RUN = 3 Result = PASS
|
||||
execution nr_dlsim.basic.test1 {Test1: 106 PRB} Run_Result = " Run_1 =PASS Run_2 =PASS Run_3 =PASS" Result = PASS
|
||||
```
|
||||
|
||||
You see:
|
||||
- the `Description` of the test cases (here, 7 tests)
|
||||
- the executable that is used, and the parameters (`main_exec` and
|
||||
`main_exec_args`), for instance `-n100 -R106 -b106 -s5` for `Test1`
|
||||
- what is the condition for a successful test
|
||||
- where are the log files
|
||||
- if the tests passed or not `PASS`, otherwise `FAIL` and a summary for the
|
||||
three runs.
|
||||
|
||||
## How to change the parameters
|
||||
|
||||
You can modify the parameters in `autotests/test_case_list.xml`. To learn about
|
||||
their meaning, run the simulator manually with flag `-h` ("help"). For
|
||||
instance, for the above run, it would be
|
||||
|
||||
```bash
|
||||
ran_build/build/nr_dlsim -h
|
||||
```
|
||||
|
||||
## How to change the code
|
||||
|
||||
You can modify the source files to adapt to your needs. For instance, for
|
||||
`nr_dlsim` above, refer to file `openair1/SIMULATION/NR_PHY/dlsim.c`.
|
||||
|
||||
# CI
|
||||
|
||||
These tests are run in this pipeline:
|
||||
[RAN-PhySim-Cluster](https://jenkins-oai.eurecom.fr/job/RAN-PhySim-Cluster/).
|
||||
@@ -0,0 +1,165 @@
|
||||
This document outlines some information specific to system tuning for
|
||||
running all executables (`nr-softmodem`, `lte-softmodem`, `nr-uesoftmodem`,
|
||||
`lte-uesoftmodem`, `nr-cuup`, called softmodems for short in the following).
|
||||
Also, it explains which Linux capabilities are required to run, and how to run
|
||||
without sudo.
|
||||
|
||||
[TOC]
|
||||
|
||||
# Performance Tuning
|
||||
|
||||
Please also refer to the [advanced configuration in the
|
||||
tutorial](NR_SA_Tutorial_COTS_UE.md#6-advanced-configurations-optional), which
|
||||
groups many tips and tricks.
|
||||
|
||||
## CPU
|
||||
|
||||
OAI used to try to set the minimum CPU-DMA latency to 2 us by writing to
|
||||
`/dev/cpu_dma_latency`. However, it is unclear if this has a significant
|
||||
effect. Further, in containerized workloads, it might not be possible to set
|
||||
this at all. We assume the user of OAI sets this value before starting OAI.
|
||||
See the [Linux kernel
|
||||
documentation](https://www.kernel.org/doc/html/latest/admin-guide/pm/cpuidle.html)
|
||||
for more information.
|
||||
|
||||
OAI used to try to set the minimum frequency of cores running OAI to the
|
||||
maximum frequency. We assume that the user sets CPU frequency policies
|
||||
accordingly; in fact, tutorials generally suggest to either set a performance
|
||||
CPU governor, or disable any sleep states at all. Hence, setting these
|
||||
low-level parameters seems useless, and we assume the user of OAI disables
|
||||
sleep states before starting OAI.
|
||||
|
||||
To disable all sleep states, simply run the following:
|
||||
```
|
||||
sudo cpupower idle-set -D0
|
||||
```
|
||||
Sometimes, dependencies might not be installed, in which case you should
|
||||
install what `cpupower` asks you to install. Note that this is not persistent.
|
||||
|
||||
See the [Linux kernel
|
||||
documentation](https://www.kernel.org/doc/html/latest/admin-guide/pm/cpufreq.html)
|
||||
for more information.
|
||||
|
||||
You can disable hyper-threading in the BIOS.
|
||||
|
||||
You can disable KPTI Protections for Spectre/Meltdown for more performance.
|
||||
**This is a security risk.** Add `mitigations=off nosmt` in your grub config and
|
||||
update grub.
|
||||
|
||||
## Ethernet-based Radios
|
||||
|
||||
For ethernet-based radios, such as AW2S, some USRPs, and 7.2 radios, increase
|
||||
the ringbuffers to a maximum. Read on interface `<fronthaul-interface-name>`
|
||||
using option `-g`, then set it with `-G`:
|
||||
|
||||
```
|
||||
ethtool -g <fronthaul-interface-name>
|
||||
ethtool -G <fronthaul-interface-name> rx <maximum-rx-value> tx <maixmum-tx-value>
|
||||
```
|
||||
|
||||
Also, you can increase the kernel's default and maximum read and write socket
|
||||
buffer sizes to a high values, e.g., 134217728:
|
||||
|
||||
```
|
||||
sudo sysctl -n -e -q -w net.core.rmem_default=134217728
|
||||
sudo sysctl -n -e -q -w net.core.rmem_max=134217728
|
||||
sudo sysctl -n -e -q -w net.core.wmem_default=134217728
|
||||
sudo sysctl -n -e -q -w net.core.wmem_max=134217728
|
||||
```
|
||||
|
||||
# Capabilities
|
||||
|
||||
Historically, all softmodems are executed as `root`, typically using `sudo`.
|
||||
This remains a possibility, but we do not recommend it due to security
|
||||
considerations. Rather, consider giving specific capabilities as outlined
|
||||
below. Read `capabilities(7)` (`man 7 capabilities`) for more information on
|
||||
each of the below capabilities.
|
||||
|
||||
Note that we tested this using 5G executables; 4G should work, but have not
|
||||
been tested as extensively. If in doubt, run eNB/lteUE using `sudo`. The below
|
||||
comments on capabilities apply in general as well; however, 4G executable might
|
||||
not warn about missing capabilities or just fail.
|
||||
|
||||
Refer to any of the docker-compose files under `ci-scripts/` to see how to give
|
||||
capabilities in docker. If you run from source, you can use `setcap` to mark a
|
||||
process to run with specific capabilities. For instance, you can add the "general
|
||||
capabilities" as described further below on the files like this:
|
||||
|
||||
```
|
||||
sudo setcap cap_sys_nice+ep ./nr-softmodem
|
||||
sudo setcap cap_ipc_lock+ep ./nr-softmodem
|
||||
sudo setcap cap_sys_nice+ep ./nr-uesoftmodem
|
||||
sudo setcap cap_ipc_lock+ep ./nr-uesoftmodem
|
||||
sudo setcap cap_net_admin+ep ./nr-uesoftmodem
|
||||
```
|
||||
|
||||
To make only temporary changes to capabilities, use `capsh`. It needs to be
|
||||
started with all capabilities, so needs `sudo`. To drop all capabilities,
|
||||
issue:
|
||||
```
|
||||
sudo -E capsh --drop="cap_sys_nice,cap_chown,cap_dac_read_search,cap_fowner,cap_fsetid,\
|
||||
cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_linux_immutable,cap_net_bind_service,\
|
||||
cap_net_broadcast,cap_net_admin,cap_net_raw,cap_ipc_lock,cap_ipc_owner,cap_sys_module,\
|
||||
cap_sys_rawio,cap_sys_chroot,cap_sys_ptrace,cap_sys_pacct,cap_sys_admin,cap_sys_boot,\
|
||||
cap_sys_resource,cap_sys_time,cap_sys_tty_config,cap_mknod,cap_lease,cap_audit_write,\
|
||||
cap_audit_control,cap_setfcap,cap_mac_override,cap_mac_admin,cap_syslog,cap_wake_alarm,\
|
||||
cap_block_suspend,cap_audit_read,cap_perfmon,cap_bpf,cap_checkpoint_restore" --print -- \
|
||||
-c "/absolute/path/to/nr-softmodem -O /absolute/path/to/config.conf"
|
||||
```
|
||||
|
||||
(For readability, the command has been separated onto multiple lines through
|
||||
`\`). To run with `SYS_NICE`, remove the first capability (`cap_sys_nice`)
|
||||
from the list of dropped capabilities.
|
||||
|
||||
## General capabilities
|
||||
|
||||
- `SYS_NICE`: required by all softmodems to assign a higher priority to
|
||||
threads to increase the likelihood that the Linux scheduler schedules a
|
||||
thread for CPU time. Also, in some configurations, CPU pinning is employed to
|
||||
ensure consistent performance, also known as setting a CPU affinity.
|
||||
|
||||
This capability is necessary by default when running any softmodem, for
|
||||
setting real-time requirements on processing threads. To allow to run without
|
||||
these requirements, the softmodems check if `SYS_NICE` is available, and
|
||||
skips any thread priority and affinity settings if the capability is not
|
||||
available. This allows to run any softmodem without root privileges in RFsim;
|
||||
you can see this by either observing a corresponding warning at the beginning
|
||||
of the execution, or the fact that no affinity/default priority is set for
|
||||
new threads.
|
||||
|
||||
- `IPC_LOCK`: OAI tries to lock all its virtual address space into RAM,
|
||||
preventing that memory from being paged to the swap area. If this capability
|
||||
is not present, a warning is printed.
|
||||
|
||||
- `NET_ADMIN`: Required at the UE to open the interface `oaitun_ue1` or eNB/gNB
|
||||
in noS1 mode. 5G executables will throw an error if they cannot create or
|
||||
modify an interface, but will continue running. It is therefore possible to
|
||||
run without this capability, but you cannot inject any traffic into the
|
||||
system. 4G executables might need this requirement, and possibly fail.
|
||||
|
||||
## Capabilities with DPDK
|
||||
|
||||
Additionally to the "general capabilities" above, you need these capabilities
|
||||
when running with 7.2 fronthaul, which uses the xran library with a dependency
|
||||
on DPDK:
|
||||
|
||||
- `IPC_LOCK` (becomes mandatory with DPDK)
|
||||
- `SYS_RESOURCE`
|
||||
- `NET_RAW`
|
||||
|
||||
## Capabilities with UHD
|
||||
|
||||
You don't need any additional capabilities for UHD beyond the "general
|
||||
capabilities" for performance outlined above. Make sure that the USB device(s)
|
||||
are readable and writable by the current user (e.g., `uhd_usrp_probe` can be
|
||||
executed by a non-root user).
|
||||
|
||||
## Capabilities with AW2S
|
||||
|
||||
You don't need any additional capabilities for AW2S beyond the "general
|
||||
capabilities" for performance outlined above.
|
||||
|
||||
## Other radios
|
||||
|
||||
Other radios have not been tested. If they do not work without additional
|
||||
capabilities beyond the "general capabilities", please file a bug report.
|
||||
Reference in New Issue
Block a user