maki/openairinterface5g
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

T2 offload - documentation update

Browse Source
This commit is contained in:
Jaroslava Fiedlerova
2023-11-29 09:12:30 +01:00
parent 7c5b945766
commit 7d7dced70e
1 changed files with 112 additions and 47 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/LDPC_T2_OFFLOAD_SETUP.md
+112 -47
View File
@@ -17,68 +17,133 @@
[[_TOC_]]
This documentation aims to provide a tutorial for AMD Xilinx T2 Telco card integration into OAI and its usage.
## Prerequisites
Offload of the channel decoding was tested with the T2 card in following setup:
**AMD Xilinx T2 card**
- bitstream image and PMD driver provided by AccelerComm
- DPDK 20.11.3 with patch from Accelercomm (also tested with DPDK 20.11.7
version)
# Requirements
- bitstream image and PMD driver for the T2 card provided by AccelerComm
- DPDK 20.11.7 with patch from Accelercomm (also tested with DPDK 20.11.3)
- tested on RHEL7.9, RHEL9.2, Ubuntu 20.04, Ubuntu 22.04
## DPDK setup
- check the presence of the card on the host
- `lspci | grep "Xilinx"`
- binding of the device with igb_uio driver
- `./dpdk-devbind.py -b igb_uio <address of the PCI of the card>`
# DPDK setup
## DPDK installation
```
# Get DPDK source code
git clone https://github.com/DPDK/dpdk-stable.git ~/dpdk-stable
cd ~/dpdk-stable
git checkout v20.11.7
git apply ~/ACL_BBDEV_DPDK20.11.3_xxx.patch
```
Replace `~/ACL_BBDEV_DPDK20.11.3_xxx.patch` by patch file provided by
Accelercomm.
```
cd ~/dpdk-stable
meson setup build
# meson setup --prefix=/opt/dpdk-t2 build for installation with non-default installation prefix
cd build
ninja
sudo ninja install
sudo ldconfig
```
## DPDK configuration
- load required kernel module
```
sudo modprobe igb_uio
sudo insmod ~/dpdk-stable/kernel/linux/igb_uio/igb_uio.ko
```
- check presence of the card and its PCI addres on the host machine
```
lspci | grep "Xilinx"
```
- bind the card with igb_uio driver
```
sudo python3 ~/dpdk-stable/usertools/dpdk-devbind.py -b igb_uio 41:00.0
```
Replace PCI address of the card *41:00.0* by address detected by *lspci | grep "Xilinx"* command
- hugepages setup (10 x 1GB hugepages)
- `./dpdk-hugepages.py -p 1G --setup 10G`
```
sudo python3 ~/dpdk-stable/usertools/dpdk-hugepages.py -p 1G --setup 10G`
```
*Note: Commands to run from dpdk/usertool folder*
*Note: device binding and hugepages setup has to be done after every reboot of
the host machine*
## Compilation
Deployment of the OAI is precisely described in the following tutorials.
- [BUILD](https://gitlab.eurecom.fr/oai/openairinterface5g/-/blob/develop/doc/BUILD.md)
- [NR_SA_CN5G_gNB_USRP_COTS_UE_Tutorial](https://gitlab.eurecom.fr/oai/openairinterface5g/-/blob/develop/doc/NR_SA_Tutorial_COTS_UE.md)
# Modifications in the OAI code
## PMD path specification
Path to the PMD for operating the card is specified in `CMakeLists.txt` file in
*LDPC OFFLOAD library* section. Modify following line based on the location of
the PMD on your system. By deafult, path to the PMD is set to `/opt/dpdk-t2/lib/x86_64-linux-gnu/`.
```
find_library(PMD_T2 NAMES rte_baseband_accl_ldpc HINTS "/opt/dpdk-t2/lib/x86_64-linux-gnu/")
```
Shared object file *libldpc_t2.so* is created during the compilation. This object is conditionally compiled. Selection of the library to compile is done using *--build-lib ldpc_t2*. Example command to build OAI with support for LDPC offload to T2 card is:
## T2 card DPDK initialization
Following lines in `openair1/PHY/CODING/nrLDPC_decoder/nrLDPC_decoder_offload.c` file has to be
modified based on your system requirements. By default, PCI address of the T2 card is set to 41:00.0 and cores 14 and 15 are assigned to the DPDK.
```
char *dpdk_dev = "41:00.0"; //PCI address of the card
char *argv_re[] = {"bbdev", "-a", dpdk_dev, "-l", "14-15", "--file-prefix=b6", "--"};
```
For the DPDK EAL initialization, device is specified by `-a` option and list
of cores to run the DPDK application on is selected by `-l` option. PCI adress of
the T2 card can be detected by `lspci | grep "Xilinx"` command.
`./build_oai -P --gNB -w USRP --build-lib "ldpc_t2" --ninja`
# OAI Build
OTA deployment is precisely described in the following tutorial:
- [NR_SA_Tutorial_COTS_UE](https://gitlab.eurecom.fr/oai/openairinterface5g/-/blob/develop/doc/NR_SA_Tutorial_COTS_UE.md)
Instead of section *3.2 Build OAI gNB* from the tutorial, run following commands:
```
# Get openairinterface5g source code
git clone https://gitlab.eurecom.fr/oai/openairinterface5g.git ~/openairinterface5g
cd ~/openairinterface5g
git checkout develop
# Install OAI dependencies
cd ~/openairinterface5g/cmake_targets
./build_oai -I
# Build OAI gNB
cd ~/openairinterface5g
source oaienv
cd cmake_targets
./build_oai -w USRP --ninja --gNB -P -C --build-lib "ldpc_t2"
```
Shared object file *libldpc_t2.so* is created during the compilation. This object is conditionally compiled. Selection of the library to compile is done using *--build-lib ldpc_t2*.
*Required poll mode driver has to be present on the host machine and required DPDK version has to be installed on the host, prior to the build of OAI*
# 5G PHY simulators
## nr_ulsim test
Offload of the channel decoding to the AMD Xilinx T2 card is in nr_ulsim specified by *-o* option. Example command for running nr_ulsim with LDPC decoding offload to the T2 card:
`sudo ./nr_ulsim -n100 -s20 -m20 -r273 -o`
Offload of the channel decoding to the T2 card is in nr_ulsim specified by *-o* option. Example command for running nr_ulsim with LDPC decoding offload to the T2 card:
```
cd ~/openairinterface5g
source oaienv
cd cmake_targets/ran_build/build
sudo ./nr_ulsim -n100 -s20 -m20 -r273 -R273 -o
```
## nr_dlsim test
Offload of the channel encoding to the AMD Xilinx T2 card is in nr_dlsim specified by *-c* option. Example command for running nr_dlsim with LDPC encoding offload to the T2 card:
```
cd ~/openairinterface5g
source oaienv
cd cmake_targets/ran_build/build
sudo ./nr_dlsim -n300 -s30 -R 106 -e 27 -c
```
`sudo ./nr_dlsim -n300 -s30 -R 106 -e 27 -c`
# OTA test
Offload of the channel encoding and decoding to the AMD Xilinx T2 card is enabled by *--ldpc-offload-enable 1* option.
## OTA and RFSIM test
Offload of the channel encoding and decoding to the AMD Xilinx T2 card is enabled by *--ldpc-offload-enable 1* option. Example command for running nr-softmodem with LDPC processing on the T2 card:
## Run OAI gNB with USRP B210
```
cd ~/openairinterface5g
source oaienv
cd cmake_targets/ran_build/build
sudo ./nr-softmodem --sa -O ../../../targets/PROJECTS/GENERIC-NR-5GC/CONF/gnb.sa.band78.fr1.106PRB.usrpb210.conf --ldpc-offload-enable 1
```
`./nr-softmodem -O config_file.conf --sa --ldpc-offload-enable 1`
# Limitations
## AMD Xilinx T2 card
- offload of the LDPC encoding implemented for MCS > 2, OAI CPU encoder is used for MCS =< 2
- functionality of the LDPC encoding and decoding offload verified in OTA SISO setup with USRP N310 and Quectel RM500Q, blocking of the card reported for MIMO setup (2 layers)
### LDPC encoding/decoding with CPU/GPU
Available LDPC implementations and loading particular libraries is well described in *openair1/PHY/CODING/DOC/LDPCImplementation.md*. Default library for LDPC processing is called `libldpc.so`. Sample command for running nr-softmodem with selected library (in this case we want to replace `libldpc.so` by `libldpc_optim.so` library):
`./nr-softmodem -O config_file.conf --sa --loader.ldpc.shlibversion _optim`
Available options:
- --loader.ldpc.shlibversion _optim8seg
- --loader.ldpc.shlibversion _optim
- --loader.ldpc.shlibversion _orig
- --loader.ldpc.shlibversion _cuda
- --loader.ldpc.shlibversion _cl
## Limitations
### AMD Xilinx T1 card
- offload of the LDPC decoding implemented only for MCS > 9, decoding of the smaller TBs on the T1 card leads to blocking of the card
- HARQ is not functional with LDPC decoding offload to the T1 card (restricted in the code to avoid blocking of the card)
- LDPC encoding offload not implemented for the T1 card
- not supported in current develop branch, please checkout to *2023.w43*
### AMD Xilinx T2 card
- offload of the LDPC encoding implemented for MCS > 2
- occasional blocking of the card in decoder function - issue is investigated
*Note: AMD Xilinx T1 Telco card is not supported anymore.*