maki/openairinterface5g
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
b6016cfc496c457433bcef97d75bee6ef809c835
openairinterface5g/ci-scripts/yaml_files/5g_l2sim_tdd/README.md
T
Raphael Defosseux ced2a66f6e fix(ci): changing the Docker Hub account to push to 
Signed-off-by: Raphael Defosseux <raphael.defosseux@eurecom.fr>
2022-04-27 18:36:01 +02:00

14 KiB
Raw Blame History

OAI Full Stack 5G-NR L2 simulation with containers and a proxy

This page is only valid for an Ubuntu18 host.

This tutorial is only valid once this file is merged into the develop branch.

1. Retrieving the images on Docker-Hub

Currently the images are hosted under the team account oaisoftwarealliance. They were previously hosted under the user account rdefosseoai.

Once again you may need to log on docker-hub if your organization has reached pulling limit as anonymous.

$ docker login
Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one.
Username:
Password:

Now pull images.

$ docker pull mysql:5.7
$ docker pull oaisoftwarealliance/oai-amf:latest
$ docker pull oaisoftwarealliance/oai-nrf:latest
$ docker pull oaisoftwarealliance/oai-smf:latest
$ docker pull oaisoftwarealliance/oai-spgwu-tiny:latest

$ docker pull oaisoftwarealliance/oai-gnb:develop
$ docker pull oaisoftwarealliance/oai-nr-ue:develop
$ docker pull oaisoftwarealliance/proxy:latest

And re-tag them for tutorials' docker-compose file to work.

$ docker image tag oaisoftwarealliance/oai-amf:latest oai-amf:latest
$ docker image tag oaisoftwarealliance/oai-nrf:latest oai-nrf:latest
$ docker image tag oaisoftwarealliance/oai-smf:latest oai-smf:latest
$ docker image tag oaisoftwarealliance/oai-spgwu-tiny:latest oai-spgwu-tiny:latest

$ docker image tag oaisoftwarealliance/oai-gnb:develop oai-gnb:develop
$ docker image tag oaisoftwarealliance/oai-nr-ue:develop oai-nr-ue:develop
$ docker image tag oaisoftwarealliance/proxy:latest oai-lte-multi-ue-proxy:latest

$ docker logout

Note that the proxy image is based on the source available at https://github.com/EpiSci/oai-lte-5g-multi-ue-proxy.

At time of writing, the latest tag corresponded to 56cfdc046a5f96d5e67d42a2fc2bf6ba2fe58b41 commit.

2. Deploy containers

CAUTION: this SHALL be done in multiple steps.

Just docker-compose up -d WILL NOT WORK!

All the following commands SHALL be run from the ci-scripts/yaml_files/5g_l2sim_tdd folder.

The gNB, proxy and NR-UE containers will be deployed in host-mode. It will use the host loopback interface to connect.

sudo ifconfig lo: 127.0.0.2 netmask 255.0.0.0 up

2.1. Deploy OAI 5G Core Network

$ cd ci-scripts/yaml_files/5g_l2sim_tdd
$ docker-compose up -d mysql oai-nrf oai-amf oai-smf oai-spgwu oai-ext-dn
Creating network "l2sim-oai-public-net" with driver "bridge"
Creating network "l2sim-oai-traffic_net-net" with driver "bridge"
Creating l2sim-oai-nrf ... done
Creating l2sim-mysql      ... done
Creating l2sim-oai-spgwu ... done
Creating l2sim-oai-amf   ... done
Creating l2sim-oai-smf   ... done
Creating l2sim-oai-ext-dn ... done

Wait for a bit.

$ docker-compose ps -a
       Name                     Command                  State                  Ports
-------------------------------------------------------------------------------------------------
l2sim-mysql        docker-entrypoint.sh mysqld      Up (healthy)   3306/tcp, 33060/tcp         
l2sim-oai-amf      /bin/bash /openair-amf/bin ...   Up (healthy)   38412/sctp, 80/tcp, 9090/tcp
l2sim-oai-ext-dn   /bin/bash -c  apt update;  ...   Up (healthy)                               
l2sim-oai-nrf      /bin/bash /openair-nrf/bin ...   Up (healthy)   80/tcp, 9090/tcp            
l2sim-oai-smf      /bin/bash /openair-smf/bin ...   Up (healthy)   80/tcp, 8805/udp, 9090/tcp  
l2sim-oai-spgwu    /openair-spgwu-tiny/bin/en ...   Up (healthy)   2152/udp, 8805/udp

At this point, you can prepare a capture on the newly-created public docker bridges:

$ ifconfig
...
l2sim-public: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.71.129  netmask 255.255.255.192  broadcast 192.168.71.191
        inet6 fe80::42:c4ff:fe2b:3d38  prefixlen 64  scopeid 0x20<link>
        ether 02:42:c4:2b:3d:38  txqueuelen 0  (Ethernet)
        RX packets 4  bytes 112 (112.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 7  bytes 626 (626.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

l2sim-traffic: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.72.129  netmask 255.255.255.192  broadcast 192.168.72.191
        inet6 fe80::42:b5ff:fed3:e732  prefixlen 64  scopeid 0x20<link>
        ether 02:42:b5:d3:e7:32  txqueuelen 0  (Ethernet)
        RX packets 2652  bytes 142335 (142.3 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 3999  bytes 23367972 (23.3 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
...

$ sudo nohup tshark -f "(host 192.168.72.135 and icmp) or (not host 192.168.72.135 and not arp and not port 53 and not port 2152 and not port 2153)" -i l2sim-public -i l2sim-traffic -w /tmp/capture_5g_l2sim_tdd.pcap > /tmp/tshark.log 2>&1 &

2.2. Deploy OAI gNB in Standalone Mode as a VNF

CAUTION: To execute this 2nd step, the whole CN5G SHALL be in healthy state (especially the mysql container).

$ docker-compose up -d oai-gnb
l2sim-oai-nrf is up-to-date
l2sim-oai-spgwu is up-to-date
l2sim-oai-ext-dn is up-to-date
Creating l2sim-oai-gnb ... done

Wait for a bit.

$ docker-compose ps -a
       Name                     Command                  State                  Ports
-------------------------------------------------------------------------------------------------
l2sim-mysql        docker-entrypoint.sh mysqld      Up (healthy)   3306/tcp, 33060/tcp         
l2sim-oai-amf      /bin/bash /openair-amf/bin ...   Up (healthy)   38412/sctp, 80/tcp, 9090/tcp
l2sim-oai-ext-dn   /bin/bash -c  apt update;  ...   Up (healthy)                               
l2sim-oai-gnb      /opt/oai-gnb/bin/entrypoin ...   Up (healthy)                               
l2sim-oai-nrf      /bin/bash /openair-nrf/bin ...   Up (healthy)   80/tcp, 9090/tcp            
l2sim-oai-smf      /bin/bash /openair-smf/bin ...   Up (healthy)   80/tcp, 8805/udp, 9090/tcp  
l2sim-oai-spgwu    /openair-spgwu-tiny/bin/en ...   Up (healthy)   2152/udp, 8805/udp

You can verify that the gNB is connected with the AMF:

$ docker logs rfsim5g-oai-amf
...
[AMF] [amf_app] [info ] |----------------------------------------------------gNBs' information-------------------------------------------|
[AMF] [amf_app] [info ] |    Index    |      Status      |       Global ID       |       gNB Name       |               PLMN             |
[AMF] [amf_app] [info ] |      1      |    Connected     |         0x0       |         gnb-l2sim-vnf    |            208, 99             |
[AMF] [amf_app] [info ] |----------------------------------------------------------------------------------------------------------------|
...

2.3. Deploy OAI NR-UE and proxy

$ docker-compose up -d proxy oai-nr-ue0
l2sim-mysql is up-to-date
l2sim-oai-nrf is up-to-date
l2sim-oai-amf is up-to-date
l2sim-oai-smf is up-to-date
l2sim-oai-spgwu is up-to-date
l2sim-oai-ext-dn is up-to-date
l2sim-oai-gnb is up-to-date
Creating l2sim-oai-nr-ue ... done
Creating l2sim-proxy ... done

Wait for a bit.

$ docker-compose ps -a
       Name                     Command                  State                  Ports
-------------------------------------------------------------------------------------------------
l2sim-mysql        docker-entrypoint.sh mysqld      Up (healthy)   3306/tcp, 33060/tcp         
l2sim-oai-amf      /bin/bash /openair-amf/bin ...   Up (healthy)   38412/sctp, 80/tcp, 9090/tcp
l2sim-oai-ext-dn   /bin/bash -c  apt update;  ...   Up (healthy)                               
l2sim-oai-gnb      /opt/oai-gnb/bin/entrypoin ...   Up (healthy)                               
l2sim-oai-nr-ue0   /opt/oai-nr-ue/bin/entrypo ...   Up (healthy)                               
l2sim-oai-nrf      /bin/bash /openair-nrf/bin ...   Up (healthy)   80/tcp, 9090/tcp            
l2sim-oai-smf      /bin/bash /openair-smf/bin ...   Up (healthy)   80/tcp, 8805/udp, 9090/tcp  
l2sim-oai-spgwu    /openair-spgwu-tiny/bin/en ...   Up (healthy)   2152/udp, 8805/udp          
l2sim-proxy        /oai-lte-multi-ue-proxy/bi ...   Up (healthy)                               

$ docker stats --no-stream --format "table {{.Container}}\t{{.CPUPerc}}	{{.MemUsage}}\t{{.MemPerc}}" l2sim-mysql l2sim-oai-amf l2sim-oai-ext-dn l2sim-oai-gnb l2sim-oai-nr-ue0 l2sim-oai-nrf l2sim-oai-smf l2sim-oai-spgwu l2sim-proxy
CONTAINER          CPU %     MEM USAGE / LIMIT     MEM %
l2sim-mysql        0.03%     206.7MiB / 62.54GiB   0.32%
l2sim-oai-amf      4.05%     29.49MiB / 62.54GiB   0.05%
l2sim-oai-ext-dn   0.00%     31.27MiB / 62.54GiB   0.05%
l2sim-oai-gnb      1.29%     1.853GiB / 62.54GiB   2.96%
l2sim-oai-nr-ue0   1.43%     350.8MiB / 62.54GiB   0.55%
l2sim-oai-nrf      0.21%     9.105MiB / 62.54GiB   0.01%
l2sim-oai-smf      3.24%     30.23MiB / 62.54GiB   0.05%
l2sim-oai-spgwu    0.00%     11.78MiB / 62.54GiB   0.02%
l2sim-proxy        6.97%     290.4MiB / 62.54GiB   0.45%

CAUTION: As you can see the CPU usage is not that important compared to a 5G RF simulator equivalent.

But the CPU speed matters; I am running on a fast server:

$ lscpu
Architecture:        x86_64
CPU(s):              16
...
Model name:          Intel(R) Xeon(R) Silver 4215R CPU @ 3.20GHz

I tried on a slower server with more CPUs and it did not work:

oaici@orion:~$ lscpu
Architecture:        x86_64
CPU(s):              48
...
Model name:          Intel(R) Xeon(R) CPU E5-2658 v3 @ 2.20GHz

We will work on this issue.

Making sure the OAI UE is connected:

$ docker exec -it l2sim-oai-nr-ue /bin/bash
root@bb4d400a832d:/opt/oai-nr-ue# ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.71.137  netmask 255.255.255.192  broadcast 192.168.71.191
        ether 02:42:c0:a8:47:89  txqueuelen 0  (Ethernet)
        RX packets 224259  bytes 5821372018 (5.8 GB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 235916  bytes 7848786376 (7.8 GB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

oaitun_ue1: flags=4305<UP,POINTOPOINT,RUNNING,NOARP,MULTICAST>  mtu 1500
        inet 12.1.1.2  netmask 255.255.255.0  destination 12.1.1.2
        unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  txqueuelen 500  (UNSPEC)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

3. Check traffic

3.1. Check your Internet connectivity

You can also check with the ext-dn container (IP address is 192.168.72.135 in docker-compose)

$ docker exec -it l2sim-oai-nr-ue /bin/bash
root@bb4d400a832d# ping -I oaitun_ue1 -c 20 192.168.72.135
PING 192.168.72.135 (192.168.72.135) from 12.1.1.2 oaitun_ue1: 56(84) bytes of data.
64 bytes from 192.168.72.135: icmp_seq=1 ttl=63 time=65.1 ms
64 bytes from 192.168.72.135: icmp_seq=2 ttl=63 time=74.2 ms
...
64 bytes from 192.168.72.135: icmp_seq=19 ttl=63 time=25.0 ms
64 bytes from 192.168.72.135: icmp_seq=20 ttl=63 time=34.7 ms

--- 192.168.72.135 ping statistics ---
20 packets transmitted, 20 received, 0% packet loss, time 19025ms
rtt min/avg/max/mdev = 16.413/120.639/209.673/57.159 ms

4. Un-deployment

$ docker-compose down
Stopping l2sim-oai-nr-ue2 ... done
Stopping l2sim-oai-nr-ue  ... done
Stopping l2sim-oai-gnb    ... done
Stopping l2sim-oai-ext-dn ... done
Stopping l2sim-oai-spgwu  ... done
Stopping l2sim-oai-smf    ... done
Stopping l2sim-oai-amf    ... done
Stopping l2sim-oai-nrf    ... done
Stopping l2sim-mysql      ... done
Removing l2sim-oai-nr-ue2 ... done
Removing l2sim-oai-nr-ue  ... done
Removing l2sim-oai-gnb    ... done
Removing l2sim-oai-ext-dn ... done
Removing l2sim-oai-spgwu  ... done
Removing l2sim-oai-smf    ... done
Removing l2sim-oai-amf    ... done
Removing l2sim-oai-nrf    ... done
Removing l2sim-mysql      ... done
Removing network l2sim-oai-public-net
Removing network l2sim-oai-traffic-net

5. Adapt the docker-compose to your environment

In the SMF section, provide your own DNS IP address:

            DEFAULT_DNS_IPV4_ADDRESS=172.21.3.100

In the gNB section, provide your docker-host primary IP address and interface name: in our case 172.21.16.128 and eno1.

            GNB_NGA_IF_NAME: eno1
            GNB_NGA_IP_ADDRESS: 172.21.16.128
            GNB_NGU_IF_NAME: eno1
            GNB_NGU_IP_ADDRESS: 172.21.16.128

Same thing in the nr-ue section:

            NR_UE_NFAPI_IF_NAME: eno1

This tutorial is a first draft. This nFAPI feature and the proxy are still under development.

At time of writing, we were able to run in host-mode, 1 NR-UE and just ping traffic.

Later development will include:

  • deploying gNB-VNF, proxy and UE in isolated containers (with their own IP address)
  • more UEs
  • more traffic (UDP and TCP)
Reference in New Issue View Git Blame Copy Permalink