Some code in T tracers is used by both LTE and NR.
In LTE, we expect 10 subframes per frame, and the logging is done with
this assumption. We have one tick per subframe.
In NR, we deal with slots, not subframes. And we have n ticks per frame
(depending on mu). As of today, only n=20 is tested (corresponding to
mu=1).
A previous commit introduced 'subframes_per_frame' which had the correct
meaning for LTE but was truly 'slots per frame' for NR. This creates
confusion.
Let's replace 'subframe' by 'tick' to reduce confusion.
Interoperability testing with srsRAN DU shows than when receiving
RRCReconfigurationCompleteIndicator in a UEContextModificationRequest
the DU replies in the UEContextModificationResponse with an empty
CellGroupConfig.
This CellGroupConfig is encoded as 2 zero bytes. Let's ignore this
message when we receive it, don't forward it to the UE in a
RRC Reconfiguration, as it should per standard (38.473 8.3.4.2):
If the CellGroupConfig IE is included in the DU to CU RRC Information
IE contained in the UE CONTEXT MODIFICATION RESPONSE message, the
gNB-CU shall perform RRC Reconfiguration as described in TS 38.331 [8].
The CellGroupConfig IE shall transparently be signaled to the UE as
specified in TS 38.331 [8].
To be removed/reworked in the future if the interoperability
situation improves.
This is in 3GPP TS 38.425.
Because it was not done, there was a problem of interoperability
between openair CU and srsran DU. No DL traffic was reaching the UE.
A new GTPU API function is introduced: gtpv1uSendDirectWithNRUSeqNum().
It is only used in the gNB CU, in PDCP. So it was decided to not add a
new parameter to gtpv1uSendDirect() but instead add a new API function.
Think: NR with different number of slots per frame depending on mu.
The name 'subframes_per_frame' is a bit ambiguous. It is valid for LTE,
but for NR it should be 'slots_per_frame'.
For example, for sub-carrier spacing of 30KHz, we have 20 slots per
frame, so in this case subframes_per_frame = 20. In reality whatever
the numerology, there is always 10 subframes per frame in NR.
But how to name this variable which is used both for LTE and NR cases?
'subframes_or_slots_per_frame'? It's a bit long and also ambiguous.
Conclusion: no big deal I think. 'subframes_per_frame' and so be it.
The CI fails sometimes, because some processes/threads are created in
unpredictable order and if the client connects before the server is
ready some tests fail, as seen in the following log:
[UTIL] client/server iq-samples: launch sub-processes
[UTIL] threadCreate() for time client: creating thread (no affinity, default priority)
[UTIL] time manager configuration: [time source: server] [mode: client] [server IP: 127.0.0.1} [server port: 7374]
[UTIL] time client: connection to 127.0.0.1:7374 failed, try again in 1s
[UTIL] threadCreate() for time source iq samples: creating thread (no affinity, default priority)
[UTIL] threadCreate() for time server: creating thread (no affinity, default priority)
[UTIL] time manager configuration: [time source: iq_samples] [mode: server] [server IP: 127.0.0.1} [server port: 7374]
[UTIL] client/server: rlc ticks: 1000 pdcp ticks: 1000 x2ap ticks: 1000
CMDLINE: "/oai-ran/build/common/utils/time_manager/tests/test_time_manager_auto" "--time_management.mode" "server" "--time_management.time_source" "iq_samples"
[UTIL] client/server: rlc ticks: 0 pdcp ticks: 0 x2ap ticks: 0
CMDLINE: "/oai-ran/build/common/utils/time_manager/tests/test_time_manager_auto" "--time_management.mode" "client"
[UTIL] client-server iq-samples: ERROR
The client/server iq-samples test is started.
Then we see this log:
[UTIL] time manager configuration: [time source: server] [mode: client] [server IP: 127.0.0.1} [server port: 7374]
Which means that the client is running.
It tries to connect to the server (not running yet), but fails, so sleeps
for 1 second before attempting another connection to the server:
[UTIL] time client: connection to 127.0.0.1:7374 failed, try again in 1s
The sleep is done by the time_client thread, the main thread of the test
program continues and does a sleep(1).
Then the server arrives:
[UTIL] time manager configuration: [time source: iq_samples] [mode: server] [server IP: 127.0.0.1} [server port: 7374]
It sends all the ticks (to no client, since the client is still sleeping).
Then the test is over, the sleep(1) of the client finishes and the program
exits. The client never connected to the server.
We have this log:
[UTIL] client/server: rlc ticks: 0 pdcp ticks: 0 x2ap ticks: 0
And the test fails.
It's complicated to write an automatic test involving several uncoordinated
programs. The operating system may create processes in any order. We use
some sleeping to wait, but it may be too short or too long, leading to test
failure.
Let's see with more sleep if the CI fails less.
This was a problem with open5gs and more than one PDU session.
The core network sets UE AMBR only once, leading to failure in the gNB
for the second PDU session.
(A proper handling of AMBR has to be develop at some point in the gNB.)
rx_list is replaced by an array of size 2^(SN size - 1)
which is 131072 for an SN size of 18 bits, leading to
1048576 bytes of memory used. (Less for smaller SN sizes.)
A lot of RX processing was to run through rx_list. When the list was
big (with some non-ideal channel), it can take some time. Removing this
list should improve things. Checking with benchmark_nr_rlc_am_entity
a speedup of more thant x10 is achieved (except for the case 1024 which
is surprisingly faster than the others in the develop branch).
The only thing that may be slower is when looking for cur->next where
the SN is different, in which case we need to look for all SN one by
one until we find something or reach the limit (rx_highest_status).
These are the functions get_first_rx_pdu() and get_next_rx_pdu().
- remove hardcoded values
- adapt gNB RRC code
- adapt gNB MAC code
- adapt physical simulators, where we put the hardcoded values
that were everywhere else before; if there is a need for those
simulators to get configurable values, more work is needed
(but it's doubtful that it is needed)
To be used in a later commit to retrieve NR PDCP configuration
parameters.
To use, add the following to the configuration file (outside of other
configurations). Adapt the values to your setup. The names ("len18bits",
"ms100", etc.) come from 3GPP TS 38.331.
pdcp = {
drb = {
sn_size = "len18bits"
t_reordering = "ms100"
discard_timer = "infinity"
}
}
Default values are applied if the configuration is absent.
The default values are the ones given above.
To be used in a later commit to retrieve NR RLC configuration
parameters.
To use, add the following to the configuration file (outside of other
configurations). Adapt the values to your setup. The names ("ms45",
"size12", etc.) come from 3GPP TS 38.331.
rlc = {
srb = {
t_poll_retransmit = "ms45"
t_reassembly = "ms25"
t_status_prohibit = "ms0"
poll_pdu = "infinity"
poll_byte = "infinity"
max_retx_threshold = "t8"
sn_field_length = "size12"
}
drb_am = {
t_poll_retransmit = "ms45"
t_reassembly = "ms15"
t_status_prohibit = "ms15"
poll_pdu = "p64"
poll_byte = "kB500"
max_retx_threshold = "t32"
sn_field_length = "size18"
}
drb_um = {
t_reassembly = "ms15"
sn_field_length = "size12"
}
}
Default values are applied if the configuration is absent.
The default values are the ones given above.
When data is generated by "normal" T tracers, it should be fine.
But it might come from unknown sources and then content might be
"strange". So let's validate input data a bit more.
The way allocation is done (granularity of 64KiB) makes a crash more
difficult to achieve, but if a trace is exactly 64KiB, we miss 4 bytes
in the realloc and then anything may happen.
In NR with several layers, we may have MAC PDU of size bigger than 64KB,
which does not fit into a UDP buffer, destroying macpdu2wireshark.
Let's add an option to dump to a pcap file instead of using UDP packets.
The DLT 252 magic is used (check wireshark documentation to understand).
Run macpdu2wireshark with -h to see how to configure wireshark to read
the produced files.
The following error was appearing when running LTE UE:
inet_pton(): cannot convert -64.-74.0.2 to IPv4 network address
The allocated address by the core actually is 192.182.0.2.
On my host "char" is signed, leading to the above error (192 and 182
are seen as signed).
Let's force "ip_addr" unsigned and everyone is happy.
In Bearer Context Modification Request, the security information
field is needed only if we change the PDCP algorithms/keys. Since we
don't, no need to pass it.
Let's keep the old code, maybe the current understanding of the
specifications is wrong.
If the Bearer Context Modification Request does not contain security
information, the PDCP entity shall keep the currently configured security
settings.
Passing -1 for ciphering_algorithm and integrity_algorithm is the way
to implement this logic.
Check if the IE is memory allocated to deal with presence/absence of this information
element in Bearer Context Modification Request and add encoding/decoding
in the code.
Note that for decoding the integrity settings may be absent, in which
case we consider NIA0 to be used (no integrity protection). To be refined
if needed.
And for encoding, we always encode integrity settings, even if it's NIA0.
May also be refined if needed.
This was not a big problem, but we were advertising wrong
algorithm and deriving keys using this wrong algorithm in case
DRB ciphering and/or integrity was not active. (Say SRBs
are configured with NIA2 but DRBs are configured without
integrity, we would advertise NIA2 and send the integrity
key derived for NIA2, instead of NIA0 for both.)
It was not a problem because in the PDU Session Resource To Setup
item, there is "securityIndication" which we use to effectively
activate/deactivate the ciphering and/or integrity.
But it did not look clean to see a SecurityInformation with
incorrect data when inspecting the message in wireshark.
So let's use the correct values.
We could also not include the SecurityInformation if both ciphering
and integrity are not used, and only include ciphering if integrity
is not used (because integrity settings are optional).
But then if only integrity is used, we still need to include
ciphering, setting algorithm to NEA0 (no ciphering), because
the ciphering settings are always included.
This logic is too complex, let's use the simple one to always
include SecurityInformation with NEA0 and/or NIA0 if ciphering
and/or integrity is not activated.
Useful to debug realtime issues, for example when calling a periodical
routine, if it's not programmed carefully enough, it may be that the first
call takes way longer than the next ones (for example it uses some memory
that is not mapped in the process, leading to many page faults the first
time it's executed).
Having a periodic log for the timing of this routine with min/avg/max/count
will reveal that a call is way longer than average and will help in
finding where exactly in the code the time is spent, thanks to the ease
of use of the T tracer; you put T(T_XX, T_INT(1)) just before the code
you want to measure, you put T(T_XX, T_INT(0)) just after and time_meas
will show the delta time between those events. Then you cut the routine
in smaller pieces, measure with T(1)/T(0), and you finally reach the small
guilty part that takes too long.
In time_meas.c, when getting 0 for the event, we log delta time with
previous 1 for the event. "start_valid" was not cleared, so receiving
a second 0 without 1 in between may lead to wrong logging.
Should not happen in practice, but still, little bug.
(Note: we don't protect against streams of 1, but this case should also
not happen.)
MIB, SIB1, random access and regular scheduling are traced.
At the beginning of a connection (rrc setup request and rrc setup), the
RNTI is reported as 0. Might be fixed, somehow, if absolutely needed (not
sure).
./run_tests.sh was not functional anymore because of the introduction
of AssertFatal() in nr_rlc_entity.c
Including the .h and providing exit_function() solves the problem.
Some people needed that.
Let's stop to 19 for the time being. It's tricky enough.
We can increase later if needed. (A previous commit
made the tracers to accept up to 32 variables.)
T_MAX_ARGS was 16 and some people start to use the T machinery with
more than 16 values. So be it. Let tracers be friendly and accept 32.
(More commits to follow for full support.)
Also why is T_MAX_ARGS in T_defs.h when it's only used by event.h?
Hum?
It is very possible that the operating system returns a file
descriptor bigger than the size of bridge->buf (MAX_FD_RFSIMU, which
is 250 as of now). As a minimum, let's fail if the socket file
descriptor is out of bounds.
A proper solution may be implemented later, if needed.
Integration: `2024.w33`
See merge request oai/openairinterface5g!2911
- !2784 Reworking beam structures at NR MAC gNB
- !2889 Fix delta for PUSCH DMRS type 2
- !2908 NR MAC UE CSI improvements
- !2896 Fix for llr functions reading/writing outside buffers
- !2902 nr rlc: reduce "SDU rejected" logging
- !2884 NR UE: Cleanup global variables used for command line parameter storage
- !2910 NR gNB DMRS symbol race
- CI: Update phytest-timing test threshold
- make it work for SRB or DRB
- adapt callers of this function
- cleanup key derivation: the User Plane keys are generated at some places
but are not used, remove the generation
Cedric Roux