nFAPI: make 4-layer on 100MHz work
5G nFAPI headers specify 32 bits for the length field, so use it also
when segmenting messages. This should stabilize 4-layer MIMO operation
on 100MHz with nFAPI.
Deadlock avoidance in rfsimulator
This change introduces a countermeasure for deadlock in rfsimulator. The
deadlock happens when all entities are waiting for new data to come in,
and happens with 2+ clients, when a new client connects. I think this
issue is due to ordering of fullwrite calls, resulting in out-of-order
delivery of packets and eventually trashing the packets on the receiving
side. The out-of-order delivery warnings are printed just before the
system deadlocks but I have not found a better solution so far. The
workaround makes the server never lock up permanently by ignoring the
client failure to write on time after 10 tries.
This was tested locally for both UE as server and gNB as server and
works correctly, causing the deadlock to clear and the added log to be
printed several times when the deadlock is detected, after which the
system goes back to normal.
I have some gdb output of the executables during deadlock:
UE:
$7 = {conn_sock = 98, lastReceivedTS = 3226163740, headerMode = true, trashingPacket = false, th = {size = 13184, nbAnt = 1, timestamp = 3226150556, option_value = 0, option_flag = 0}, transferPtr = 0x7f6a500018a8 "\200\063", remainToTransfer = 24,
circularBufEnd = 0x7f6a503b3ac0 "", circularBuf = 0x7f6a501f1ac0, channel_model = 0x0}
(gdb) p t->buf[5]
$8 = {conn_sock = 97, lastReceivedTS = 0, headerMode = true, trashingPacket = false, th = {size = 0, nbAnt = 0, timestamp = 0, option_value = 0, option_flag = 0}, transferPtr = 0x7f6a50001900 "", remainToTransfer = 24, circularBufEnd = 0x7f6a50575ad0 "",
circularBuf = 0x7f6a503b3ad0, channel_model = 0x0}
nextRxTimestamp 3225937740
nsamps = 30720
gNB 1:
(gdb) p t->buf[0]
$4 = {conn_sock = 95, lastReceivedTS = 3226026876, headerMode = true, trashingPacket = false, th = {size = 1, nbAnt = 1, timestamp = 3226026875, option_value = 0, option_flag = 0},
transferPtr = 0x7f8dfc003ab8 "\001", remainToTransfer = 24, circularBufEnd = 0x7f8e1c3ff010 "", circularBuf = 0x7f8e1c23d010, channel_model = 0x0}
nextRxTimestamp 3225996956
gNB 2:
lastReceivedTS = 3226026875
$2 = {conn_sock = 95, lastReceivedTS = 3226026875, headerMode = true, trashingPacket = false, th = {size = 1, nbAnt = 1, timestamp = 3226026875, option_value = 0, option_flag = 0},
transferPtr = 0x744898003ab8 "\001", remainToTransfer = 24, circularBufEnd = 0x7448bc2e7010 "", circularBuf = 0x7448bc125010, channel_model = 0x0}
nextRxTimestamp 3226026875
As you can see all executables are in have_to_wait state.
Make the same change as in parent commit, i.e., write the full 32-bit
segment size.
I could not test this, as we do not reach these rx_data.indication
length. It's based on the fix in the parent commit, and to avoid future
bad surprises.
The 5G nFAPI message length is 32bits. In particular tx_data.requests
can be longer than 64kB. When segmenting, we should correctly write the
message of the current segment (across all 32bits), because the
length would interpreted wrongly otherwise.
This fixes a bug in which tx_data.requests were discarded for 4-layer DL
MIMO on 100 MHz with this message:
P7 unpack message length is greater than the message buffer
Further, increase the type of various (segment-related) variables to 32
bits. Currently, the maximum segment size is sxt to 65000 bytes (and in
will likely remain, because the maximum UDP size is 65536);
nevertheless, increase it in case we will ever go beyond this.
See also commit dee68e6319 ("nFAPI: increase maximum segment size to
65000")
Simplify usage of the old segment decoding libraries with the slot coding interface
This MR transforms the segment coding libraries into slot coding
libraries by statically linking the adaptation layer and the segment
coding libraries.
Therefore they can be used directly in ldpctest which uses the segment
coding interface as well as in all the other PHY simulators and the NR
softmodem which use the slot coding interface.
It is not anymore needed to use the
--nrLDPC_coding_segment.segment_shlibversion
flag. Instead a segment coding library can be directly chosen with
--loader.ldpc.shlibversion
since it implements the slot coding interface.
Downgrade gNB power limit LOG from "warning" to "debug"
Downgrade the UE power limited LOG from LOG_W to LOG_D. This is to avoid
flooding stdout when UE in low coverage enters power limited state for a
longer period of time. The same information can be inferred from the
periodic UE print which also contains PH value (negative PH values
indicate that the UE is power limited)
Also, correct some typos in PH calculation.
Closes#885
Changes to support multiple TDD patterns
- Updated the configuration changes for 2 Patterns
- Update the TDD table configuration for NFAPI
- Added the tdd bitmap for the period
- Adapted the bitmap for UL/DL for the multi TDD pattern
- Updated the RACH procedure for multi TDD pattern
- Updated DL and UL scheduler for multi TDD pattern
Parametrized JenkinsNode and JenkinsResource
Parametrized variables in Jenkinsfile for automated tests on Colosseum
after update of OAI Jenkins server.
NAS 5GS refactor
The goal of this MR is to reorganize the NR NAS code into:
- enc/dec library functions (5GS, 5GMM, 5GSM, NR_NAS_defs.h,
nr_fgs_nas_lib.c)
- handlers and callbacks (nr_nas_msg.c)
The following changes are introduced:
1. migrate 5GS enc/dec lib (5GMM and 5GSM) to a dedicated folder
2. add a nested CMakeLists structure
3. refactor NR NAS enc/dec functions
4. refactor NR NAS messages struct definitions
5. remove most of dependency from NAS LTE
6. further refactor of PDU Session Establishment Accept processing to
separate enc/dec from handling
7. made naming of 5GS definitions consistent whenever necessary
8. cleanup unused code
9. remove redundant struct definitions
Also:
1. fix decoding of AMF Set ID in 5G-S-TMSI UE identity
2. replace Byte_t directly with uint8_t
FHI72: fix for a distributed antenna array for xran F release
in the Rx callback I reset the number of section descriptions/number of
fragments to 0, but for each antenna up to max number of antennas per
RU, instead of up to max number of distributed antenna array
Ensure execution of processSlotTX in order in NR UE
Make sure that processSlotTX is executed in slot order.
There is a bug that causes decoding error when processSlotTX is executed
out of order. The root cause is not known at this point, but this change
avoids the bug altogether.
Add IQ file recording and IQ file viewer to ImScope
This changeset introduces the following changes to ImScope:
- Added the ability to record IQ data using the same mechanism as
ImScope.
- Added a new executable target imscope_iq_file_viewer for viewing IQ
files.
- Introduced a command line argument --imscope-record that enables data
recording mode.
- Refactored a lot of the scope code
Usage:
- To record IQ data, run a modem with --imscope-record flag. The
recording happens whenever a modem calls dumpScopeData, which
currently is done only in gNB on PUSCH DTX and PUSCH CRC NOK. There
is a limit of ~1GB of files written by the thread in a signle run to
avoid accidentally exhausting disk space. If a directory imscope-dump
is available the data will be recorded there in .imscope files.
Otherwise files are written to the directory the modem was run. This
is done so that the directory can be mapped in a docker container.
- To view recorded IQ files, use the new executable
imscope_iq_file_viewer. This can be done by running:
./imscope_iq_file_viewer <path_to_iq_file>
Replace <path_to_iq_file> with the path to the IQ file you want to view.
Dockerized development environment
This commit introduces a fully dockerized dev env you can start on any
machine that has docker without installing any direct OAI dependencies
on host.
This allows to migrate development environments between machines easily.
Several complementary scripts were added to enable smooth transition
between your docker environment and host environment. Your user and sudo
should work inside the environment.
In its current state, the environment can be used to compile
nr-softmodem with 7.2 FHI but it cannot run it due to missing
dependencies.
This function packs P7 messages. At least RX_data.indication messages
might be much bigger than the global buffer that is used here. Allocate
the buffer on the stack, and make it bigger. Do not use the global
buffer, it's simply not necessary; increasing the global tx buffer size
might have negative effects elsewhere in the code.
That change might make the function reentrant. The mutex seems to only
guards the codec config. However, leave it to not introduce any
regressions as of now.
E1 Setup enc/dec library and unit test
This MR is adding a library for E1 Setup enc/dec functions:
- E1 CU-UP Setup Request
- E1 CU-UP Setup Response
- E1 Setup Failure
and relevant unit tests
F1AP IDs: Add update, fix concurrency bug
I suspect a concurrency bug: For instance, during reestablishment, the
CU needs to update the DU UE ID under which the UE is identified in the
DU. Previously, the CU would remove, then add the DU UE ID info. At the
same time, the PDCP thread might look up the information. This can lead
to asserts. Modify the code to do the update under (one) lock.
Also, refactor some code.
Similar to the parent commit, make the numerology in the VNF
configurable. Unlike for the PNF change, the VNF frame/slot info is in a
per-PNF connection structure to which the "oai_integration" code has no
access. So we need to hack the nfapi_vnf_p7_add_pnf() function signature
to take the numerology, to be able to save this on a per-PNF basis.
The fact that we store this on a per-PNF basis is not strictly required,
as the VNF cannot handle multiple numerologies as of now, either.
However, we might want to extend this in the future, so it seems prudent
to store this on a per-PNF basis, so that we could start handling
multiple numerologies at the L2 without the need to change the L1 or
nFAPI code itself.
Also, the numerology is not needed except for some code that deals with
timing over nFAPI. As of now, we don't use this code (nFAPI gets a
trigger every slot, as per the VNF request, see an earlier commit in
this series), but also here, let's not make the future more complicated
by storing the numerology for each PNF now (this could always be reverted).
For 4G, put numerology 0 (15kHZ SCS), which is the SCS that LTE uses.
Fill in the dynamically received numerology, and make the numerology at
the PNF configurable, following the changes in the parent commit.
The change for the reading is somewhat of a hack, because nFAPI, as of
now, does not really foresee to store the numerology, so we fill the
numerology during the start request, when the numerlogy has been
received in the config request, but at that time, the structure for the
time information (frame/slot) does not exist yet.
At both the PNF and the VNF, introduce a separate numerology field, and
pass the numerology when converting time information using macros from
nfapi_interface.h. The actually numerology is still hardcoded to 1 (as it
was before), but the follow-up commits will put a suitable numerology.
In both cases, but the numerology next to the frame/slot information.
Similar as the parent comment, remove time macros. In this particular
case, we can just pass the exact frame/slot information, instead of
squeezing it in a single integer.
The next commits will modify the time macros used in the 5G nFAPI code
to handle different numerologies. As a preparation, remove all the
usages of these macros where they really don't matter (e.g., don't
convert if we don't need to).
the L1 needs both a PDU (in TX_data.req) as well instructions for
the actual transmission (in DL_TTI.req) to encode a PDSCH message. In
nFAPI, it can happen that a DL_TTI.request message has been received
(configuring the PDSCH transmission), without the TX_data.request having
reached the PNF. The L1 assumes to have the PDU.
To avoid problems, ensure that only the pair of DL_TTI.req/TX_data.req
is delivered. Otherwise, drop either message.
nFAPI has a mechanism to segment messages that are too big for transport
over a given medium (see e.g. SCF 225, section 2.3.2).
The maximum segment size of 10000 makes that for larger payloads, e.g.
TX_data.request, many small segments are to be sent. This can create or
increase delays on the transport. On the other hand, the currently only
available transport mechanism, UDP, allows to transport packets of up to
almost 65535 bytes. Correspondingly, increase the maximum segment size
so that less segments are to be created, and potentially, less delay is
to be incurred. # Please enter the commit message for your changes.
Lines starting
Avoid delays in tx_data.request handling by avoiding big malloc()s and
copy operations. Reimplement function to (1) peek the frame/slot
numbers, (2) decide on buffer, and (3) unpack directly into
pre-allocated memory.
Co-authored-by: hsum <ming-hong.hsu@eurecom.fr>
Co-authored-by: chenyi <Yi-Quan.Chen@eurecom.fr>
Co-authored-by: Rúben Soares Silva <rsilva@allbesmart.pt>
The time that a message between PNF/VNF to arrive depends mostly on the
transport. To allow for some delays, there is a slot_ahead time, during
which the VNF is allowed to schedule and send instructions to the PNF.
This can be multiple slots; the 1ms hitherto given might typically too
short. Increase to 10ms, to encompass a wider range of slot_ahead times.
Make the corresponding log message of when old (stale) message are
removed a bit clearer with respect to times.
This is the first commit in which nFAPI works. Follow-up commits improve
performance.
The usage of this flag in the L1 is simply wrong: either, the NFAPI mode
runs L1, or not. Correspondingly, add an assert to check this invariant.
In the NR_IF_module, the correct check to do is for VNF (implying no L1,
or "emulation of L1" [which is not true]), in which case we use
queues.
The L1 receives a pointer to the L1 payload to use for PDSCH generation.
This would overwrite an existing pre-allocated pointer. Remove this
preallocation. Instead, allocate it in the simulator (the only place
that actively uses this buffer allocation; in nr-softmodem, it's lost by
assigning a pointer). Allocate with an extra byte, because the payload
(in bits) might not be a multiple of 8 bits/1 byte, so the CRC is in the
last 3 bytes.
Also, improve the log with frame/slot information if the PDSCH payload
pointer is NULL.
Since commit defc97ec16 ("Msg2 and Msg3 at gNB: check allocation before
committing to Msg2"), the scheduler calculate Msg2/Msg3 frame/slot
before allocation of Msg2, and not when receiving Msg1. It also respects
the RA response window correctly.
Correspondingly, we don't need to calculate here if the RACH.indication
is within the time window. The scheduler will do this.
Reset the counters. In monolithic, this happens in the scheduler; in the
PNF, without this change, it would trigger PDCCH again.
Co-authored-by: hsum <ming-hong.hsu@eurecom.fr>
Refactor pnf_phy_ul_dci_req(), pnf_phy_dl_tti_req(),
pnf_phy_tx_data_req(), pnf_phy_ul_tti_req(), to use L1 functions to
"load" L1 jobs to be executed, via L1 functions.
In monolithic, the sched_response mechanism ensures that FAPI messages
for one slot are not overwritten by scheduler instructions for the
following slots (i.e., it is like a memory allocation pool).
The allocation happens in nr_schedule_response(), not used by the PNF.
Hence, only deallocate when in monolithic.
The PNF uses a big ring buffer over all slots in a frame to ensure the
same.
This function applied an sf_ahead, bigger than the actual slot
indication message sending (so a message could NEVER arrive on time!).
Reduce to zero, because this is when it should arrive for us.
FAPI says these lists should be for MAX_TDD_PERIODICITY (which is maybe
a frame, or a TDD period? but likely not two frames). Reduce to one
frame.
Also, the nFAPI code assumes one entire frame, so use that (this commit
necessary to actually make the PNF work, otherwise, the
packing/unpacking of the FAPI message would need to do 2 frames).
Closes: #876
Upon reception of FAPI UL messages (e.g., RACH.indication), those
messages are put into queues, to be delivered in a single call,
NR_UL_indication(). Call the scheduler to trigger processing in UL.
Note that ideally, these queues should not exist at all, and messages
should just be delivered asynchronously, as this would lead to less
copying of messages. Currently, we fulfill the scheduler interface
instead.
Important! Trigger UL after having run DL scheduler and sent the
messages, to ensure short delay between Slot.indication, and the
response to those messages.
The previous designs seems to do:
loop {
poll_ind_queue()
if (msg)
scheduler;
pselect() on messages;
handle_message {
if slot_ind
put_ind_queue()
}
}
So basically, we artificially put a queue for slot indications in the
middle, but still handle it in the same thread(!). This for some reason
results in a big slow down if the PNF runs faster.
Now, we just do pselect(), waiting for messages. We handle the slot
indication immediately (the schedule should take some microseconds),
then return to pselect().
Use this function to implement distinct behavior for monolithic or PNF:
- monolithic: run the scheduler, as normal
- PNF: send a slot indication; the VNF will send an answer which the PNF
will handle elsewhere.
Add function declarations to make nr_ulsim compile, as NR_IF_Module.c is
part of the MAC sources. As of now, this is necessary; I think the right
way would be to take NR_IF_Module.c out of the MAC, and make this a
"glue" library that sticks L1 and L2 together, in monolithic (in VNF, in
that case, we should not link in L1, and for the PNF not the L2,
creating separate executables... Rome was not built in one day).
We are required to store multiple PUCCH (e.g., for multiple UEs).
However, nr_fill_pucch() automatically freed the previously active PUCCH
structure. Of course(!) this does not make sense.
However, the problem is that init_nr_transport() allocated only one
PUCCH structure. Also this does not make sense to me. The problem was
that slot_ahead was 0, which logically cannot happen in this function
(because slot_ahead is illogical, we must give the radio some time). The
reason is that it uses if_inst->sl_ahead, which, prior to this commit,
was not set.
The init_nr_transport() function is called normally from main() through
init_eNB_afterRU(). In this case, we initialize if_inst->sl_ahead to
what is set in main(). In the PNF case, however, we call
init_nr_transport() from the PNF thread, when receiving the start
request. In the main(), we wait for this to happen, before setting
if_inst->sl_ahead. Hence, initializing this variable (which we should
safely be able to do) before solves the problem.
The radio gives the clock. Therefore, we do not need to keep the time
separately in the PNF.
Also, the VNF should not run the ru_thread, as this is an L1 thread.
This is to align with monolithic/FAPI, where we get slot indications in
each slot. In the nFAPI, this is replaced by timing information, such
that the VNF ticks automatically, and gets indications from the PNF on
early/late messages. The timing information handling is not implemented
in the present nFAPI implementation. Instead, we adopt the FAPI behavior
and send timing information in every slot, abusing it to be a slot
indication.
There is no need to add the PNF in nr_start_resp_cb(), because it has
already been done in nr_param_resp_cb(). This might be wrong, because 4G
does it only in nr_start_resp_cb(), but for the moment, since it is also
used in nr_param_resp_cb(), avoid the double add.
The same function is called in nr_start_resp_cb(), why there and not here?
At reception of nr_param_response, we receive the remote PNF information
on PNF/VNF. We have to store it.
In nr_start_response, it might not be correct and/or can be taken out.
The next commit will remove it.
Prior to this this commit, the structure NR_UE_info_t is 4881632 bytes.
On some machines, e.g., one of the CI systems, this can lead to realtime
problems when doing the initial calloc() of the data structure.
Commit e586efb29d ("Enable 32 HARQ
Processes in DL, UL") introduced the real-time problems by increasing
the number of HARQ processes.
The reason NR_UE_info_t is that big is that it contains buffers for DL
HARQ data that might need to be retransmitted (the L1 is stateless, it
cannot store this for the L2). To reduce the size, dynamically allocate
the DL HARQ buffers when they are needed. This also reduces the size of
NR_UE_info_t to 15840 bytes.
Encapsulate transportBlock in byte_array and accessory functions, and
allocate in big chunks of multiple of ~32kB.
Reported-By: Cedric Roux <cedric.roux@eurecom.fr>
Fixes: e586efb29dCloses: #875
ran_func_kpm.c uses cp_ba_to_str(), but it is not defined in OAI. Define
it so we can link properly, instead of relying on FlexRIC (where it was
defined).
Introduce function cu_update_f1_ue_data() which does remove+add under
one lock. This fixes a concurrency bug. For instance, during
reestablishment, the CU needs to update the DU UE ID under which the UE
is identified in the DU. Previously, the CU would remove, then add the
DU UE ID info. At the same time, the PDCP thread might look up the
information. This can lead to asserts. Use the above addmod
functionality to do the remove-add under a mutex.
Modify test to verify this works as expected.
Robert Schmidt