CI: phytest-timing: detect deviation from normalized values above/below
The CI would only detect a deviation from the normalized value based on a configurable threshold, typically 1.25 (so we would tolerate a 25% increase). However, it often happens that a timing gets shorter (e.g., forgot to put the timing measurements). This commit introduces a "Deviation Threshold" (smaller than 1) to not only check that we exceed 1.0 + "Deviation Threshold" (as before), but also that are not below 1.0 - "Deviation Threshold". In other words, assuming a maximum divergence of 25%, instead of just checking if val > 1.25, we now also check if val < 0.75 (and flag error).
This commit is contained in:
+4
-2
@@ -1179,8 +1179,10 @@ class RANManagement():
|
||||
self.datalog_rt_stats=datalog_rt_stats
|
||||
#check if there is a fail => will render the test as failed
|
||||
for k in datalog_rt_stats['Data']:
|
||||
if float(datalog_rt_stats['Data'][k][3])> datalog_rt_stats['Threshold'][k]: #condition for fail : avg/ref is greater than the fixed threshold
|
||||
logging.debug('\u001B[1;30;43m datalog_rt_stats metric ' + k + '=' + datalog_rt_stats['Data'][k][3] + ' > threshold ' + str(datalog_rt_stats['Threshold'][k]) + ' \u001B[0m')
|
||||
valnorm = float(datalog_rt_stats['Data'][k][3])
|
||||
dev = datalog_rt_stats['DeviationThreshold'][k]
|
||||
if valnorm > 1.0 + dev or valnorm < 1.0 - dev: # condition for fail : avg/ref deviates by more than "deviation threshold"
|
||||
logging.debug(f'\u001B[1;30;43m normalized datalog_rt_stats metric {k}={valnorm} deviates by more than {dev}\u001B[0m')
|
||||
RealTimeProcessingIssue = True
|
||||
else:
|
||||
statMsg = 'No real time stats found in the log file\n'
|
||||
|
||||
Reference in New Issue
Block a user