Implement generic CI step "may_fail" option

There might be various occasions where a CI step should be attempted to
be executed, despite a chance of failing.  For instance, we might want
to try executing an instable Iperf test, and still mark the CI as
passing if it fails.

This commit introduces a generic "may_fail" option that will mark any CI
test as succeeded, despite having failed.

Ideally, we would print a warning in the HTML code, but this is not
possible because each step emits its own HTML code, instead of the main
loop handling this. We can improve this in the future.
This commit is contained in:
Robert Schmidt
2025-06-17 19:21:21 +02:00
parent bb9fa2c6a0
commit 97d0b6534a
2 changed files with 23 additions and 1 deletions
+4 -1
View File
@@ -626,6 +626,7 @@ elif re.match('^TesteNB$', mode, re.IGNORECASE) or re.match('^TestUE$', mode, re
HTML.testCase_id=CiTestObj.testCase_id
CiTestObj.desc = test.findtext('desc')
always_exec = test.findtext('always_exec') in ['True', 'true', 'Yes', 'yes']
may_fail = test.findtext('may_fail') in ['True', 'true', 'Yes', 'yes']
HTML.desc=CiTestObj.desc
action = test.findtext('class')
if (CheckClassValidity(xml_class_list, action, id) == False):
@@ -639,7 +640,9 @@ elif re.match('^TesteNB$', mode, re.IGNORECASE) or re.match('^TestUE$', mode, re
break
try:
test_succeeded = ExecuteActionWithParam(action)
if not test_succeeded:
if not test_succeeded and may_fail:
logging.warning(f"test ID {test_case_id} action {action} may or may not fail, proceeding despite error")
elif not test_succeeded:
logging.error(f"test ID {test_case_id} action {action} failed ({test_succeeded}), skipping next tests")
task_set_succeeded = False
except Exception as e:
+19
View File
@@ -27,6 +27,8 @@
<htmlTabIcon>wrench</htmlTabIcon>
<TestCaseRequestedList>
000001
000011
000012
000002
000003
000004
@@ -41,6 +43,23 @@
<command_fail>yes</command_fail>
</testCase>
<testCase id="000011">
<class>Custom_Command</class>
<desc>This may fail</desc>
<may_fail>true</may_fail>
<node>localhost</node>
<command>false</command>
<command_fail>yes</command_fail>
</testCase>
<testCase id="000012">
<class>Custom_Command</class>
<desc>This is still executed</desc>
<node>localhost</node>
<command>true</command>
<command_fail>yes</command_fail>
</testCase>
<testCase id="000002">
<class>Custom_Command</class>
<desc>This should fail</desc>