action #174634
open[BCI] Have an improved paper-trail of FIPS tests showing commands executed and the resulting output
0%
Description
In order to have something to show as proof of vendor affirmation having been done, we would need similar level of output than openQA perl tests, including exact commands being executed and the output that is gotten.
Currently in the BCI FIPS tests like https://openqa.suse.de/tests/16219210#external even the "External results" or the uploaded logs (like https://openqa.suse.de/tests/16219210/logfile?filename=bci_test_podman-tox_fips.txt) only show the results PASSED/FAILED, but the python test_fips.py does not show the commands being executed and the results. On other products we get the output as well https://openqa.suse.de/tests/15893425#step/gnutls_base_check/22. We could possibly construct "proof" from showing the code at the git date point and assuring that was executed when it passed, but more paper trail would be much better.
Acceptance Criteria¶
- For all of FIPS BCI testing, at least when run in the vendor affirmation job group (see ticket #174631, special variable possible if normal runs in other job groups are wanted to be kept as is) have commands executed and the output of the commands saved as part of the test execution, to have a paper trail of what exactly was tested attached to each test run.