action #65375
closed
[sle][security][sle15sp2][FIPS] Workaround s390x test suspend after test fail
Added by bchou about 4 years ago.
Updated about 4 years ago.
Category:
Enhancement to existing tests
Description
There are some test cases does not run while test fail in s390x arch. The problem will not be reproduced in x86_64. Compare some test case in s390x, there are some cases can be continued. Double-check the settings and test code. I think we can do some enhancement for them to add the test_flag{} to do the workaround.
Refer to doc:
http://open.qa/docs/#_how_to_write_tests
fatal: The whole test suite is aborted if the test module fails. The overall state is set to failed.
ignore_failure: If this module fails, it will not affect the overall result at all.
milestone: After this test succeeds, update the 'lastgood' snapshot.
no_rollback: Don’t roll back to the 'lastgood' snapshot if the test module fails.
always_rollback: Roll back to the 'lastgood' snapshot even if test was successful.
We could add the fatal flag to do the workaround.
There are some test cases does not run while test fail in s390x arch. The problem will not be reproduced in x86_64. Compare some test case in s390x, there are some cases can be continued.
i.g.,
https://openqa.nue.suse.com/tests/4064666#step/test_1_to_1_events/1
https://openqa.nue.suse.com/tests/4065904#
Double-check the settings and test code. There is no any difference in settings.
I think we can do some enhancement for them to use the test_flag{} doing the workaround.
Refer to doc:
http://open.qa/docs/#_how_to_write_tests
fatal: The whole test suite is aborted if the test module fails. The overall state is set to failed.
ignore_failure: If this module fails, it will not affect the overall result at all.
milestone: After this test succeeds, update the 'lastgood' snapshot.
no_rollback: Don’t roll back to the 'lastgood' snapshot if the test module fails.
always_rollback: Roll back to the 'lastgood' snapshot even if test was successful.
We could add the fatal flag to do the workaround.
There are lots of cases are impacted as follows in s390x.
1. fips_env_mode_tests_crypt_core
openssl_fips_alglistdirmngr_daemonsshdssh_pubkeyssh_cleanup
2. fips_env_mode_tests_crypt_tool
gpgclamavopenvswitch_ssl
3. fips_env_mode_tests_crypt_web
4. fips_env_mode_tests_crypt_x11_s390x
- Status changed from New to Resolved
- Status changed from Resolved to In Progress
On s390x are snapshots not supported, because the backend is not qemu.
With the PR you just forced the test to continue, but some tests, as you can see on your verification runs, continue on the previously broken test.
This just makes the s390x tests run longer without any result.
Please remove milestone => 1, only fatal => 0 helped on s390x
- % Done changed from 0 to 50
dzedro wrote:
On s390x are snapshots not supported, because the backend is not qemu.
With the PR you just forced the test to continue, but some tests, as you can see on your verification runs, continue on the previously broken test.
This just makes the s390x tests run longer without any result.
Please remove milestone => 1, only fatal => 0 helped on s390x
Hi dzedro.
Yes, I am preparing a PR to fix this error. milestone => 1 is incorrect and it will impact the following test under the unclear environment. Thanks a lot for your highlight.
- Status changed from In Progress to Resolved
- % Done changed from 50 to 100
Also available in: Atom
PDF