action #65375
closed[sle][security][sle15sp2][FIPS] Workaround s390x test suspend after test fail
100%
Description
There are some test cases does not run while test fail in s390x arch. The problem will not be reproduced in x86_64. Compare some test case in s390x, there are some cases can be continued. Double-check the settings and test code. I think we can do some enhancement for them to add the test_flag{} to do the workaround.
Refer to doc:
http://open.qa/docs/#_how_to_write_tests
fatal: The whole test suite is aborted if the test module fails. The overall state is set to failed.
ignore_failure: If this module fails, it will not affect the overall result at all.
milestone: After this test succeeds, update the 'lastgood' snapshot.
no_rollback: Don’t roll back to the 'lastgood' snapshot if the test module fails.
always_rollback: Roll back to the 'lastgood' snapshot even if test was successful.
We could add the fatal flag to do the workaround.
Updated by bchou about 4 years ago
There are some test cases does not run while test fail in s390x arch. The problem will not be reproduced in x86_64. Compare some test case in s390x, there are some cases can be continued.
i.g.,
https://openqa.nue.suse.com/tests/4064666#step/test_1_to_1_events/1
https://openqa.nue.suse.com/tests/4065904#
Double-check the settings and test code. There is no any difference in settings.
I think we can do some enhancement for them to use the test_flag{} doing the workaround.
Refer to doc:
http://open.qa/docs/#_how_to_write_tests
fatal: The whole test suite is aborted if the test module fails. The overall state is set to failed.
ignore_failure: If this module fails, it will not affect the overall result at all.
milestone: After this test succeeds, update the 'lastgood' snapshot.
no_rollback: Don’t roll back to the 'lastgood' snapshot if the test module fails.
always_rollback: Roll back to the 'lastgood' snapshot even if test was successful.
We could add the fatal flag to do the workaround.
Updated by bchou about 4 years ago
There are lots of cases are impacted as follows in s390x.
1. fips_env_mode_tests_crypt_core
openssl_fips_alglistdirmngr_daemonsshdssh_pubkeyssh_cleanup
2. fips_env_mode_tests_crypt_tool
gpgclamavopenvswitch_ssl
3. fips_env_mode_tests_crypt_web
links_httpslynx_https
4. fips_env_mode_tests_crypt_x11_s390x
firefox_nss
Updated by bchou about 4 years ago
- Status changed from New to Resolved
PR was merged.
https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/9996
Set this one as resolved.
Updated by dzedro about 4 years ago
- Status changed from Resolved to In Progress
On s390x are snapshots not supported, because the backend is not qemu.
With the PR you just forced the test to continue, but some tests, as you can see on your verification runs, continue on the previously broken test.
This just makes the s390x tests run longer without any result.
Please remove milestone => 1, only fatal => 0 helped on s390x
Updated by bchou almost 4 years ago
- % Done changed from 0 to 50
dzedro wrote:
On s390x are snapshots not supported, because the backend is not qemu.
With the PR you just forced the test to continue, but some tests, as you can see on your verification runs, continue on the previously broken test.
This just makes the s390x tests run longer without any result.
Please remove milestone => 1, only fatal => 0 helped on s390x
Hi dzedro.
Yes, I am preparing a PR to fix this error. milestone => 1 is incorrect and it will impact the following test under the unclear environment. Thanks a lot for your highlight.
Updated by bchou almost 4 years ago
- Status changed from In Progress to Resolved
- % Done changed from 50 to 100
The enhancement PR is merged.
Remove milestone testflag to avoid test not-clean snapshot
https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/10196
Thanks.