QA » openQA Project » openQA Tests

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: fips_env_mode_tests_crypt_tool_intel_ipmi@64bit-ipmi
https://openqa.suse.de/tests/11181584#step/evolution_prepare_servers/1

To prevent further reminder comments one of the following options should be followed:

1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
3. The bugref in the openQA scenario is removed or replaced, e.g. label:wontfix:boo1234

Expect the next reminder at the earliest in 28 days if nothing changes in this ticket.

This scenario looks like a waste of time from the point of test maintenance (reproducing it again running multiple times the job and I hit all those several strange issues), it fails sporadically in too many places and no other squad is using similar setup, not even QE virtualization!, probably drop it is the best option.
Slack conversation: https://suse.slack.com/archives/C02CANHLANP/p1695192194470199
wdyt?

We should not let go of the wishful thinking that IPMI backend usage would be reliable for us to use in the future, as our stakeholders request it.

In practice we are however quite used to expecting IPMI failures all over. It is a valid point though if we should do something similar ie have separate IPMI tests if for example https://openqa.suse.de/tests/12180565 (sle-15-SP6-Online-x86_64-Build20.1-sev-es-gi-guest_developing-on-host_developing-kvm) is somehow more reliable.

But files disappearing right after installation, I'm not sure if that kind of problem even can be related to the way of being run. Could we try the same MACHINE=64bit-ipmi-amd-zen3 as they are using in that test, would that be any more reliable?

There is something different at least that on qemu more packages are installed https://openqa.suse.de/tests/11175697#step/evolution_prepare_servers/6 than on our IPMI: https://openqa.suse.de/tests/11181584#step/evolution_prepare_servers/6 - maybe somehow recommended packages installation is disabled in our create_hdd_textmode_intel_ipmi? Not that it would affect this missing mkcert.sh, as I double-checked it's in the "dovecot23" package.

Anyway, we might want to add automatic soft-fails for IPMI cases to make the results more readable.

I understand better the scope now, which is bigger that the test suite to fix in the description, I understand now that ipmi is important for our scope, thanks for the clarification.
I was doing some testing to see the options. The first thing I noticed is that we are not testing in the same way than with qemu, using setting START_DIRECTLY_AFTER_TEST in the children test suite we are reusing the same VM, first the parent is executed and then the children are executed one after the other in alphabetical order, when in case of qemu we get a fresh VM after parent for each children.
This is what made me suspect about this recurrent error, https://openqa.suse.de/tests/11781313#step/consoletest_setup/33
this is most likely happening because we setup everything for each children, repeating even the console setup, then I tried to remove those setup from the 2nd and 3rd children executed and the first time was a success for not the second, there are other sporadic errors in different places. Besides I realized dropping the setup is not a safe option, because if the 1st child fails, 2nd and 3rd might succeed without having the fips setup done, leading to false positive.

Next option I thought was to duplicate the parent to basically mimic what we have in qemu, would be really really slow, one parent for each child, if it would worth it, but seems not because we hit this https://openqa.suse.de/tests/12365064#step/fips_setup/65 very often as well in the first child we run.

What helped was to use that worker as you mentioned to at least run the parent https://gitlab.suse.de/qe-security/osd-sle15-security/-/merge_requests/181
But if this area is owned by virtualization, we should try to provide them simple failure that they could fix one by one, otherwise is really messy.

In QU we also have issues with ipmi, ie: https://openqa.suse.de/tests/12426812
in product also don't even boot: https://openqa.suse.de/tests/12430609#step/boot_from_pxe/13

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: create_hdd_textmode_intel_ipmi@64bit-ipmi
https://openqa.suse.de/tests/12505996#step/boot_from_pxe/1

To prevent further reminder comments one of the following options should be followed:

1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
3. The bugref in the openQA scenario is removed or replaced, e.g. label:wontfix:boo1234

Expect the next reminder at the earliest in 28 days if nothing changes in this ticket.

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: create_hdd_textmode_intel_ipmi@64bit-ipmi
https://openqa.suse.de/tests/12426812#step/boot_from_pxe/1

To prevent further reminder comments one of the following options should be followed:

1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
3. The bugref in the openQA scenario is removed or replaced, e.g. label:wontfix:boo1234

Expect the next reminder at the earliest in 56 days if nothing changes in this ticket.

We haven't had ipmi results for a while now. FIPS itself would be testable now.