QA » openQA Project » openQA Tests » qe-yam

Triggered the jobs with pvm_hmc machine, will check the results later.
https://openqa.suse.de/tests/overview?distri=sle&version=15-SP6&build=73.1&groupid=251

zoecao wrote in #note-5:

Triggered the jobs with pvm_hmc machine, will check the results later.
https://openqa.suse.de/tests/overview?distri=sle&version=15-SP6&build=73.1&groupid=251

I think we need much more verification runs of different test suite to consider it.
It would be better that you provide the verification using a loop changing the name of the test suite for each iteration with a post-fixed number, so it is more clear for reviewer to see how many you run it with each test suite without navigating inside and kind of ensure that you did all of them with the same commit.

JERiveraMoya wrote in #note-6:

zoecao wrote in #note-5:

Triggered the jobs with pvm_hmc machine, will check the results later.
https://openqa.suse.de/tests/overview?distri=sle&version=15-SP6&build=73.1&groupid=251

I think we need much more verification runs of different test suite to consider it.
It would be better that you provide the verification using a loop changing the name of the test suite for each iteration with a post-fixed number, so it is more clear for reviewer to see how many you run it with each test suite without navigating inside and kind of ensure that you did all of them with the same commit.

Yes, it's better to run more testsuites numbers then re-run one testsuite for several times. I added some more testsuites and triggered the jobs:
https://openqa.suse.de/tests/overview?distri=sle&version=15-SP6&build=73.1&groupid=251

zoecao wrote in #note-7:

JERiveraMoya wrote in #note-6:

zoecao wrote in #note-5:

Triggered the jobs with pvm_hmc machine, will check the results later.
https://openqa.suse.de/tests/overview?distri=sle&version=15-SP6&build=73.1&groupid=251

I think we need much more verification runs of different test suite to consider it.
It would be better that you provide the verification using a loop changing the name of the test suite for each iteration with a post-fixed number, so it is more clear for reviewer to see how many you run it with each test suite without navigating inside and kind of ensure that you did all of them with the same commit.

Yes, it's better to run more testsuites numbers then re-run one testsuite for several times. I added some more testsuites and triggered the jobs:
https://openqa.suse.de/tests/overview?distri=sle&version=15-SP6&build=73.1&groupid=251

There is a way to run in a loop and change the test suite name, so we can be sure you run all with the same commit, without visually checking it, something like:
for i in {1..10} ; do openqa-clone-custom-git-refspec https://github.com/<your-user>/os-autoinst-distri-opensuse/tree/<your-branch> https://<openqa-instance>/tests/<job-id> TEST=<test-name>_$i _SKIP_POST_FAIL_HOOKS=1 ; done

This is the results of using 'pvm-hmc':
https://openqa.suse.de/tests/overview?distri=sle&version=15-SP6&build=73.1&groupid=251
In my opine, both 'pvm-hmc' and 'spvm' are stable now.
Some of the failures in migration daily group (using 'spvm') are caused by incorrect settings, I'll submit a MR to fix them, these failures also happened when using 'pvm-hmc', I already correct the settings in my VRs.

JERiveraMoya wrote in #note-10:

Adding 1 more day to this spike ticket,
It would be interesting to know if this also happens with hmc: https://openqa.suse.de/tests/13959269#step/bootloader_start/22

This issue is caused by incorrect setting of testsuite, not related with spvm machine. I submit a MR and verified with hmc machine:
MR: https://gitlab.suse.de/qe-yam/openqa-job-groups/-/merge_requests/143 (please help to review)
VRs on hmc: https://openqa.suse.de/tests/overview?distri=sle&version=15-SP6&build=73.1&groupid=251
The MR is to fix the settings issues, not to switch spvm to pvm-hmc.
And I believe this MR could fix the issues on spvm in daily group, because when I reviewing RC1 milestone build, I met the same errors with https://openqa.suse.de/tests/13959269#step/bootloader_start/22, I had submitted MR and re-triggered the milestone build, it worked fine (milestone is also using spvm), so I think the same fix would work for daily group jobs too.