QA » openQA Project » openQA Infrastructure

Can we consider that this problem is fixed and close the ticket?

in latest build we could only find this one, not sure if related: https://openqa.suse.de/tests/7403395#step/validate_lvm_raid1/11
for now we thought it could be this: https://progress.opensuse.org/issues/100970

It is related, additional modules do not make any difference: https://openqa.suse.de/tests/7476991

I truly have no idea what happens over here. The other jobs are passing, therefore the serial line does not get reset or connection seems to be active.
With lvm+RAID1@svirt-hyperv-uefi I have noticed the same behaviour regarding serial console. When it comes to hyperv, the serial line is over TCP/IP Named pipes. While my jobs were running, I have seen both connections active as long as the jobs ran.
I have cloned the same problematic job on both hyperv servers:

• 2016
• 2019

In both cases the serial connection is active hence message attaching console,wait ...connected!, but after a while it seems like it freezes.

Thanks for taking a look, we also found this one in new build, which is a bit different because ended up in graphical system after installation, but we identified the same problem changing root console: https://openqa.suse.de/tests/7591216#step/integration_services/1

JERiveraMoya wrote:

Thanks for taking a look, we also found this one in new build, which is a bit different because ended up in graphical system after installation, but we identified the same problem changing root console: https://openqa.suse.de/tests/7591216#step/integration_services/1

HyperV just created a snapshot, not sure if that can have any effect on gnome. It does not really seem to be a problem related to the hypervisor itself.

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: lvm+RAID1@svirt-hyperv-uefi
https://openqa.suse.de/tests/7656832

To prevent further reminder comments one of the following options should be followed:

1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
3. The bugref in the openQA scenario is removed or replaced, e.g. label:wontfix:boo1234

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: lvm+RAID1@svirt-hyperv-uefi
https://openqa.suse.de/tests/7793640

To prevent further reminder comments one of the following options should be followed:

1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
3. The bugref in the openQA scenario is removed or replaced, e.g. label:wontfix:boo1234

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: lvm+RAID1@svirt-hyperv2016
https://openqa.suse.de/tests/7924467

To prevent further reminder comments one of the following options should be followed:

1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
3. The bugref in the openQA scenario is removed or replaced, e.g. label:wontfix:boo1234

As Joaquín linked this happens in the functional group too. From what I can tell it seems intermittent. I have made a build group to force the issue to appear, but I cannot reliably trigger it.

The times it happened the bootloader module warned that there was a boot parameter mismatch even though the video shows no obvious mismatch. The installation proceeded normally and then the test failed at the first assert_script_run, which happens to be at the ensure_serialdev_permissions module.

Here is the "normal" run with the failures https://openqa.suse.de/tests/latest?arch=x86_64&distri=sle&flavor=Online&machine=svirt-hyperv&test=default&version=15-SP4#next_previous

I took the time to re-trigger jobs for both hyperv machines in order to compare in latest product validation, this was the result:
https://openqa.suse.de/tests/overview?arch=&flavor=&machine=svirt-hyperv%2Csvirt-hyperv-uefi%2Csvirt-hyperv2016%2Csvirt-hyperv2016-uefi&test=&modules=&module_re=&distri=sle&version=15-SP4&build=79.1&groupid=129#

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: minimal+base_yast@svirt-hyperv
https://openqa.suse.de/tests/8010746

To prevent further reminder comments one of the following options should be followed:

1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
3. The bugref in the openQA scenario is removed or replaced, e.g. label:wontfix:boo1234

Oli, I move the ticket to the tools team, if that is not correct please reassign to the right team.

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: lvm+RAID1@svirt-hyperv-uefi
https://openqa.suse.de/tests/8162314

To prevent further reminder comments one of the following options should be followed:

1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
3. The bugref in the openQA scenario is removed or replaced, e.g. label:wontfix:boo1234

xlai wrote:

okurz wrote:

@xlai I have linked related tickets for this story. I think the problems mentioned in this ticket also boil down to the unstable solution of forwarding the serial port from within the Windows server. I recommend you look into the proposed "socat" solution we discussed lately. Assigning to you and team scope "[virtualization]" for followup.

@okurz I have the same feeling that this is related to unstable serial forwarding issue. I can have a look, but this is not the area that I am expert in. And Nan, who is responsible for hyperv in VT team, is too busy on fulfilling test and automation requirements and won't have time for this in very long time.

I thought this was tools team's scope and @jstehlik said he would further talk with you about the socat implementation. Jan, right?

Yes, the tools team can do everything ;) Well, we have to be realistic with what to expect You have 7 members in your team with your domain being virtualization including HyperV. We have currently 5 FTE + 3 part time workers with most of us hired to do software development plus hardware maintenance. We already are stretching our competences with taking over maintainership for backends that no one of us has developed. We have no or very little experience with administrating something on Windows servers. I am sure you will benefit in your team if you build up the necessary competence to solve problems related to services on the Windows host. As I already explained in #105473#note-4 I think ensuring necessary requirements from within the test automation code can have multiple benefits and would likely solve the problem and stabilize the setup. As you noted that currently one person should be responsible for HyperV I suggest to build up the competence within the team to not again run into the situation that a single person leaving a team would cause such damage as happened previously with members of the virtualization team building up the solutions that you rely upon. Of course you can decide on your own how you select priorities and plans for the individual tasks as you mentioned that Nan would be currently busy with other tasks.

If you consider other tasks, e.g. VMWare related backend implementations, less important than this task here of course we can try to free up capacity to work on HyperV related topics instead.

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: lvm+RAID1@svirt-hyperv-uefi
https://openqa.suse.de/tests/8292191

To prevent further reminder comments one of the following options should be followed:

1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
3. The bugref in the openQA scenario is removed or replaced, e.g. label:wontfix:boo1234

Expect the next reminder at the earliest in 28 days if nothing changes in this ticket.

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: minimal+base_yast@svirt-hyperv
https://openqa.suse.de/tests/8496177#step/system_prepare/1

To prevent further reminder comments one of the following options should be followed:

1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
3. The bugref in the openQA scenario is removed or replaced, e.g. label:wontfix:boo1234

Expect the next reminder at the earliest in 28 days if nothing changes in this ticket.

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: minimal+base_yast@svirt-hyperv
https://openqa.suse.de/tests/8570379#step/system_prepare/1

To prevent further reminder comments one of the following options should be followed:

1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
3. The bugref in the openQA scenario is removed or replaced, e.g. label:wontfix:boo1234

Expect the next reminder at the earliest in 28 days if nothing changes in this ticket.

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: lvm+RAID1@svirt-hyperv
https://openqa.suse.de/tests/8752318#step/validate_lvm_raid1/1

To prevent further reminder comments one of the following options should be followed:

1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
3. The bugref in the openQA scenario is removed or replaced, e.g. label:wontfix:boo1234

Expect the next reminder at the earliest in 56 days if nothing changes in this ticket.

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: minimal+base_yast@svirt-hyperv
https://openqa.suse.de/tests/9340996#step/first_boot/1

To prevent further reminder comments one of the following options should be followed:

1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
3. The bugref in the openQA scenario is removed or replaced, e.g. label:wontfix:boo1234

Expect the next reminder at the earliest in 112 days if nothing changes in this ticket.