action #61994

VNC console corruption on aarch64

Added by MDoucha about 1 month ago. Updated 6 days ago.

Status:ResolvedStart date:10/01/2020
Priority:LowDue date:21/01/2020
Assignee:okurz% Done:

0%

Category:-
Target version:openQA Project - Ready
Duration: 8

Description

A random problem sometimes appears on aarch64 test machines where the VM screen isn't properly cleared after boot and console output gets drawn over remnants of boot splash screen. Then the job fails because needles don't match. The problem appears less than once a week and job restart usually fixes it but it might be worth investigating further.
https://openqa.suse.de/tests/3773959#step/update_kernel/6

Screenshot_20200111_110145.png (396 KB) okurz, 11/01/2020 10:02 am

9233

History

#1 Updated by coolo about 1 month ago

How do you imagine we investigate this? I expect a qemu problem - and all such random qemu problems we filed, just got old in bugzilla.

#2 Updated by okurz about 1 month ago

  • File Screenshot_20200111_110145.png added
  • Due date set to 21/01/2020
  • Status changed from New to Feedback
  • Assignee set to okurz
  • Target version set to Current Sprint

While I agree with coolo that the fix probability is low I also think that mdoucha picked a valid approach. I prefer having actual issues reported and trying to at least handle workarounds automatically rather than repeatedly manually retriggering jobs :)

I reported https://bugzilla.opensuse.org/show_bug.cgi?id=1160707 and will create a workaround needle referencing the bug. This should be good enough to keep tests running. If the test will be closed without a fix we can replace the workaround needle with a normal needle that is forgiving enough to also accept the distorted screen as usable.

Setting to "Feedback" with due date for myself to check if the workaround needle is effective.

Created new needle "linux-login-corrupted_boot_screen-boo1160707-20200111" like this:

Screenshot_20200111_110145.png

#3 Updated by okurz about 1 month ago

  • Status changed from Feedback to Resolved
  • Target version changed from Current Sprint to Done

The needle hasn't matched anywhere yet but is used according to https://openqa.suse.de/admin/needles . As the tests did not fail again in the same step in the original scenario I will resolve this ticket. The bug mentions a potential QEMU patch to fix corruptions though: https://lists.gnu.org/archive/html/qemu-devel/2020-01/msg03545.html

#4 Updated by MDoucha 24 days ago

  • Status changed from Resolved to Feedback
  • Target version changed from Done to Ready

The bug made another appearance and the workaround needle was not used. You'll need to create another needle for SLE-15+ and add "tty1-selected" tag to the old one.
https://openqa.suse.de/tests/3850002#step/boot_ltp/11
https://openqa.suse.de/tests/3849935#step/boot_ltp/6

#5 Updated by okurz 21 days ago

  • Status changed from Feedback to Resolved

no problem to create an updated needled: tty1-selected-corrupted_boot_screen-boo1160707-20200203

#6 Updated by MDoucha 7 days ago

It looks like creating workaround needles for this issue will be a lot of work.
https://openqa.suse.de/tests/3889963#step/boot_ltp/6

#7 Updated by okurz 7 days ago

Oh, I see. I didn't know that the white circles can be covered by text as well. I assumed the animation would keep a detectable picture in foreground often enough. Ok, created "linux-login-corrupted_boot_screen-boo1160707-20200217" which should be even more resilient.

#8 Updated by MDoucha 6 days ago

okurz wrote:

Oh, I see. I didn't know that the white circles can be covered by text as well. I assumed the animation would keep a detectable picture in foreground often enough. Ok, created "linux-login-corrupted_boot_screen-boo1160707-20200217" which should be even more resilient.

If this is a VNC bug, there is no animation running in the background anymore when this corruption happens. Then animation ended. The console was cleared with solid black color but the VNC client didn't notice. THEN the text was printed on empty black screen but the VNC client rendered it into its dirty buffer.

Also available in: Atom PDF