Project

General

Profile

action #42671

[opensuse][functional][u] test fails in vlc - needle vlc-done-playing does not match

Added by mloviska over 1 year ago. Updated over 1 year ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
Bugs in existing tests
Target version:
SUSE QA tests - Milestone 22
Start date:
2018-10-18
Due date:
% Done:

0%

Estimated time:
Difficulty:
Duration:

Description

Observation

openQA test in scenario opensuse-15.1-NET-x86_64-xfce@64bit-2G fails in
vlc

Needle vlc-done-playing does not match, because vlc window is shifted. … or because the video never started. We should have a bug for that -> boo#1102838 which is assigned to … okurz because we need to crosscheck better in tests what is happening.

Reproducible

Fails since (at least) Build 319.3 (current job)

Expected result

Last good: 319.2 (or more recent)

Acceptance criteria

  • AC1: The VLC test module runs stable (statistics) being able to play the video or reference boo#1102838 when it cannot then closes VLC

Suggestions

  • Read our story from #37713 when we tried to catch the situation that the pause button shows up including the duration of the video 0:23 but the video never actually starts to play. The problem is that in this situation VLC does not react on a click on the "close" icon
  • Add back a needle to detect "pause button and 0:23 shown", e.g. as in https://openqa.opensuse.org/tests/835278/file/video.ogv#t=318.29,318.33
  • If the above detected record_soft_failure pointing to boo#1102838 and make sure VLC can be closed, if not by click then "xkill" or similar

Further details

Always latest result in this scenario: latest

video.jpg (110 KB) video.jpg screenshot of VLC stuck with pause button and duration 0:23 shown okurz, 2019-01-22 20:19
7445

Related issues

Related to openQA Tests - action #42674: [opensuse][functional][u][sporadic] test fails in vlc - cannot open videoResolved2018-10-18

Related to openQA Tests - action #42773: [functional][u] extend VLC test to check correct renderingNew2018-10-22

Related to openQA Tests - action #37713: [functional][u] test fails in vlc - sometimes fails to close vlcResolved2018-06-232018-10-23

Related to openQA Tests - action #47132: [opensuse][u][sporadic] test fails in vlc - vlc does not closeResolved2019-02-05

Blocks openQA Tests - action #39926: [functional][u][sporadic][virtio] test fails in vlc - vlc is started twice, needle target_match_vlc seems wrongRejected2018-08-17

History

#1 Updated by okurz over 1 year ago

  • Related to action #42674: [opensuse][functional][u][sporadic] test fails in vlc - cannot open video added

#2 Updated by okurz over 1 year ago

  • Target version set to Milestone 22

#3 Updated by okurz over 1 year ago

  • Description updated (diff)
  • Status changed from New to Feedback
  • Assignee set to okurz

well, the original job is gone and the latest one or predecessors do not really reveal what you have seen originally. E.g. https://openqa.opensuse.org/tests/833129#step/vlc/26 fails (again) because of the check for the completely typed URL. I don't understand why people create another bunch of needles, not cleaning up old ones but not creating new ones with lower match level. I did that now again same as I did for other window managers already. Also I deleted needles that did not match for the past 3 months anywhere over https://openqa.opensuse.org/admin/needles . Also adapted description to describe what I see in current jobs still, that the playing of the video actually never starts.

#4 Updated by okurz over 1 year ago

  • Related to action #42773: [functional][u] extend VLC test to check correct rendering added

#5 Updated by okurz over 1 year ago

  • Related to action #37713: [functional][u] test fails in vlc - sometimes fails to close vlc added

#6 Updated by okurz over 1 year ago

  • File video.jpg video.jpg added
  • Description updated (diff)
  • Status changed from Feedback to In Progress
  • Priority changed from Normal to High
7445

adapted description and attached screenshot

#8 Updated by okurz over 1 year ago

one test was fine but did not trigger the actual workaround (I have not even created the needle to detect the bug situation).

Retriggered a custom scenario for statistical investigation:

for i in {001..001}; do openqa-clone-job --within-instance https://openqa.opensuse.org --skip-chained-deps 835553 _GROUP="Development Tumbleweed" TEST="update_Leap_42.3_gnome@okurz__os-autoinst-distri-opensuse_fix_vlc_$i" EXCLUDE_MODULES=keymap_or_locale,textinfo,hostname,xorg_vt,ncurses,yast2_lan,curl_https,salt,glibc_sanity,zypper_in,yast2_i,yast2_bootloader,vim,firewall_enabled,sshd,ssh_cleanup,sshfs,mtab,orphaned_packages_check,user_gui_login,desktop_runner,xterm,sshxterm,gnome_control_center,gnome_tweak_tool,gnome_terminal,gedit,firefox,firefox_audio,thunderbird,chromium,ooffice,oomath,oocalc,glxgears,nautilus,gnome_music,evolution,desktop_mainmenu,inkscape,gimp,hexchat,reboot_gnome,cleanup_before_shutdown,shutdown; done

-> Created job #835900: opensuse-Tumbleweed-DVD-x86_64-Buildokurz_os-autoinst-distri-opensuse_6614-update_Leap_42.3_gnome@okurz__os-autoinst-distri-opensuse_fix_vlc@64bit -> https://openqa.opensuse.org/t835900

passed. Let's trigger more

for i in {002..040}; do openqa-clone-job --within-instance https://openqa.opensuse.org --skip-chained-deps 835553 _GROUP="Development Tumbleweed" TEST="update_Leap_42.3_gnome@okurz__os-autoinst-distri-opensuse_fix_vlc_$i" EXCLUDE_MODULES=keymap_or_locale,textinfo,hostname,xorg_vt,ncurses,yast2_lan,curl_https,salt,glibc_sanity,zypper_in,yast2_i,yast2_bootloader,vim,firewall_enabled,sshd,ssh_cleanup,sshfs,mtab,orphaned_packages_check,user_gui_login,desktop_runner,xterm,sshxterm,gnome_control_center,gnome_tweak_tool,gnome_terminal,gedit,firefox,firefox_audio,thunderbird,chromium,ooffice,oomath,oocalc,glxgears,nautilus,gnome_music,evolution,desktop_mainmenu,inkscape,gimp,hexchat,reboot_gnome,cleanup_before_shutdown,shutdown; done

-> https://openqa.opensuse.org/tests/overview?build=okurz%2Fos-autoinst-distri-opensuse%236614&version=Tumbleweed&groupid=38&distri=opensuse

One failed in force_scheduled_tasks for what is another ticket, one incompleted, reported as #46556, others passed. So ignoring other failures we can see 40/40 passed meaning that I can not reproduce the error. This might be due to excluding too many modules which might have an impact so I will include them back and let the tests run again.

for i in {001..040}; do openqa-clone-job --within-instance https://openqa.opensuse.org --skip-chained-deps 835553 _GROUP="Development Tumbleweed" TEST="update_Leap_42.3_gnome@okurz__os-autoinst-distri-opensuse_fix_vlc_$i" BUILD=okurz/os-autoinst-distri-opensuse#6614_all_modules EXCLUDE_MODULES=reboot_gnome,cleanup_before_shutdown,shutdown; done

-> https://openqa.opensuse.org/tests/overview?distri=opensuse&version=Tumbleweed&build=okurz%2Fos-autoinst-distri-opensuse%236614_all_modules&groupid=38

#9 Updated by okurz over 1 year ago

this will most likely take until tomorrow – or fail fast early ;)

#10 Updated by okurz over 1 year ago

Unexpectedly 1/40 failed. I guess I need to trigger some more to have good statistics?

waiting … (same link)

EDIT: So 3/80 failed, that is 3.8%. Not much but ok for reproducability I guess.

#11 Updated by okurz over 1 year ago

ok, I think I will play it safe and create an explicit new needle tag and created a needle "vlc-stuck-never-played-boo1102838-20190125" for that already. Updated code, retriggering …

for i in {001..080}; do openqa-clone-job --within-instance https://openqa.opensuse.org --skip-chained-deps 835553 _GROUP="Development Tumbleweed" TEST="update_Leap_42.3_gnome@okurz__os-autoinst-distri-opensuse_fix_vlc_$i" BUILD=okurz/os-autoinst-distri-opensuse#6614_all_modules_with_workaround EXCLUDE_MODULES=reboot_gnome,cleanup_before_shutdown,shutdown; done

-> https://openqa.opensuse.org/tests/overview?build=okurz%2Fos-autoinst-distri-opensuse%236614_all_modules_with_workaround&version=Tumbleweed&groupid=38&distri=opensuse

stupid me. This can not work because still we need to pass first the step of "done-playing". Well, let's look for the alternative explicitly but earlier then?

adapted, pushed, retriggered:

for i in {001..080}; do openqa-clone-job --within-instance https://openqa.opensuse.org --skip-chained-deps 838337 _GROUP="Development Tumbleweed" TEST="update_Leap_42.3_gnome@okurz_os-autoinst-distri-opensuse_fix_vlc_$i" BUILD=okurz/os-autoinst-distri-opensuse#6614_all_modules_with_workaround2 EXCLUDE_MODULES=reboot_gnome,cleanup_before_shutdown,shutdown; done

-> https://openqa.opensuse.org/tests/overview?distri=opensuse&version=Tumbleweed&build=okurz%2Fos-autoinst-distri-opensuse%236614_all_modules_with_workaround2&groupid=38

Made a mistake by using a "latest" job but not specifying CASEDIR again

https://github.com/okurz/os-autoinst-distri-opensuse.git#fix/vlc

for i in {001..100}; do openqa-clone-job --within-instance https://openqa.opensuse.org --skip-chained-deps 838337 _GROUP="Development Tumbleweed" TEST="update_Leap_42.3_gnome@okurz_os-autoinst-distri-opensuse_fix_vlc_$i" BUILD=okurz/os-autoinst-distri-opensuse#6614_all_modules_with_workaround2 EXCLUDE_MODULES=reboot_gnome,cleanup_before_shutdown,shutdown CASEDIR=https://github.com/okurz/os-autoinst-distri-opensuse.git#fix/vlc; done

-> https://openqa.opensuse.org/tests/overview?version=Tumbleweed&distri=opensuse&groupid=38&build=okurz%2Fos-autoinst-distri-opensuse%236614_all_modules_with_workaround2

All failed in zypper_ref because apparently the older snapshot (it's actually just HEAD~1) is gone already. Retriggering based on latest, job https://openqa.opensuse.org/tests/839882

for i in {001..100}; do openqa-clone-job --within-instance https://openqa.opensuse.org --skip-chained-deps 839882 _GROUP="Development Tumbleweed" TEST="update_Leap_42.3_gnome@okurz_os-autoinst-distri-opensuse_fix_vlc_$i" BUILD=okurz/os-autoinst-distri-opensuse#6614_all_modules_with_workaround2 EXCLUDE_MODULES=reboot_gnome,cleanup_before_shutdown,shutdown CASEDIR=https://github.com/okurz/os-autoinst-distri-opensuse.git#fix/vlc; done

-> https://openqa.opensuse.org/tests/overview?version=Tumbleweed&groupid=38&distri=opensuse&build=okurz%2Fos-autoinst-distri-opensuse%236614_all_modules_with_workaround2&flavor=NET

based on NET now
https://openqa.opensuse.org/tests/840423#step/vlc/27
Searching over https://openqa.opensuse.org/admin/needles for the needle tag "vlc-stuck-never-played" reveals the last match https://openqa.opensuse.org/tests/840423#step/vlc/27 with the workaround of "killall -9 vlc" triggered in https://openqa.opensuse.org/tests/840423#step/vlc/31 so the workaround is effective.

#12 Updated by okurz over 1 year ago

  • Blocks action #39926: [functional][u][sporadic][virtio] test fails in vlc - vlc is started twice, needle target_match_vlc seems wrong added

#13 Updated by okurz over 1 year ago

  • Status changed from In Progress to Feedback

#15 Updated by okurz over 1 year ago

  • Status changed from Feedback to Resolved

Adressed the usual openSUSE test reviewers additionally in [#opensuse-factory[01/02/2019 09:02:19] DimStar, maxlin, Marcus, lnussel, michel_mno: I tried to improve the stability of the openQA test module "vlc" with https://progress.opensuse.org/issues/42671 . If you see any further instabilities or random failures please let me know, e.g. in the ticket. I really assume it to be rock-solid stable now :)

#16 Updated by okurz over 1 year ago

  • Related to action #47132: [opensuse][u][sporadic] test fails in vlc - vlc does not close added

Also available in: Atom PDF