Project

General

Profile

action #31048

action #33208: [sle15sp1][desktop] Integrating of fix for openQA firefox test cases issues.

[sle15][desktop] test fails in firefox_smoke due to performance issue

Added by qkzhu over 3 years ago. Updated over 3 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Bugs in existing tests
Target version:
-
Start date:
2018-01-30
Due date:
% Done:

0%

Estimated time:
Difficulty:

Description

Observation

openQA test in scenario sle-15-Installer-DVD-x86_64-desktopapps-firefox@64bit-virtio-vga fails in
firefox_smoke

We should replace type_string $site. "\n"; with type_string_slow $site. "\n";

History

#1 Updated by qkzhu over 3 years ago

  • Status changed from New to In Progress

#2 Updated by okurz over 3 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: desktopapps-firefox
https://openqa.suse.de/tests/1475990

#3 Updated by qkzhu over 3 years ago

  • Status changed from In Progress to Resolved

#4 Updated by qkzhu over 3 years ago

  • Parent task set to #33208

#5 Updated by qkzhu over 3 years ago

  • Status changed from Resolved to In Progress

subroutine firefox_check_default is not stable. We have too many needles with tag firefox_clean and this needle will be matched before the default browser window pop up.

I will improve the stability by using:

`- assert_screen [qw(firefox_default_browser firefox_trackinfo firefox_readerview_window firefox_clean)], 120;

  • check_screen 'firefox_default_browser'; if (match_has_tag('firefox_default_browser')) { wait_screen_change { assert_and_click 'firefox_default_browser_yes';`

#7 Updated by okurz over 3 years ago

As commented in https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/4645#discussion_r175716539 I don't think this is a good idea. Many people - mainly QAM - invested a lot of time to get the firefox tests working properly so we should not accept a revert lightly. Simply stated: Sorry, check_screen with default timeout is not acceptable. I do not understand yet how too many needles for firefox_clean make a problem unless a dialog on the SUT would disappear in time. However, looking at https://openqa.suse.de/admin/needles I can confirm that there are many needles for firefox_clean where many of them seem to have never matched. I will try to clean up some of them. Maybe this helps already.

I deleted many firefox_clean tests. Can you try to retrigger tests and see if this helps?

#8 Updated by qkzhu over 3 years ago

Hi Oliver,

Thanks for your advice, FF cases works fine in QAM tests however it seems more unreliable in SLED15.
I thought the reason is gdm will eat more RAM after 3.16 (according to: https://bugzilla.gnome.org/show_bug.cgi?id=747339).

After some discussion with Yifan, we agree that we should allocate more RAM to the worker of SLED15, and the situation got much better now (with QEMURAM=2048):
https://openqa.suse.de/tests/1560033

My modification to firefox_check_default is not necessary anymore, I will update my PR.

Thanks!

#9 Updated by okurz over 3 years ago

Ah, that's also a very good finding

#10 Updated by qkzhu over 3 years ago

  • Status changed from In Progress to Resolved

Also available in: Atom PDF