Project

General

Profile

action #31954

coordination #31972: [qe-core][opensuse][functional][epic][userspace][medium]Harmonize openSUSE/SLE test module schedule

[functional][y][medium]harmonize updating handling in openSUSE/SLE schedule

Added by okurz almost 4 years ago. Updated 3 months ago.

Status:
Resolved
Priority:
Low
Assignee:
Category:
Enhancement to existing tests
Target version:
SUSE QA - Milestone 19
Start date:
2018-02-19
Due date:
% Done:

0%

Estimated time:
8.00 h
Difficulty:
medium

Description

Motivation

The test schedule for applying updates during the functional validation tests differ among openSUSE and SLE. This should be harmonized.

Acceptance criteria

  • AC1: The same schedule is applied for both openSUSE/SLE or differences are explained with good comments

Suggestions

SLE has:

    if (gnomestep_is_applicable()) {
        loadtest "update/updates_packagekit_gpk" unless is_staging;
        loadtest "x11/gnome_control_center";
        loadtest "x11/gnome_terminal";
        loadtest "x11/gedit";
    }

in load_x11tests, and

    if (!gnomestep_is_applicable()) {
        loadtest "update/zypper_up";
    }

in load_consoletests whereas openSUSE has a helper function load_system_update_tests including both above parts at one common location which I think makes more sense. With https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/4432/files this deviation becomes more apparent and could be solved after that.

  • Take a look on git log who has added individual parts which introduce different behavior
  • Test on different scenarios if we can use same approach for all of them (which may different from current)
  • Leaving some scheduling parts as they are with comment in the code why it is so

Checklist

  • Refcator code
  • Apply same schedule for all distribution with least possible number of exceptions
  • All not obvious exceptions are explained with the comment

Related issues

Related to openQA Tests - action #37979: [SLE] test fails in updates_packagekit_gpk - gpk-update-viewer doesn't startResolved2018-06-28

Related to openQA Tests - action #38393: [functional][u] updates_packagekit_kde must not be scheduled before zypper_clear_reposResolved2018-07-132018-08-14

History

#1 Updated by okurz over 3 years ago

  • Subject changed from [functional]harmonize updating handling in openSUSE/SLE schedule to [functional][medium]harmonize updating handling in openSUSE/SLE schedule
  • Due date changed from 2018-03-27 to 2018-04-10
  • Status changed from In Progress to Workable
  • Assignee deleted (okurz)

https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/4432 merged. Now the next step could be conducted to check for a common way to handle updates in both openSUSE+SLE cases.

#2 Updated by cwh over 3 years ago

  • Difficulty set to medium

#3 Updated by okurz over 3 years ago

  • Subject changed from [functional][medium]harmonize updating handling in openSUSE/SLE schedule to [functional][u][medium]harmonize updating handling in openSUSE/SLE schedule
  • Due date changed from 2018-04-10 to 2018-04-24

not enough capacity in S14, moving, sigh

#4 Updated by okurz over 3 years ago

  • Due date changed from 2018-04-24 to 2018-06-05
  • Priority changed from Normal to Low
  • Target version changed from Milestone 15 to Milestone 16

not enough capacity in S15-S17.

#5 Updated by okurz over 3 years ago

  • Parent task set to #31972

#6 Updated by mgriessmeier over 3 years ago

  • Due date changed from 2018-06-05 to 2018-06-19

#7 Updated by mgriessmeier over 3 years ago

  • Subject changed from [functional][u][medium]harmonize updating handling in openSUSE/SLE schedule to [functional][y][medium]harmonize updating handling in openSUSE/SLE schedule

[y] team has more capacity

#8 Updated by JERiveraMoya over 3 years ago

  • Assignee set to JERiveraMoya

#9 Updated by JERiveraMoya over 3 years ago

  • Status changed from Workable to Feedback

#10 Updated by okurz over 3 years ago

  • Target version changed from Milestone 16 to Milestone 17

M17 by now

#11 Updated by okurz over 3 years ago

  • Target version changed from Milestone 17 to Milestone 17

#12 Updated by riafarov over 3 years ago

  • Due date changed from 2018-06-19 to 2018-07-03

#13 Updated by JERiveraMoya over 3 years ago

  • Estimated time set to 5.00 h

#14 Updated by riafarov over 3 years ago

  • Related to action #37979: [SLE] test fails in updates_packagekit_gpk - gpk-update-viewer doesn't start added

#15 Updated by okurz over 3 years ago

  • Status changed from Feedback to In Progress

We had to revert the PR with https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/5308 (riafarov convinced me)

https://openqa.suse.de/tests/1790992# really looks strange. The x11 test module "updates_packagekit_gpk" definitely should not be scheduled somewhere in the middle of console tests. Please try to fix that and I suggest to test multiple scenarios without needing openQA itself but just evaluate the schedule and provide that in the PR as text output, e.g. download vars.json files from all the relevant scenarios and call
isotovideo -d _exit_after_schedule=1 in each directory where you prepared the vars.json file.

#16 Updated by riafarov over 3 years ago

  • Due date changed from 2018-07-03 to 2018-07-17

#17 Updated by riafarov over 3 years ago

  • Description updated (diff)

#18 Updated by riafarov over 3 years ago

  • Estimated time changed from 5.00 h to 8.00 h

#19 Updated by JERiveraMoya over 3 years ago

  • Status changed from In Progress to Feedback

#20 Updated by riafarov over 3 years ago

  • Due date changed from 2018-07-17 to 2018-07-31

#21 Updated by okurz over 3 years ago

  • Related to action #38393: [functional][u] updates_packagekit_kde must not be scheduled before zypper_clear_repos added

#22 Updated by okurz over 3 years ago

  • Target version changed from Milestone 17 to Milestone 18

#23 Updated by riafarov over 3 years ago

  • Due date changed from 2018-07-31 to 2018-08-14

#24 Updated by riafarov over 3 years ago

  • Status changed from Feedback to Workable
  • Assignee deleted (JERiveraMoya)

jrivera did a great job to refactor the code, so now we have problem isolated and concentrated in one place.
Remaining questions,
why do we need

loadtest "update/prepare_system_for_update_tests" if !is_sle;
loadtest "update/check_system_is_updated" if !is_sle;

How is it done in sle?

Why we have issue for sle && staging, and not for staging in general?

#25 Updated by riafarov over 3 years ago

  • Checklist item changed from to [x] Refcator code, [ ] Apply same schedule for all distribution with least possible number of exceptions, [ ] All not obvious exceptions are explained with the comment

#26 Updated by riafarov over 3 years ago

  • Due date changed from 2018-08-14 to 2018-09-25
  • Target version changed from Milestone 18 to Milestone 19

#27 Updated by riafarov about 3 years ago

  • Assignee set to riafarov

#29 Updated by riafarov about 3 years ago

  • Due date changed from 2018-09-25 to 2018-10-09

#30 Updated by okurz about 3 years ago

PR is merged, seems to work. Thanks for your work. I guess we still have the two questions https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/5826/files#r220340942 and https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/5826/files#r220341144 open so that we could cross off the next two points on the checklist?

#31 Updated by riafarov about 3 years ago

I've replied to both questions. If you feel brave, I would simply remove that change to zypper. As for the first case, problem is that next test either expects root-console or x11, which is tricky to detect. Simply adding select-console('x11') in the next one did't help, and I believe may break quite some tests too. Therefore I would not proceed with these 2 considering risks, efforts and gain we get.

#32 Updated by riafarov about 3 years ago

  • Checklist item changed from [x] Refcator code, [ ] Apply same schedule for all distribution with least possible number of exceptions, [ ] All not obvious exceptions are explained with the comment to [x] Refcator code, [x] Apply same schedule for all distribution with least possible number of exceptions, [x] All not obvious exceptions are explained with the comment

#33 Updated by riafarov about 3 years ago

  • Status changed from Feedback to Resolved

#34 Updated by okurz 3 months ago

  • Due date deleted (2018-10-09)

Also available in: Atom PDF