Project

General

Profile

Actions

action #31954

closed

coordination #31972: [qe-core][opensuse][functional][epic][userspace][medium]Harmonize openSUSE/SLE test module schedule

[functional][y][medium]harmonize updating handling in openSUSE/SLE schedule

Added by okurz about 6 years ago. Updated over 2 years ago.

Status:
Resolved
Priority:
Low
Assignee:
Category:
Enhancement to existing tests
Target version:
SUSE QA - Milestone 19
Start date:
2018-02-19
Due date:
% Done:

0%

Estimated time:
8.00 h
Difficulty:
medium

Description

Motivation

The test schedule for applying updates during the functional validation tests differ among openSUSE and SLE. This should be harmonized.

Acceptance criteria

  • AC1: The same schedule is applied for both openSUSE/SLE or differences are explained with good comments

Suggestions

SLE has:

    if (gnomestep_is_applicable()) {
        loadtest "update/updates_packagekit_gpk" unless is_staging;
        loadtest "x11/gnome_control_center";
        loadtest "x11/gnome_terminal";
        loadtest "x11/gedit";
    }

in load_x11tests, and

    if (!gnomestep_is_applicable()) {
        loadtest "update/zypper_up";
    }

in load_consoletests whereas openSUSE has a helper function load_system_update_tests including both above parts at one common location which I think makes more sense. With https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/4432/files this deviation becomes more apparent and could be solved after that.

  • Take a look on git log who has added individual parts which introduce different behavior
  • Test on different scenarios if we can use same approach for all of them (which may different from current)
  • Leaving some scheduling parts as they are with comment in the code why it is so

Checklist

  • Refcator code
  • Apply same schedule for all distribution with least possible number of exceptions
  • All not obvious exceptions are explained with the comment

Related issues 2 (0 open2 closed)

Related to openQA Tests - action #37979: [SLE] test fails in updates_packagekit_gpk - gpk-update-viewer doesn't startResolvedriafarov2018-06-28

Actions
Related to openQA Tests - action #38393: [functional][u] updates_packagekit_kde must not be scheduled before zypper_clear_reposResolveddheidler2018-07-132018-08-14

Actions
Actions #1

Updated by okurz about 6 years ago

  • Subject changed from [functional]harmonize updating handling in openSUSE/SLE schedule to [functional][medium]harmonize updating handling in openSUSE/SLE schedule
  • Due date changed from 2018-03-27 to 2018-04-10
  • Status changed from In Progress to Workable
  • Assignee deleted (okurz)

https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/4432 merged. Now the next step could be conducted to check for a common way to handle updates in both openSUSE+SLE cases.

Actions #2

Updated by cwh about 6 years ago

  • Difficulty set to medium
Actions #3

Updated by okurz about 6 years ago

  • Subject changed from [functional][medium]harmonize updating handling in openSUSE/SLE schedule to [functional][u][medium]harmonize updating handling in openSUSE/SLE schedule
  • Due date changed from 2018-04-10 to 2018-04-24

not enough capacity in S14, moving, sigh

Actions #4

Updated by okurz about 6 years ago

  • Due date changed from 2018-04-24 to 2018-06-05
  • Priority changed from Normal to Low
  • Target version changed from Milestone 15 to Milestone 16

not enough capacity in S15-S17.

Actions #5

Updated by okurz almost 6 years ago

  • Parent task set to #31972
Actions #6

Updated by mgriessmeier almost 6 years ago

  • Due date changed from 2018-06-05 to 2018-06-19
Actions #7

Updated by mgriessmeier almost 6 years ago

  • Subject changed from [functional][u][medium]harmonize updating handling in openSUSE/SLE schedule to [functional][y][medium]harmonize updating handling in openSUSE/SLE schedule

[y] team has more capacity

Actions #8

Updated by JERiveraMoya almost 6 years ago

  • Assignee set to JERiveraMoya
Actions #9

Updated by JERiveraMoya almost 6 years ago

  • Status changed from Workable to Feedback
Actions #10

Updated by okurz almost 6 years ago

  • Target version changed from Milestone 16 to Milestone 17

M17 by now

Actions #11

Updated by okurz almost 6 years ago

  • Target version changed from Milestone 17 to Milestone 17
Actions #12

Updated by riafarov almost 6 years ago

  • Due date changed from 2018-06-19 to 2018-07-03
Actions #13

Updated by JERiveraMoya almost 6 years ago

  • Estimated time set to 5.00 h
Actions #14

Updated by riafarov almost 6 years ago

  • Related to action #37979: [SLE] test fails in updates_packagekit_gpk - gpk-update-viewer doesn't start added
Actions #15

Updated by okurz almost 6 years ago

  • Status changed from Feedback to In Progress

We had to revert the PR with https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/5308 (riafarov convinced me)

https://openqa.suse.de/tests/1790992# really looks strange. The x11 test module "updates_packagekit_gpk" definitely should not be scheduled somewhere in the middle of console tests. Please try to fix that and I suggest to test multiple scenarios without needing openQA itself but just evaluate the schedule and provide that in the PR as text output, e.g. download vars.json files from all the relevant scenarios and call
isotovideo -d _exit_after_schedule=1 in each directory where you prepared the vars.json file.

Actions #16

Updated by riafarov almost 6 years ago

  • Due date changed from 2018-07-03 to 2018-07-17
Actions #17

Updated by riafarov almost 6 years ago

  • Description updated (diff)
Actions #18

Updated by riafarov almost 6 years ago

  • Estimated time changed from 5.00 h to 8.00 h
Actions #19

Updated by JERiveraMoya almost 6 years ago

  • Status changed from In Progress to Feedback
Actions #20

Updated by riafarov almost 6 years ago

  • Due date changed from 2018-07-17 to 2018-07-31
Actions #21

Updated by okurz almost 6 years ago

  • Related to action #38393: [functional][u] updates_packagekit_kde must not be scheduled before zypper_clear_repos added
Actions #22

Updated by okurz almost 6 years ago

  • Target version changed from Milestone 17 to Milestone 18
Actions #23

Updated by riafarov almost 6 years ago

  • Due date changed from 2018-07-31 to 2018-08-14
Actions #24

Updated by riafarov almost 6 years ago

  • Status changed from Feedback to Workable
  • Assignee deleted (JERiveraMoya)

jrivera did a great job to refactor the code, so now we have problem isolated and concentrated in one place.
Remaining questions,
why do we need

loadtest "update/prepare_system_for_update_tests" if !is_sle;
loadtest "update/check_system_is_updated" if !is_sle;

How is it done in sle?

Why we have issue for sle && staging, and not for staging in general?

Actions #25

Updated by riafarov almost 6 years ago

  • Checklist item changed from to [x] Refcator code, [ ] Apply same schedule for all distribution with least possible number of exceptions, [ ] All not obvious exceptions are explained with the comment
Actions #26

Updated by riafarov almost 6 years ago

  • Due date changed from 2018-08-14 to 2018-09-25
  • Target version changed from Milestone 18 to Milestone 19
Actions #27

Updated by riafarov over 5 years ago

  • Assignee set to riafarov
Actions #29

Updated by riafarov over 5 years ago

  • Due date changed from 2018-09-25 to 2018-10-09
Actions #30

Updated by okurz over 5 years ago

PR is merged, seems to work. Thanks for your work. I guess we still have the two questions https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/5826/files#r220340942 and https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/5826/files#r220341144 open so that we could cross off the next two points on the checklist?

Actions #31

Updated by riafarov over 5 years ago

I've replied to both questions. If you feel brave, I would simply remove that change to zypper. As for the first case, problem is that next test either expects root-console or x11, which is tricky to detect. Simply adding select-console('x11') in the next one did't help, and I believe may break quite some tests too. Therefore I would not proceed with these 2 considering risks, efforts and gain we get.

Actions #32

Updated by riafarov over 5 years ago

  • Checklist item changed from [x] Refcator code, [ ] Apply same schedule for all distribution with least possible number of exceptions, [ ] All not obvious exceptions are explained with the comment to [x] Refcator code, [x] Apply same schedule for all distribution with least possible number of exceptions, [x] All not obvious exceptions are explained with the comment
Actions #33

Updated by riafarov over 5 years ago

  • Status changed from Feedback to Resolved
Actions #34

Updated by okurz over 2 years ago

  • Due date deleted (2018-10-09)
Actions

Also available in: Atom PDF