action #56132: [functional][y][timeboxed:6h] Setup inter-machine dependencies for SLE 15 for the scenarios where it makes sense - openQA Tests - openSUSE Project Management Tool

Custom queries

All open Feature tests
openQA Infrastructure Project
openqa-review - Closed tickets last updated by openqa-review, last 30 days
QA roadmap long-term
QA SLE functional
QA SLE Functional - closed in last 14 days
QA SLE Functional - High, need to be refined
QA SLE Functional - over cycle time median
QA SLE u
QA SLE y
QA tools (tag not necessary in openQA and subprojects)
QA tools tag (tag not necessary in openQA and subprojects; excluding tickets in "Ready" version as they are already on the backlog)
QAC - Backlog
QAM
QE tools team - backlog (ready issues)
QE tools team - backlog (w/o infra)
QE tools team - backlog SLA high
QE tools team - backlog SLA immediate
QE tools team - backlog SLA no immediate/urgent in feedback/blocked
QE tools team - backlog SLA normal
QE tools team - backlog SLA urgent
QE tools team - backlog SLO high
QE tools team - backlog SLO normal
QE tools team - backlog SLO urgent
QE tools team - backlog, high-level view (epics and higher)
QE tools team - backlog, non-reactive work, needs parent
QE tools team - backlog, top-level view (all sagas)
QE tools team - closed within last 14 days
QE tools team - closed within last 60 days
QE tools team - closed yesterday
QE Tools Team - Collaborative Session
QE tools team - due date forecast
QE tools team - exceeding due-date
QE tools team - infrastructure backlog
QE tools team - next - sorted by update time
QE tools team - next issues
QE tools team - non-estimated (unblocked) issues (w/o infra)
QE tools team - ready issues - Workable
QE tools team - ready, not assigned/blocked/low
QE tools team - update forecast
QE tools team - updated by priority
QE tools team - what members of the team are working on - Feedback (not-low)
QE Tools Team Backlog By Assignee
SLE15 Migration Open Tickets
SLE15 SP1 Migration Open Tickets
SLE15SP3 Migration open ticket
SLE15SP3 Security open ticket
Tools Team Retrospective
Tools Team Retrospective (not estimated or assigned)

Actions

Copy link

action #56132

closed

coordination #43769: [functional][y][epic] Use inter-machine dependencies to ease our review

[functional][y][timeboxed:6h] Setup inter-machine dependencies for SLE 15 for the scenarios where it makes sense

Added by riafarov over 4 years ago. Updated over 4 years ago.

Status:

Resolved

Priority:

Normal

Assignee:

JRivrain

Category:

Spike/Research

Target version:

SUSE QA - Milestone 28

Start date:

2019-09-24

Due date:

2019-10-22

% Done:

100%

Estimated time:

(Total: 2.00 h)

Difficulty:

Description

Motivation¶

See motivation in the parent ticket. In short, if test fails even on 64-bit, for some scenarios it might be waste of time to test it in the other environments.

Acceptance criteria¶

List of scenarios, where we can avoid unnecessary test runs is identified

Here we have description of the test suites.
https://gitlab.suse.de/riafarov/qa-sle-functional-y/blob/master/SLES_Integration_Level_Testplan.md

Suggested scope: SLE 15 only.

Subtasks 1 (0 open — 1 closed)

action #57311: [functional][y] Make machine inter-machine dependencies for almost identical SUTs running on same hardware

Resolved

oorlov

2019-09-24

2019-10-22

Actions

History
Notes
Property changes

Actions

Copy link

Updated by riafarov over 4 years ago

Subject changed from [functional][y] Setup inter-machine dependencies for SLE 15 for the scenarois where it makes sense to [functional][y] Setup inter-machine dependencies for SLE 15 for the scenarios where it makes sense

Actions

Copy link

Updated by riafarov over 4 years ago

Subject changed from [functional][y] Setup inter-machine dependencies for SLE 15 for the scenarios where it makes sense to [functional][y][timeboxed:6h] Setup inter-machine dependencies for SLE 15 for the scenarios where it makes sense
Description updated (diff)
Status changed from New to Workable

Actions

Copy link

Updated by JRivrain over 4 years ago

Assignee set to JRivrain

Actions

Copy link

Updated by JRivrain over 4 years ago

Status changed from Workable to Feedback

I think the whole idea of this ticket is debatable. It it is very hard to tell where it is relevant to set inter-machines dependencies, because we tend to test things differently on different machines, and the result of a same exact test may differ anyway, for example, here we can get a soft-failure on x86 but not on aarch64 for the same test.

If the goal is to lose less time:

It is a bad idea to prevent tests from running in parallel.
In review, dependencies may force us to re-launch the failing parent and wait for its execution, even in cases where children might have worked. We would lose more time than we'd gain.

This being said, it may be interesting for limiting power consumption. But then, it would make sense to make all other architectures to depend on x86_64, and it should be done globally rather than per-test.

What makes sense IMHO is to make dependencies only for almost identical SUTs, running on the same hardware, to avoid useless concurrent runs. It leaves us with few cases:

USBinstall@uefi                          < USBinstall:64bit
btrfs_libstorage-ng@64bit-ipmi           < btrfs_libstorage-ng:64bit
btrfs_libstorage-ng@s390x-kvm-sle15      < btrfs_libstorage-ng:s390x-kvm-sle12 (currently, default for s390)
crypt_no_lvm@uefi                        < crypt_no_lvm:64bit
cryptlvm_uefi@uefi                       < cryptlvm_uefi:64bit
btrfs_libstorage-ng@s390x-zVM*           < btrfs_libstorage-ng@s390x-zVM-ctc
ext4_yast@svirt-xen*                     < ext4_yast:64bit
ext4_yast@uefi                           < ext4_yast:64bit
lvm+RAID1@*                              < lvm+RAID1:64bit
mediacheck@*                             < mediacheck:64bit
minimal+base_yast@svirt*                 < minimal+base_yast:64bit
skip_registration@s390x*                 < skip_registration@s390x-zVM-ctc
skip_registration@svirt*                 < skip_registration:64bit
skip_registration@uefi                   < skip_registration:64bit
xfs@svirt*                               < xfs:64bit

Actions

Copy link