action #25210

action #23798: [sles][functional][u][epic] Add systemd test suite execution to openQA

[sle][functional] Add test suite patched_systemd

Added by SLindoMansilla over 2 years ago. Updated 1 day ago.

Status:In ProgressStart date:12/09/2017
Priority:LowDue date:31/12/2019
Assignee:tsaupe% Done:

0%

Category:New test
Target version:QA - future
Difficulty:
Duration: 601

Description

****## User story

As a SUSE systemd developer I want to automatically test our patches for systemd on released distribution versions to have earlier feedback about potential regressions introduced by SUSE patches on architectures I do not have available for testing

Acceptance criteria

  • AC1 The test should install the patched systemd version provided by systemd department.
  • AC2 The test should install the systemd-testsuite that matches the patched systemd version.
  • AC3 The test should work for all the released version (at the moment SLE <= 12-SP3 and Leap <= 42.3)

Tasks

  • create new test module that executes the systemd test suite
  • create new test schedule (main.pm) that executes the installation of systemd - compare to gnuhealth but in textmode, reboot, install of testsuite, execution
  • add this test suite to osd
  • add this test suite to o3

Settings

Test suite name: suse_patches-systemd_testsuite

O3 42.3

BOOT_HDD_IMAGE=1
DESKTOP=textmode
HDD_1=%DISTRI%-%VERSION%-%ARCH%-%BUILD%-%DESKTOP%@%MACHINE%.qcow2
QA_HEAD_REPO=https://download.opensuse.org/repositories/home:/tsaupe:/testsuites/openSUSE_Leap_42.3
START_AFTER_TEST=create_hdd_textmode
SYSTEMD_TESTSUITE=1

OSD 12-SP2

BOOT_HDD_IMAGE=1
DESKTOP=textmode
HDD_1=%DISTRI%-%VERSION%-%ARCH%-%BUILD%-%DESKTOP%@%MACHINE%.qcow2
QA_HEAD_REPO=http://download.suse.de/ibs/QA:/Head/SLE-12-SP2/
START_AFTER_TEST=create_hdd_textmode
SYSTEMD_TESTSUITE=1

OSD 12-SP3

BOOT_HDD_IMAGE=1
DESKTOP=textmode
HDD_1=%DISTRI%-%VERSION%-%ARCH%-%BUILD%-%DESKTOP%@%MACHINE%.qcow2
QA_HEAD_REPO=http://download.suse.de/ibs/QA:/Head/SLE-12-SP3/
START_AFTER_TEST=create_hdd_textmode
SYSTEMD_TESTSUITE=1

Info

This approach should be applied to test a patched systemd version for an already released product. For example, Leap 42.3 and SLE-12-SP3.

As this request would be about testing systemd patches on released products with custom repositories it is neither part of main QA SLE responsibility for build validation of new product versions nor QAM because the patches should be tested before an incident update is prepared.


Related issues

Related to openQA Tests - action #45158: [systemd] Implement systemd testsuite as openQA perl module In Progress 18/01/2019

History

#1 Updated by SLindoMansilla over 2 years ago

  • Subject changed from [sles][functional] Add systemd test suite for SLE<15 and Leap to [sles][functional] Add test suite patched_systemd
  • Description updated (diff)

#2 Updated by SLindoMansilla over 2 years ago

  • Description updated (diff)

#3 Updated by okurz over 2 years ago

  • Category set to New test

#4 Updated by okurz over 2 years ago

  • Subject changed from [sles][functional] Add test suite patched_systemd to [sle][functional] Add test suite patched_systemd
  • Description updated (diff)

#5 Updated by okurz over 2 years ago

  • Due date set to 25/10/2017
  • Status changed from New to In Progress
  • Assignee set to SLindoMansilla
  • Target version set to Milestone 11

#6 Updated by SLindoMansilla over 2 years ago

  • Description updated (diff)

#7 Updated by SLindoMansilla over 2 years ago

  • Description updated (diff)

#8 Updated by SLindoMansilla over 2 years ago

  • Description updated (diff)

WIP: https://github.com/SergioAtSUSE/os-autoinst-distri-opensuse/commit/7c3a9df40512e800889d7bf207e2aa560b7e8780

  • I need to clarify with tblume how do I get those patched systemd.
  • I need him to prepare the packages on QA:Head and on OBS.

#9 Updated by SLindoMansilla over 2 years ago

  • Status changed from In Progress to Feedback

tblume is preparing the packages on OBS for Leap and IBS/QA:HEAD for SLE-12.
He will contact me when he finishes.

#10 Updated by SLindoMansilla over 2 years ago

  • Description updated (diff)
  • Status changed from Feedback to In Progress

Thomas Blume will provide the testsuite for opensuse at: https://build.opensuse.org/project/show/home:tsaupe:testsuites

#11 Updated by SLindoMansilla over 2 years ago

  • Status changed from In Progress to Feedback
  • Assignee changed from SLindoMansilla to sebchlad

As spoken with okurz and jrivera, this implementation is on the area of QAM, we should keep in contact with them to:

  • Know what are "maintenance requests" and "incidents"
  • Know how is their workflow since a bug is discovered, until it gets to the product.
  • How would this systemd-testsuite affects them and how would they use it.

okurz suggested to make a meeting where we can talk with them about this. sebchlad could you prepare this appointment?

#12 Updated by okurz over 2 years ago

  • Due date changed from 25/10/2017 to 08/11/2017
  • Status changed from Feedback to In Progress

PO can lead this within next sprint

#13 Updated by sebchlad over 2 years ago

Some talks with the QAM has happened however there is no easy conclusion on this one as of now. I would suggest moving it to the next sprint and keep in assigned to me.
If I won't be able to address it in the next sprint we should down-prioritize it or remove it from QA SLE Functional product backlog altogether.

#14 Updated by okurz over 2 years ago

  • Due date deleted (08/11/2017)
  • Target version deleted (Milestone 11)

not done, needs re-evaluation

#15 Updated by okurz over 2 years ago

  • Target version set to Milestone 12

#16 Updated by okurz over 2 years ago

  • Due date set to 19/12/2017

#17 Updated by sebchlad over 2 years ago

  • Due date changed from 19/12/2017 to 02/01/2018

#18 Updated by okurz over 2 years ago

  • Due date changed from 02/01/2018 to 30/01/2018
  • Target version changed from Milestone 12 to Milestone 13

#19 Updated by okurz about 2 years ago

So @sebchlad your statement from 3 months ago was "If I won't be able to address it in the next sprint we should down-prioritize it or remove it from QA SLE Functional product backlog altogether." Should we do the latter now?

#20 Updated by SLindoMansilla about 2 years ago

  • Status changed from In Progress to Feedback

Waiting for feedback from sebchlad.
He told me he rejected this task.

#21 Updated by riafarov about 2 years ago

  • Due date changed from 30/01/2018 to 13/02/2018
  • Target version changed from Milestone 13 to Milestone 14

@Sergio please either reject or ask Seba once again

#22 Updated by SLindoMansilla about 2 years ago

Seba doesn't want to reject it. Waiting for him to put feedback here.

#23 Updated by okurz about 2 years ago

  • Due date changed from 13/02/2018 to 27/02/2018
  • Priority changed from Normal to High

#24 Updated by riafarov about 2 years ago

  • Due date deleted (27/02/2018)

Removing due date, so is not shown in the sprint backlog. Seba, please contact Olie to schedule this one, once we are clear on goal.

#25 Updated by okurz about 2 years ago

  • Assignee deleted (sebchlad)
  • Priority changed from High to Low
  • Target version changed from Milestone 14 to future

I guess it's not really important considering that we are just moving the issue back and forth. @slindomansilla do it whenever you fancy doing something with low prio and low effort or never.

#26 Updated by okurz almost 2 years ago

  • Subject changed from [sle][functional] Add test suite patched_systemd to [sle][functional][u] Add test suite patched_systemd

#27 Updated by okurz almost 2 years ago

  • Target version changed from future to future

#28 Updated by okurz over 1 year ago

  • Due date set to 31/12/2019

#29 Updated by SLindoMansilla over 1 year ago

  • Status changed from Feedback to Workable

I suspect this is going nowhere as "waiting for feedback" without assignee.

#30 Updated by okurz over 1 year ago

  • Description updated (diff)
  • Status changed from Workable to New

Yes, true. But not "Workable" either as I am missing a bit more involvement of systemd dev team here as well as probably ACs regarding how we can provide better feedback to dev. Feel free to bring this topic up again in person when we meet in the office to clarify.

#31 Updated by SLindoMansilla over 1 year ago

I agree, I didn't find the "new" state. It seems a ticket in feedback cannot be directly put in "new" state.

#32 Updated by SLindoMansilla over 1 year ago

  • Related to action #45158: [systemd] Implement systemd testsuite as openQA perl module added

#33 Updated by szarate 17 days ago

  • Subject changed from [sle][functional][u] Add test suite patched_systemd to [sle][functional] Add test suite patched_systemd

#34 Updated by szarate 17 days ago

  • Assignee set to tsaupe

If you need help, let us know

For now an idea from us, is to write a setup guide to have this solved easily.

#35 Updated by tsaupe 1 day ago

  • Status changed from New to In Progress

Testing code to run the systemd-qa-package has been added to openQA.
See for example: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/9822

If there is no objection I will close this project.

Also available in: Atom PDF