Project

General

Profile

Actions

action #113297

closed

coordination #108797: [qe-core] Leap setup 15.4 openqa for updates

coordination #111258: [qe-core] Increase the coverage of test scenarios in Leap maintainance

[qe-core] Increase the coverage of test scenarios in Leap Maintainance - Proposal for tests from SLES to be added to leap

Added by szarate over 1 year ago. Updated over 1 year ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Spike/Research
Target version:
Start date:
2022-07-06
Due date:
% Done:

0%

Estimated time:
Difficulty:
Sprint:
QE-Core: August Sprint (Aug 03 - Aug 31)

Description

As #111258 describes right now the testsuites running in Leap 15.3 Updates are not covering many scenarios: https://openqa.opensuse.org/tests/overview?distri=opensuse&version=15.3&build=20220518-1&groupid=80.

The purpose of this ticket is to gather candidate testsuites, to be scheduled in another ticket.

##Acceptance criteria

  1. AC.1 Gather the testsuites that can be added in Leap maintenance in order to have broader testing.
  2. AC.2 Contact Lubos and gather his feedback on said list

Related issues 1 (1 open0 closed)

Copied to openQA Tests - action #115913: [qe-core] Increase the coverage of test scenarios in Leap Maintainance - Further enhancements to leap test scenariosNewszarate2022-07-06

Actions
Actions #1

Updated by szarate over 1 year ago

  • Tracker changed from coordination to action
Actions #2

Updated by punkioudi over 1 year ago

@szarate is this ticket for July sprint?

Actions #3

Updated by szarate over 1 year ago

punkioudi wrote:

@szarate is this ticket for July sprint?

Yes, this ticket is the action, the epic stays blocked :)

Actions #4

Updated by mloviska over 1 year ago

  • Status changed from Workable to In Progress
  • Assignee set to mloviska
Actions #5

Updated by mloviska over 1 year ago

  • Status changed from In Progress to Feedback

According to discussion with Lubos, the highest priority is put on migration test scenarios, as of now we are executing zdup cases.

For Leap15.4 it lists below test suites.

 zdup-Leap-15.2-gnome 
 zdup-Leap-15.2-kde 
 zdup-Leap-15.3-gnome 
 zdup-Leap-15.3-kde 

Other than that Lubos mentioned that we should cover autoyast, apparmor and security_* test suites.

As a QA I would also like to have upgrade_Leap_15.2+? scenarios, however we need to add online repos along with Untested Updates repository that is omitted by default.
Along with this, I would suggest following list of test suites that should be added

apparmor
autoyast_gnome
security_audit
transactional_server
extra_tests_textmode_podman_containers 
extra_tests_textmode_docker_containers
upgrade_Leap_15.2+ + enable untested package repo
extra_tests_transactional_server

Also, we should add aarch64.

Actions #6

Updated by szarate over 1 year ago

Thanks a lot Martin! could you give on top of this a look at what from SLES maintenance could be added too?, for instance the extratests aren't complete from what I can see (there's no textmode for instance)

Actions #7

Updated by mloviska over 1 year ago

szarate wrote:

Thanks a lot Martin! could you give on top of this a look at what from SLES maintenance could be added too?, for instance the extratests aren't complete from what I can see (there's no textmode for instance)

I have checked that as well, maybe for leap makes sense to test clamav. We do test textmode in Leap Updates, but indeed you are right that more extratests can be added there as well.

Actions #8

Updated by mloviska over 1 year ago

Should I start adding more tests as part of this ticket ? Maybe in one group as of now as a PoC?

Actions #9

Updated by szarate over 1 year ago

mloviska wrote:

Should I start adding more tests as part of this ticket ? Maybe in one group as of now as a PoC?

Please do :)

Actions #10

Updated by mloviska over 1 year ago

  • Status changed from Feedback to In Progress
Actions #12

Updated by mloviska over 1 year ago

  • Status changed from In Progress to Feedback
Actions #13

Updated by szarate over 1 year ago

Let's follow up walking over eggshells:

  • Prioritize anything that doesn't depend on a desktop environment (consoletests)
  • with extratests_*_console
  • Incindents and Backports haven't been toched now
  • Review coverage of X11 tests before adding more stuff (specially on aarch64)
Actions #14

Updated by mloviska over 1 year ago

  • Status changed from Feedback to In Progress
Actions #15

Updated by szarate over 1 year ago

  • Sprint changed from QE-Core: July Sprint (Jul 06 - Aug 03) to QE-Core: August Sprint (Aug 03 - Aug 31)
Actions #16

Updated by szarate over 1 year ago

  • Tags set to qe-core-august-sprint
  • Status changed from In Progress to Resolved

Will folllow up in a separate ticket following https://progress.opensuse.org/issues/113297#note-13

Actions #17

Updated by szarate over 1 year ago

  • Copied to action #115913: [qe-core] Increase the coverage of test scenarios in Leap Maintainance - Further enhancements to leap test scenarios added
Actions

Also available in: Atom PDF