Project

General

Profile

Actions
Copy link

action #113297

closed

coordination #108797: [qe-core] Leap setup 15.4 openqa for updates

coordination #111258: [qe-core] Increase the coverage of test scenarios in Leap maintainance

[qe-core] Increase the coverage of test scenarios in Leap Maintainance - Proposal for tests from SLES to be added to leap

Added by szarate over 2 years ago. Updated over 2 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
mloviska
Category:
Spike/Research
Target version:
QA (public) - QE-Core: Ready
Start date:
2022-07-06
Due date:
% Done:

0%

Estimated time:
Difficulty:
Sprint:
QE-Core: August Sprint (Aug 03 - Aug 31)
Tags:
qe-core-august-sprint

Description

As #111258 describes right now the testsuites running in Leap 15.3 Updates are not covering many scenarios: https://openqa.opensuse.org/tests/overview?distri=opensuse&version=15.3&build=20220518-1&groupid=80.

The purpose of this ticket is to gather candidate testsuites, to be scheduled in another ticket.

##Acceptance criteria

  1. AC.1 Gather the testsuites that can be added in Leap maintenance in order to have broader testing.
  2. AC.2 Contact Lubos and gather his feedback on said list

Related issues 1 (1 open — 0 closed)

Copied to openQA Tests (public) - action #115913: [qe-core] Increase the coverage of test scenarios in Leap Maintainance - Further enhancements to leap test scenariosNewszarate2022-07-06

Actions
Actions
Copy link
 #1

Updated by szarate over 2 years ago

  • Tracker changed from coordination to action
Actions
Copy link
 #2

Updated by punkioudi over 2 years ago

@szarate is this ticket for July sprint?

Actions
Copy link
 #3

Updated by szarate over 2 years ago

punkioudi wrote:

@szarate is this ticket for July sprint?

Yes, this ticket is the action, the epic stays blocked :)

Actions
Copy link
 #4

Updated by mloviska over 2 years ago

  • Status changed from Workable to In Progress
  • Assignee set to mloviska
Actions
Copy link
 #5

Updated by mloviska over 2 years ago

  • Status changed from In Progress to Feedback

According to discussion with Lubos, the highest priority is put on migration test scenarios, as of now we are executing zdup cases.

For Leap15.4 it lists below test suites.

 zdup-Leap-15.2-gnome 
 zdup-Leap-15.2-kde 
 zdup-Leap-15.3-gnome 
 zdup-Leap-15.3-kde

Other than that Lubos mentioned that we should cover autoyast, apparmor and security_* test suites.

As a QA I would also like to have upgrade_Leap_15.2+? scenarios, however we need to add online repos along with Untested Updates repository that is omitted by default.
Along with this, I would suggest following list of test suites that should be added

apparmor
autoyast_gnome
security_audit
transactional_server
extra_tests_textmode_podman_containers 
extra_tests_textmode_docker_containers
upgrade_Leap_15.2+ + enable untested package repo
extra_tests_transactional_server

Also, we should add aarch64.

Actions
Copy link
 #6

Updated by szarate over 2 years ago

Thanks a lot Martin! could you give on top of this a look at what from SLES maintenance could be added too?, for instance the extratests aren't complete from what I can see (there's no textmode for instance)

Actions
Copy link
 #7

Updated by mloviska over 2 years ago

szarate wrote:

Thanks a lot Martin! could you give on top of this a look at what from SLES maintenance could be added too?, for instance the extratests aren't complete from what I can see (there's no textmode for instance)

I have checked that as well, maybe for leap makes sense to test clamav. We do test textmode in Leap Updates, but indeed you are right that more extratests can be added there as well.

Actions
Copy link
 #8

Updated by mloviska over 2 years ago

Should I start adding more tests as part of this ticket ? Maybe in one group as of now as a PoC?

Actions
Copy link
 #9

Updated by szarate over 2 years ago

mloviska wrote:

Should I start adding more tests as part of this ticket ? Maybe in one group as of now as a PoC?

Please do :)

Actions
Copy link
 #10

Updated by mloviska over 2 years ago

  • Status changed from Feedback to In Progress
Actions
Copy link
 #12

Updated by mloviska over 2 years ago

  • Status changed from In Progress to Feedback
Actions
Copy link
 #13

Updated by szarate over 2 years ago

Let's follow up walking over eggshells:

  • Prioritize anything that doesn't depend on a desktop environment (consoletests)
  • with extratests_*_console
  • Incindents and Backports haven't been toched now
  • Review coverage of X11 tests before adding more stuff (specially on aarch64)
Actions
Copy link
 #14

Updated by mloviska over 2 years ago

  • Status changed from Feedback to In Progress
Actions
Copy link
 #15

Updated by szarate over 2 years ago

  • Sprint changed from QE-Core: July Sprint (Jul 06 - Aug 03) to QE-Core: August Sprint (Aug 03 - Aug 31)
Actions
Copy link
 #16

Updated by szarate over 2 years ago

  • Tags set to qe-core-august-sprint
  • Status changed from In Progress to Resolved

Will folllow up in a separate ticket following https://progress.opensuse.org/issues/113297#note-13

Actions
Copy link
 #17

Updated by szarate over 2 years ago

  • Copied to action #115913: [qe-core] Increase the coverage of test scenarios in Leap Maintainance - Further enhancements to leap test scenarios added
Actions
Copy link

Also available in: Atom PDF