Project

General

Profile

action #113297

coordination #108797: [qe-core] Leap setup 15.4 openqa for updates

coordination #111258: [qe-core] Increase the coverage of test scenarios in Leap maintainance

[qe-core] Increase the coverage of test scenarios in Leap Maintainance - Proposal for tests from SLES to be added to leap

Added by szarate about 1 month ago. Updated 12 days ago.

Status:
In Progress
Priority:
Normal
Assignee:
Category:
Spike/Research
Target version:
Start date:
2022-07-06
Due date:
% Done:

0%

Estimated time:
Difficulty:

Description

As #111258 describes right now the testsuites running in Leap 15.3 Updates are not covering many scenarios: https://openqa.opensuse.org/tests/overview?distri=opensuse&version=15.3&build=20220518-1&groupid=80.

The purpose of this ticket is to gather candidate testsuites, to be scheduled in another ticket.

##Acceptance criteria

  1. AC.1 Gather the testsuites that can be added in Leap maintenance in order to have broader testing.
  2. AC.2 Contact Lubos and gather his feedback on said list

History

#1 Updated by szarate about 1 month ago

  • Tracker changed from coordination to action

#2 Updated by punkioudi about 1 month ago

szarate is this ticket for July sprint?

#3 Updated by szarate about 1 month ago

punkioudi wrote:

szarate is this ticket for July sprint?

Yes, this ticket is the action, the epic stays blocked :)

#4 Updated by mloviska about 1 month ago

  • Status changed from Workable to In Progress
  • Assignee set to mloviska

#5 Updated by mloviska about 1 month ago

  • Status changed from In Progress to Feedback

According to discussion with Lubos, the highest priority is put on migration test scenarios, as of now we are executing zdup cases.

For Leap15.4 it lists below test suites.

 zdup-Leap-15.2-gnome 
 zdup-Leap-15.2-kde 
 zdup-Leap-15.3-gnome 
 zdup-Leap-15.3-kde 

Other than that Lubos mentioned that we should cover autoyast, apparmor and security_* test suites.

As a QA I would also like to have upgrade_Leap_15.2+? scenarios, however we need to add online repos along with Untested Updates repository that is omitted by default.
Along with this, I would suggest following list of test suites that should be added

apparmor
autoyast_gnome
security_audit
transactional_server
extra_tests_textmode_podman_containers 
extra_tests_textmode_docker_containers
upgrade_Leap_15.2+ + enable untested package repo
extra_tests_transactional_server

Also, we should add aarch64.

#6 Updated by szarate about 1 month ago

Thanks a lot Martin! could you give on top of this a look at what from SLES maintenance could be added too?, for instance the extratests aren't complete from what I can see (there's no textmode for instance)

#7 Updated by mloviska about 1 month ago

szarate wrote:

Thanks a lot Martin! could you give on top of this a look at what from SLES maintenance could be added too?, for instance the extratests aren't complete from what I can see (there's no textmode for instance)

I have checked that as well, maybe for leap makes sense to test clamav. We do test textmode in Leap Updates, but indeed you are right that more extratests can be added there as well.

#8 Updated by mloviska 28 days ago

Should I start adding more tests as part of this ticket ? Maybe in one group as of now as a PoC?

#9 Updated by szarate 28 days ago

mloviska wrote:

Should I start adding more tests as part of this ticket ? Maybe in one group as of now as a PoC?

Please do :)

#10 Updated by mloviska 28 days ago

  • Status changed from Feedback to In Progress

#12 Updated by mloviska 26 days ago

  • Status changed from In Progress to Feedback

#13 Updated by szarate 13 days ago

Let's follow up walking over eggshells:

  • Prioritize anything that doesn't depend on a desktop environment (consoletests)
  • with extratests_*_console
  • Incindents and Backports haven't been toched now
  • Review coverage of X11 tests before adding more stuff (specially on aarch64)

#14 Updated by mloviska 13 days ago

  • Status changed from Feedback to In Progress

#15 Updated by szarate 12 days ago

  • Sprint changed from QE-Core: July Sprint (Jul 06 - Aug 03) to QE-Core: August Sprint (Aug 03 - Aug 31)

Also available in: Atom PDF