Project

General

Profile

Actions

coordination #91193

open

coordination #91142: [qe-core][epic] add product QE exclusive tests to QEM runs to increase coverage

[epic][qe-core][qem] Add existing console Product QE tests to QEM

Added by tjyrinki_suse almost 3 years ago. Updated almost 2 years ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
New test
Target version:
-
Start date:
2021-05-31
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Difficulty:

Description

After comparing test coverage of SLE 15 SP2 and SP3 in opneQA I identified several tests that are exclusive to either pre-release or post-release testing and could be easily used to increase our coverage just by using what we already have.

It is possible that are some errors and some of the tests from the list are already being used. It is also just my opinion, therefor review from other colleagues is more then welcome.

These are the console tests that run only before release and could be used also after the release (product QE -> QEM)

tests/console/btrfsmaintenance.pm (runs on mau-filesystem)
tests/console/journald_fss.pm
tests/console/lvm_thin_check.pm
tests/console/ndctl.pm
tests/console/network_hostname.pm
tests/console/nvme_checks.pm
tests/console/openvswitch_ssl.pm
tests/console/snapper_cleanup_timeline.pm
tests/console/systemd_nspawn.pm
tests/console/verify_default_target.pm
tests/console/verify_network.pm
tests/console/vsftpd.pm

Acceptance criteria

  • AC1 All aforementioned tests are scheduled in maintenance tests (aggregated or incidents), in their corresponding categories (When in doubt ask in qe-core channel)
  • AC2 Cross reference with openqa-module-mapper to figure what's already done and what's running where.
  • AC3 If a test module needs black magic to work (i.e, takes more than half a day), a corresponding ticket is created and it is removed from this list.

Subtasks 3 (0 open3 closed)

action #93264: [qe-core][qem] Add journald_fss, openvswitch_ssl, vsftpd tests to QEMResolvedpunkioudi2021-05-31

Actions
action #93270: [qe-core][qem] Schedule vsftpd in s390x runsResolvedpunkioudi2021-07-21

Actions
action #93814: [qe-core][qem] Schedule network_hostname in maintenance runsResolvedpunkioudi2021-06-10

Actions
Actions #1

Updated by tjyrinki_suse almost 3 years ago

  • Category set to New test
  • Status changed from New to Workable
  • Start date deleted (2021-04-15)
Actions #2

Updated by tjyrinki_suse almost 3 years ago

  • Target version set to QE-Core: Ready
Actions #3

Updated by tjyrinki_suse almost 3 years ago

If there's a test that will be more work intensive, split out it to another ticket and work on the others.

Actions #4

Updated by szarate almost 3 years ago

  • Description updated (diff)
Actions #5

Updated by punkioudi almost 3 years ago

  • Status changed from Workable to In Progress
  • Assignee set to punkioudi
Actions #6

Updated by szarate almost 3 years ago

  • Description updated (diff)
  • Assignee deleted (punkioudi)
Actions #7

Updated by szarate almost 3 years ago

  • Assignee set to punkioudi
Actions #8

Updated by punkioudi almost 3 years ago

Draft [PR] for scheduling

  • tests/console/journald_fss.pm
  • tests/console/openvswitch_ssl.pm
  • tests/console/vsftpd.pm
Actions #9

Updated by dzedro almost 3 years ago

Please stop blindly adding broken tests.

Actions #10

Updated by punkioudi almost 3 years ago

dzedro wrote:

Please stop blindly adding broken tests.

Why do you really think that they were added blindly?
In the [PR], you have mentioned two failed jobs:

  1. 12-sp5 in s390x failed today. As you can see from the VRs that are provided in the description of the pr, this test has passed https://openqa.suse.de/tests/6143653#step/journald_fss/20 .
  2. As for the 12-sp2 x86_84 fail, it was one case that I missed running, which as far as I can see from the fail message, the test installs openvswitch, then tries to start the service and then it fails, which doesn't seem at all to me like a broken test.

In any case, I had also no intention to ignore any issues that they would happen in the 16 jobgroups/sle-versions that they were scheduled.

Actions #11

Updated by punkioudi over 2 years ago

  • Description updated (diff)
  • Assignee deleted (punkioudi)
Actions #12

Updated by tjyrinki_suse over 2 years ago

  • Tracker changed from action to coordination
  • Subject changed from [qe-core][qem] Add existing console Product QE tests to QEM to [qe-core][epic][qem] Add existing console Product QE tests to QEM
Actions #13

Updated by tjyrinki_suse over 2 years ago

  • Subject changed from [qe-core][epic][qem] Add existing console Product QE tests to QEM to [epic][qe-core][qem] Add existing console Product QE tests to QEM
Actions #14

Updated by szarate almost 2 years ago

  • Status changed from In Progress to New
Actions #15

Updated by szarate almost 2 years ago

  • Target version deleted (QE-Core: Ready)
Actions

Also available in: Atom PDF