Project

General

Profile

Actions

action #45152

closed

[sle][functional][y][rt] extend test coverage of RT installation

Added by mloviska over 5 years ago. Updated about 5 years ago.

Status:
Resolved
Priority:
Low
Assignee:
Category:
New test
Target version:
SUSE QA - Milestone 23
Start date:
2018-12-13
Due date:
2019-03-26
% Done:

0%

Estimated time:
3.00 h
Difficulty:

Description

I would like to initiate a discussion regarding possible modules to use in installation validation.
Let me propose the bear minimum modules, which might be useful and already exists in our test directory.

  • console/firewall_enabled
  • console/aplay
  • console/btrfs_autocompletition
  • console/glibc_sanity
  • console/gpt_ptable
  • console/gpg
  • console/journalctl
  • console/kdump_and_crash
  • console/rsync
  • console/http_srv
  • console/sshd
  • console/sysctl
  • console/sysstat
  • console/cron
  • console/yast2_lan
  • console/yast2_bootloader
  • console/yast2_i
  • console/yast2_clone_system
  • toolchain/gcc_compilation
  • toolchain/gcc_fortran_compilation
  • x11/firefox
  • x11/wireshark
  • x11/yast2_snapper
  • x11/yast2_hostnames
  • x11/yast2_firewall
  • console/consoletest_setup
  • console/zypper.ref
  • console/zypper.lr
  • rt/kmp_modules
  • rt/rt_devel_packages
  • rt/rt_is_realtime
  • rt/rt_peak_pci
  • rt/rt_preempt_test
  • console/hostname

Acceptance criteria

  1. List of modules to be used is identified
  2. The identified set of modules is enabled in the test suite for RT product

Suggestions

We could use new scheduling mechanism not to deal with main.pm complexity.


Related issues 2 (0 open2 closed)

Related to openQA Tests - action #45149: [sle][functional][y][rt][bsc#1119412] validate booted kernel (or workaround for kernel selection)Resolvedmloviska2018-12-132019-02-12

Actions
Blocked by openQA Tests - action #46874: [sle][functional][rt][y] - replace rt_preempt_test by rt-tests ( cyclictest & hackbench )Resolvedmloviska2019-01-302019-02-26

Actions
Actions #1

Updated by mloviska over 5 years ago

  • Description updated (diff)
Actions #2

Updated by mloviska over 5 years ago

  • Category set to Enhancement to existing tests
Actions #3

Updated by okurz over 5 years ago

  • Category changed from Enhancement to existing tests to New test
  • Target version set to Milestone 23

Hi Martin, I will change the category to "New test" as we use that commonly for any extension of test coverage, regardless if we would actually need to add any new test modules, scenarios, etc. See https://progress.opensuse.org/projects/openqatests/wiki#ticket-workflow for details. OK for you?

I think this is a good idea and thanks for the detailed list. I would go with #45149 first and then this one so maybe something for M23 assuming that M22 will not have much non-Christmas-vacation time left :)

Actions #4

Updated by okurz about 5 years ago

  • Status changed from New to Feedback
  • Assignee set to okurz

waiting for clarification (email)

Actions #5

Updated by mloviska about 5 years ago

  • Related to action #45149: [sle][functional][y][rt][bsc#1119412] validate booted kernel (or workaround for kernel selection) added
Actions #6

Updated by mloviska about 5 years ago

  • Description updated (diff)

Recently added test modules for #45149

Actions #7

Updated by mloviska about 5 years ago

  • Due date set to 2018-12-26
Actions #8

Updated by mloviska about 5 years ago

  • Due date changed from 2018-12-26 to 2019-02-26
Actions #9

Updated by riafarov about 5 years ago

  • Status changed from Feedback to Blocked

Let's resolve #46874 first, as anyway image is not yet created.

Actions #10

Updated by riafarov about 5 years ago

  • Blocked by action #46874: [sle][functional][rt][y] - replace rt_preempt_test by rt-tests ( cyclictest & hackbench ) added
Actions #11

Updated by okurz about 5 years ago

  • Due date changed from 2019-02-26 to 2019-03-12
  • Status changed from Blocked to New
  • Assignee deleted (okurz)
  • Priority changed from Normal to Low

blocker resolved. Also setting to "Low" as in comparison to other components where are not covered at all here it is only about executing the modules we already execute in other cases for the RT-case where I see only minor risk that something is breaking for RT, e.g. on the RT kernel, but not on normal SLES.

So I see three different options:

  1. Keep the existing "just boot, validate the kernel and stop" flow -> least effort, least coverage
  2. Select the modules especially useful for RT and run these in a custom schedule flow -> highest effort but less risk of modules failing for the same reason as in SLES and we might need to add label multiple times
  3. Just run the standard console tests + gnome tests based on the default schedule flow as we also have for default SLES -> middle effort, best coverage

Did I miss something? What do you guys prefer?

Actions #12

Updated by riafarov about 5 years ago

  • Description updated (diff)
Actions #13

Updated by riafarov about 5 years ago

  • Status changed from New to Workable
Actions #14

Updated by riafarov about 5 years ago

  • Due date changed from 2019-03-12 to 2019-03-26
Actions #15

Updated by okurz about 5 years ago

  • Status changed from Workable to New

Please let's discuss my questions #45152#note-11

Actions #16

Updated by mloviska about 5 years ago

I would definitely go for option 2.

Actions #17

Updated by okurz about 5 years ago

Alright. It's the "highest effort" but I consider it still feasible to do within the next sprint in theory. Currently we have quite some tasks already planned for the sprint so we might need to delay this ticket.

Actions #18

Updated by mloviska about 5 years ago

  • Description updated (diff)
Actions #19

Updated by riafarov about 5 years ago

  • Description updated (diff)
  • Estimated time set to 3.00 h

Kernel is tested in another test suite.
No modifications of the test modules should be required, only bug report as it should work as good as for SLES product.

Actions #20

Updated by riafarov about 5 years ago

  • Status changed from New to Workable
Actions #21

Updated by mloviska about 5 years ago

  • Assignee set to mloviska
Actions #22

Updated by mloviska about 5 years ago

  • Status changed from Workable to Feedback
Actions #23

Updated by JERiveraMoya about 5 years ago

  • Status changed from Feedback to In Progress
Actions #24

Updated by mloviska about 5 years ago

  • Status changed from In Progress to Feedback
Actions #25

Updated by mloviska about 5 years ago

  • Status changed from Feedback to Resolved
Actions

Also available in: Atom PDF