Project

General

Profile

Actions

action #42503

closed

coordination #40475: [functional][y][saga] Establish YaST team split

coordination #42191: [functional][y][epic] Have separate job group for YaST subteam

[functional][y] Define test matrix for the YaST job group

Added by riafarov about 6 years ago. Updated almost 6 years ago.

Status:
Resolved
Priority:
Low
Assignee:
Category:
Enhancement to existing tests
Target version:
SUSE QA (private) - Milestone 23
Start date:
2018-10-15
Due date:
2019-03-26
% Done:

0%

Estimated time:
3.00 h
Difficulty:

Description

Motivation

When we'll have separate job group, we need good understanding of what we have in it and what we want to have. By that we also can decide where how we want to test one thing and another.
There are many examples where we could be more efficient and gain time for increasing test coverage for more critical areas instead of covering e.g. firefox on all possible environments.
This also includes proper evaluation of the risks when not running certain test module.
As a starting point we already have test suites descriptions + test modules descriptions. But it's hard to identify which modules are executed where, and on which env do we cover certain setups. That is visible only in the openQA dashboard.

Suggestions

There is one team which uses testopia exactly for this purpose. We need to identify good format which could be reused among teams and used e.g. as a reporting template to visualize coverage per build.
Markdown is more preferable

Acceptance criteria

  1. All existing test suites in the YaST job group have description
  2. Test suite includes env description, executed test cases summary and purpose

NOTE: should be handled once we have current state of the job group, meaning after at least #42494 and #42503 are done.

Actions #1

Updated by riafarov about 6 years ago

  • Description updated (diff)
Actions #2

Updated by okurz about 6 years ago

  • Category set to Enhancement to existing tests
  • Target version set to Milestone 22

Please try to always assign a category

Actions #3

Updated by riafarov about 6 years ago

  • Status changed from New to Workable

@okurz: Will do, thanks for the feedback.

Actions #4

Updated by riafarov about 6 years ago

  • Due date set to 2018-12-04
Actions #5

Updated by riafarov about 6 years ago

  • Estimated time set to 3.00 h
Actions #6

Updated by riafarov about 6 years ago

  • Assignee set to riafarov
Actions #7

Updated by riafarov about 6 years ago

  • Status changed from Workable to Feedback

After spending 1 full day I was not able to describe all test suites: https://gitlab.suse.de/riafarov/qa-sle-functional-y/blob/master/SLES_Integration_Level_Testplan.md
As of now, we agreed to gather some feedback before proceeding further. Reviewing current coverage has already helped to identify multiple action points.

Actions #8

Updated by riafarov about 6 years ago

  • Due date changed from 2018-12-04 to 2018-12-18
Actions #9

Updated by okurz about 6 years ago

my feedback from the sprint review meeting: A link to the existing test plan would be great (two-way)

also the document mentions "SLES" but we do not cover only SLES, I would extend that to SLE and potentially also include openSUSE

Actions #10

Updated by riafarov about 6 years ago

@okurz, yes, there are quite some parts missing. I have limited scope to YaST job group only for now. Sebastian supports me on this too, as we are trying to solve it not for a single team. Thanks for the feedback!

Actions #11

Updated by riafarov about 6 years ago

  • Due date changed from 2018-12-18 to 2019-01-15

Waiting for the feedback from Sebastian.

Actions #12

Updated by riafarov almost 6 years ago

  • Due date deleted (2019-01-15)
  • Priority changed from Normal to Low
  • Target version changed from Milestone 22 to Milestone 23

Keep collecting the feedback to decide how to proceed here.

Actions #13

Updated by okurz almost 6 years ago

  • Due date set to 2019-03-26
Actions #14

Updated by riafarov almost 6 years ago

  • Status changed from Feedback to Resolved

All scenarios currently enabled in YaST job group are defined: https://gitlab.suse.de/riafarov/qa-sle-functional-y/blob/master/SLES_Integration_Level_Testplan.md
Let's keep it up to date now.

Actions

Also available in: Atom PDF