Project

General

Profile

action #42503

action #40475: [functional][y][saga] Establish YaST team split

action #42191: [functional][y][epic] Have separate job group for YaST subteam

[functional][y] Define test matrix for the YaST job group

Added by riafarov over 1 year ago. Updated about 1 year ago.

Status:
Resolved
Priority:
Low
Assignee:
Category:
Enhancement to existing tests
Target version:
SUSE QA tests - Milestone 23
Start date:
2018-10-15
Due date:
2019-03-26
% Done:

0%

Estimated time:
3.00 h
Difficulty:
Duration: 117

Description

Motivation

When we'll have separate job group, we need good understanding of what we have in it and what we want to have. By that we also can decide where how we want to test one thing and another.
There are many examples where we could be more efficient and gain time for increasing test coverage for more critical areas instead of covering e.g. firefox on all possible environments.
This also includes proper evaluation of the risks when not running certain test module.
As a starting point we already have test suites descriptions + test modules descriptions. But it's hard to identify which modules are executed where, and on which env do we cover certain setups. That is visible only in the openQA dashboard.

Suggestions

There is one team which uses testopia exactly for this purpose. We need to identify good format which could be reused among teams and used e.g. as a reporting template to visualize coverage per build.
Markdown is more preferable

Acceptance criteria

  1. All existing test suites in the YaST job group have description
  2. Test suite includes env description, executed test cases summary and purpose

NOTE: should be handled once we have current state of the job group, meaning after at least #42494 and #42503 are done.

History

#1 Updated by riafarov over 1 year ago

  • Description updated (diff)

#2 Updated by okurz over 1 year ago

  • Category set to Enhancement to existing tests
  • Target version set to Milestone 22

Please try to always assign a category

#3 Updated by riafarov over 1 year ago

  • Status changed from New to Workable

okurz: Will do, thanks for the feedback.

#4 Updated by riafarov over 1 year ago

  • Due date set to 2018-12-04

#5 Updated by riafarov over 1 year ago

  • Estimated time set to 3.00 h

#6 Updated by riafarov over 1 year ago

  • Assignee set to riafarov

#7 Updated by riafarov over 1 year ago

  • Status changed from Workable to Feedback

After spending 1 full day I was not able to describe all test suites: https://gitlab.suse.de/riafarov/qa-sle-functional-y/blob/master/SLES_Integration_Level_Testplan.md
As of now, we agreed to gather some feedback before proceeding further. Reviewing current coverage has already helped to identify multiple action points.

#8 Updated by riafarov over 1 year ago

  • Due date changed from 2018-12-04 to 2018-12-18

#9 Updated by okurz over 1 year ago

my feedback from the sprint review meeting: A link to the existing test plan would be great (two-way)

also the document mentions "SLES" but we do not cover only SLES, I would extend that to SLE and potentially also include openSUSE

#10 Updated by riafarov over 1 year ago

okurz, yes, there are quite some parts missing. I have limited scope to YaST job group only for now. Sebastian supports me on this too, as we are trying to solve it not for a single team. Thanks for the feedback!

#11 Updated by riafarov over 1 year ago

  • Due date changed from 2018-12-18 to 2019-01-15

Waiting for the feedback from Sebastian.

#12 Updated by riafarov over 1 year ago

  • Due date deleted (2019-01-15)
  • Priority changed from Normal to Low
  • Target version changed from Milestone 22 to Milestone 23

Keep collecting the feedback to decide how to proceed here.

#13 Updated by okurz over 1 year ago

  • Due date set to 2019-03-26

#14 Updated by riafarov about 1 year ago

  • Status changed from Feedback to Resolved

All scenarios currently enabled in YaST job group are defined: https://gitlab.suse.de/riafarov/qa-sle-functional-y/blob/master/SLES_Integration_Level_Testplan.md
Let's keep it up to date now.

Also available in: Atom PDF