Project

General

Profile

Actions
Copy link

coordination #36709

closed

[functional][y][epic] Improve informing about test scenarios

Added by riafarov almost 6 years ago. Updated over 3 years ago.

Status:
Resolved
Priority:
High
Assignee:
okurz
Category:
Enhancement to existing tests
Target version:
SUSE QA - Milestone 23
Start date:
2019-01-09
Due date:
2019-02-12
% Done:

100%

Estimated time:
(Total: 8.00 h)
Difficulty:

Description

Motivation

As a developer I want to see test suite description on the test run page (not only in the job group overview) and additionally include it in the bug template, so it's clear which scenario is it.

Sometimes, when bug is created, it's not clear what is the setup. On top it would be nice to have test module description to understand what the test does.
This information is already there and we can simply extract it. It will also help to maintain proper description of the test suites and test modules.

As of now we force putting these details manually, but still it happens that one believes that failure is obvious but it's not.

Acceptance criteria

  • AC1: Test suite description can be found on the test run page -> #45887
  • AC2: Test suite description is at least referenced in the bug creation template -> #45890

Subtasks 2 (0 open — 2 closed)

openQA Project - action #45887: [functional][y] testsuite description on the test run pageResolvedandriinikitin2019-01-092019-02-12

Actions
openQA Project - action #45890: [tools] testsuite description in the bug reporting templateResolvedokurz2019-01-09

Actions
Actions
Copy link
 #1

Updated by okurz almost 6 years ago

  • Subject changed from [functional][tools] Improve informing about test scenarios to [functional][tools][epic] Improve informing about test scenarios
  • Category set to Enhancement to existing tests
  • Target version set to Milestone 18

riafarov wrote:

Motivation

As a developer I want to see test suite description on the test run page (not only in the job group overview)

Sounds like a distinct feature request for openQA itself. We can extract this into another ticket after we clarified some more details.

… and additionally include it in the bug template, so it's clear which scenario is it.

Could work when we access the description over a specfic route within openQA

Sometimes, when bug is created, it's not clear what is the setup. On top it would be nice to have test module description to understand what the test does.
This information is already there and we can simply extract it.

Yes, it's already available by clicking on the test module name. If we would "extract it", where to put it? Publish on github pages linked to os-autoinst-distri-opensuse?

It will also help to maintain proper description of the test suites and test modules.

What do you mean with "maintain" here? We do have a description of test suites, at least in functional. Should we have some automatic check for description and create a TODO page or alerts for undescribed test suites? We already have a check for test modules as part of os-autoinst-distri-opensuse so every test module must have a description, i.e. a "Summary:"-line.

Actions
Copy link
 #3

Updated by riafarov almost 6 years ago

okurz wrote:

riafarov wrote:

Motivation

As a developer I want to see test suite description on the test run page (not only in the job group overview)

Sounds like a distinct feature request for openQA itself. We can extract this into another ticket after we clarified some more details.

… and additionally include it in the bug template, so it's clear which scenario is it.

Could work when we access the description over a specfic route within openQA
Including it in bug report is one thing, other thing is to display this info on the job run. Any of those two will do the trick. Feature is based on the feedback from YaST team.

Sometimes, when bug is created, it's not clear what is the setup. On top it would be nice to have test module description to understand what the test does.
This information is already there and we can simply extract it.

Yes, it's already available by clicking on the test module name. If we would "extract it", where to put it? Publish on github pages linked to os-autoinst-distri-opensuse?
We have at least summary which is forced for each test module, so we can show it as a pop-up, for example.

It will also help to maintain proper description of the test suites and test modules.

What do you mean with "maintain" here? We do have a description of test suites, at least in functional. Should we have some automatic check for description and create a TODO page or alerts for undescribed test suites? We already have a check for test modules as part of os-autoinst-distri-opensuse so every test module must have a description, i.e. a "Summary:"-line.
Some test suites have to general description, on o3, for example. For test modules sometimes description doesn't match 100%, so there is space for the improvement.

Actions
Copy link
 #4

Updated by okurz almost 6 years ago

sorry, seems you have messed up markdown formatting in your quoting. Please add blank lines to separate your answers from mine. In the email it looked ok but the ticket rendered in the browser does not show it correctly.

And based on your answers I think you can already create individual feature requests in the openQA project and probably also move this ticket there. What I do not see covered yet is a workflow how it can be ensured that all test suites have a description. The "test module description does not match 100%" IMHO can only be solved by a case-by-case update and looking for this in PR review.

Actions
Copy link
 #5

Updated by okurz almost 6 years ago

  • Target version changed from Milestone 18 to Milestone 18
Actions
Copy link
 #6

Updated by okurz over 5 years ago

  • Subject changed from [functional][tools][epic] Improve informing about test scenarios to [functional][y][epic] Improve informing about test scenarios
  • Target version changed from Milestone 18 to Milestone 20

@riafarov do you think this could be something for @y-team? if not, please remove "[y]" again and move it to the parent project as a generic idea please.

Actions
Copy link
 #7

Updated by okurz over 5 years ago

  • Due date set to 2018-11-20
Actions
Copy link
 #8

Updated by okurz over 5 years ago

  • Due date changed from 2018-11-20 to 2018-12-04
  • Target version changed from Milestone 20 to Milestone 21

It seems this was missed in the last sprint?

Actions
Copy link
 #9

Updated by okurz over 5 years ago

  • Target version changed from Milestone 21 to Milestone 22

let's try again

Actions
Copy link
 #10

Updated by okurz over 5 years ago

  • Due date changed from 2018-12-04 to 2019-01-29
Actions
Copy link
 #11

Updated by okurz over 5 years ago

  • Description updated (diff)
  • Status changed from New to In Progress
  • Assignee set to okurz

fleshing out subtasks

Actions
Copy link
 #12

Updated by okurz over 5 years ago

  • Description updated (diff)
  • Status changed from In Progress to Blocked

two subtasks created for the two ACs

Actions
Copy link
 #13

Updated by riafarov over 5 years ago

  • Due date changed from 2019-01-29 to 2019-02-12

due to changes in a related task

Actions
Copy link
 #14

Updated by okurz about 5 years ago

  • Target version changed from Milestone 22 to Milestone 23

I would have hoped the tools team takes more responsibility to what broken state they deployed on osd :(

Actions
Copy link
 #15

Updated by okurz about 5 years ago

  • Status changed from Blocked to Resolved

fixed the subtask myself. The test description is visible on test run pages as well as in the bug reporting templates – deployed at least on o3. Considering ourselves done here.

Actions
Copy link
 #16

Updated by riafarov about 5 years ago

\o/

Actions
Copy link
 #17

Updated by szarate over 3 years ago

  • Tracker changed from action to coordination
Actions
Copy link

Also available in: Atom PDF