Project

General

Profile

action #49370

Get rid of tests in "test development" that no one cares about (was: Ensure all tests in "test development" have a open ticket reference)

Added by okurz about 2 years ago. Updated 8 months ago.

Status:
Workable
Priority:
Normal
Assignee:
-
Category:
Feature requests
Target version:
Start date:
2019-03-15
Due date:
% Done:

0%

Estimated time:
42.00 h
Difficulty:

Description

Motivation

There are quite some jobs in "Test Development" that actually finish successfully but are hardly looked at when they are in test development. Also there are reproducibly failing tests that just waste ressources. We should be "good citizens" and clean up.

Acceptance criteria

  • AC1: No reproducibly failing tests in test development without a ticket label referencing an open ticket
  • AC2: No stable tests in the job group
  • AC3: Jobs outside the responsibility area of "QSF-u" are at least covered in other open tickets on other teams

Suggestions

  • Talk with mgriessmeier about the "zkvm" scenarios, there are some which are green or soft-fail :)
  • Take a look in the test development job groups and move out stable QSF-u scenarios into the validation job groups
  • Search for open tickets to use as label on consistently failing scenarios, create tickets for others, or delete

History

#1 Updated by mgriessmeier almost 2 years ago

  • Target version changed from Milestone 25 to Milestone 30+

#2 Updated by SLindoMansilla over 1 year ago

  • Status changed from Workable to Rejected
  • Assignee set to mgriessmeier

Test development is exactly for that purpose.

#3 Updated by SLindoMansilla over 1 year ago

  • Estimated time set to 42.00 h

#4 Updated by okurz over 1 year ago

  • Subject changed from [functional][u] Ensure all tests in "test development" have a open ticket reference to [tools] Get rid of tests in "test development" that no one cares about (was: Ensure all tests in "test development" have a open ticket reference)
  • Status changed from Rejected to Workable
  • Assignee changed from mgriessmeier to okurz
  • Target version deleted (Milestone 30+)

SLindoMansilla wrote:

Test development is exactly for that purpose.

not sure what you mean. It seems like you did not read the ticket description. It seems you don't care about the test scenarios in "Test Development" so I propose I will – from administration point of view – delete scenarios based on my estimation where not useful anymore to achieve the goal. Speak up if you have a different idea.

#5 Updated by okurz over 1 year ago

  • Project changed from openQA Tests to openQA Project
  • Category changed from Enhancement to existing tests to Feature requests
  • Assignee deleted (okurz)

coolo do you think we should

  1. cleanup tests in "test development" as a single shot
  2. implement a feature or script or bot to do this recurringly as part of a) upstream openQA or b) downstream infrastructure helper tool
  3. completely ignore and live with the wasted ressources

#6 Updated by coolo over 1 year ago

From an admin point of view they are not wasting resources unless their priority is wrong or the job group is too greedy. Both can be fixed on shot - and I guess we can also eliminated jobs obviously old, but I for one can't judge and I can't think of any good metric.

So this goes back to having job group changes actually reviewed - but I really don't see this as part of the openQA backlog. Could be blocked by a Future 'Track job group yaml in git' feature in that backlog, but this one as written is really duty of those creating the entries IMO.

So I'd go and announce that you will remove every scenario there not having a reference october first and then do it one shot.

#7 Updated by okurz 9 months ago

  • Subject changed from [tools] Get rid of tests in "test development" that no one cares about (was: Ensure all tests in "test development" have a open ticket reference) to Get rid of tests in "test development" that no one cares about (was: Ensure all tests in "test development" have a open ticket reference)
  • Target version set to Ready

#8 Updated by okurz 8 months ago

  • Target version changed from Ready to future

Also available in: Atom PDF