action #49370
open
Get rid of tests in "test development" that no one cares about (was: Ensure all tests in "test development" have a open ticket reference)
Added by okurz about 5 years ago.
Updated over 3 years ago.
Category:
Feature requests
Description
Motivation¶
There are quite some jobs in "Test Development" that actually finish successfully but are hardly looked at when they are in test development. Also there are reproducibly failing tests that just waste ressources. We should be "good citizens" and clean up.
Acceptance criteria¶
- AC1: No reproducibly failing tests in test development without a ticket label referencing an open ticket
- AC2: No stable tests in the job group
- AC3: Jobs outside the responsibility area of "QSF-u" are at least covered in other open tickets on other teams
Suggestions¶
- Talk with mgriessmeier about the "zkvm" scenarios, there are some which are green or soft-fail :)
- Take a look in the test development job groups and move out stable QSF-u scenarios into the validation job groups
- Search for open tickets to use as label on consistently failing scenarios, create tickets for others, or delete
- Target version changed from Milestone 25 to Milestone 30+
- Status changed from Workable to Rejected
- Assignee set to mgriessmeier
Test development is exactly for that purpose.
- Estimated time set to 42.00 h
- Subject changed from [functional][u] Ensure all tests in "test development" have a open ticket reference to [tools] Get rid of tests in "test development" that no one cares about (was: Ensure all tests in "test development" have a open ticket reference)
- Status changed from Rejected to Workable
- Assignee changed from mgriessmeier to okurz
- Target version deleted (
Milestone 30+)
SLindoMansilla wrote:
Test development is exactly for that purpose.
not sure what you mean. It seems like you did not read the ticket description. It seems you don't care about the test scenarios in "Test Development" so I propose I will – from administration point of view – delete scenarios based on my estimation where not useful anymore to achieve the goal. Speak up if you have a different idea.
- Project changed from openQA Tests to openQA Project
- Category changed from Enhancement to existing tests to Feature requests
- Assignee deleted (
okurz)
@coolo do you think we should
- cleanup tests in "test development" as a single shot
- implement a feature or script or bot to do this recurringly as part of a) upstream openQA or b) downstream infrastructure helper tool
- completely ignore and live with the wasted ressources
From an admin point of view they are not wasting resources unless their priority is wrong or the job group is too greedy. Both can be fixed on shot - and I guess we can also eliminated jobs obviously old, but I for one can't judge and I can't think of any good metric.
So this goes back to having job group changes actually reviewed - but I really don't see this as part of the openQA backlog. Could be blocked by a Future 'Track job group yaml in git' feature in that backlog, but this one as written is really duty of those creating the entries IMO.
So I'd go and announce that you will remove every scenario there not having a reference october first and then do it one shot.
- Subject changed from [tools] Get rid of tests in "test development" that no one cares about (was: Ensure all tests in "test development" have a open ticket reference) to Get rid of tests in "test development" that no one cares about (was: Ensure all tests in "test development" have a open ticket reference)
- Target version set to Ready
- Target version changed from Ready to future
Also available in: Atom
PDF