action #113408
open
bot-ng trigger test two times for incidents
Added by asmorodskyi almost 2 years ago.
Updated almost 2 years ago.
Category:
Feature requests
Description
Observations¶
Most of incidents tested several times , mostly two times in some cases it can even be 3 times .
Example :
https://openqa.suse.de/tests/9010214
https://openqa.suse.de/tests/9009883
This is waste of resources because in this certain case only change between two runs is addition of RRID which brings 0 functionality so there is no good reason to run tests second time
Acceptance criteria¶
- find a way to skip triggers for incidents w/o RRID
- Priority changed from Normal to High
increasing priority of the ticket , considering this critical because incidents also triggering Public Cloud tests so we loosing some real money for nothing
I would be also nice to include at least the relevant part of the discussion: https://suse.slack.com/archives/C02CANHLANP/p1657279161559939
Long story short, there is nothing wrong with the bot-ng; this can be probably seen as a feature request to not schedule Public Cloud test for incidents without release request.
- Category set to Feature requests
- Priority changed from High to Normal
- Target version set to future
If Jan wouldn't have stated it, I would have it :) please include the relevant details from the discussion
okurz wrote:
If Jan wouldn't have stated it, I would have it :) please include the relevant details from the discussion
Jan already gave URL to the discussion , from my POV most relevant part of it is that currently triggering happens when incident appears in the repo ignoring if it is ready for release or not ( no check for RRID ) which cause more than one triggering per incident because sometimes developers may add some things in later phases .
I personally disagree with treating this problem as Public Cloud specific one but again for me personally most important part if it will be fixed for Public Cloud and when
It's feature and important ... It triggers job when repo is changed. --> see REPOHASH variable in vars.json
Start testing incidents in staging area was decided years ago as part of 'move left initiative'.
Not sure "move left initiative" was "years ago", but 0.5 years is "years ago" too, I take it.
And let's ask this: has any openQA result of those left moving ever stopped incident managers to create a RR?
coolo wrote:
And let's ask this: has any openQA result of those left moving ever stopped incident managers to create a RR?
Likely very hard to know if people have not opened an RR for any reason. What's your point?
With my current, limited understanding I see openQA tests in general as rather cheap with the exception of tests in public cloud so I agree with Jan when he states "feature request to not schedule Public Cloud test for incidents without release request."
My point is to evalute cost and benefit of running (public cloud) tests as they are. If noone reviews them they are just wasted (money for public cloud, energy for the rest)
osukup wrote:
It's feature and important ... It triggers job when repo is changed. --> see REPOHASH variable in vars.json
Start testing incidents in staging area was decided years ago as part of 'move left initiative'.
AFAIK at least in classical https://en.wikipedia.org/wiki/Shift-left_testing initiative together with moving left approach you also need to switch from more expensive testing approach to faster and cheaper one . And if you "moving left" but same test set as you have "on the right" you doing something wrong . And my idea is exactly about that - define smaller and bigger test suites and run smaller one on every commit blindly and bigger one only after we got RRID
Sure, I agree. As a first step I recommend to remove public cloud tests that you don't consider critical from the current incident tests that retrigger that often.
Also available in: Atom
PDF