QA (public) » openQA Project (public)

I would be also nice to include at least the relevant part of the discussion: https://suse.slack.com/archives/C02CANHLANP/p1657279161559939

Long story short, there is nothing wrong with the bot-ng; this can be probably seen as a feature request to not schedule Public Cloud test for incidents without release request.

okurz wrote:

If Jan wouldn't have stated it, I would have it :) please include the relevant details from the discussion

Jan already gave URL to the discussion , from my POV most relevant part of it is that currently triggering happens when incident appears in the repo ignoring if it is ready for release or not ( no check for RRID ) which cause more than one triggering per incident because sometimes developers may add some things in later phases .

I personally disagree with treating this problem as Public Cloud specific one but again for me personally most important part if it will be fixed for Public Cloud and when

It's feature and important ... It triggers job when repo is changed. --> see REPOHASH variable in vars.json

Start testing incidents in staging area was decided years ago as part of 'move left initiative'.

Not sure "move left initiative" was "years ago", but 0.5 years is "years ago" too, I take it.

And let's ask this: has any openQA result of those left moving ever stopped incident managers to create a RR?

coolo wrote:

And let's ask this: has any openQA result of those left moving ever stopped incident managers to create a RR?

Likely very hard to know if people have not opened an RR for any reason. What's your point?

With my current, limited understanding I see openQA tests in general as rather cheap with the exception of tests in public cloud so I agree with Jan when he states "feature request to not schedule Public Cloud test for incidents without release request."

My point is to evalute cost and benefit of running (public cloud) tests as they are. If noone reviews them they are just wasted (money for public cloud, energy for the rest)

osukup wrote:

It's feature and important ... It triggers job when repo is changed. --> see REPOHASH variable in vars.json

Start testing incidents in staging area was decided years ago as part of 'move left initiative'.

AFAIK at least in classical https://en.wikipedia.org/wiki/Shift-left_testing initiative together with moving left approach you also need to switch from more expensive testing approach to faster and cheaper one . And if you "moving left" but same test set as you have "on the right" you doing something wrong . And my idea is exactly about that - define smaller and bigger test suites and run smaller one on every commit blindly and bigger one only after we got RRID

Sure, I agree. As a first step I recommend to remove public cloud tests that you don't consider critical from the current incident tests that retrigger that often.