Project

General

Profile

action #113408

bot-ng trigger test two times for incidents

Added by asmorodskyi 7 months ago. Updated 7 months ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
Feature requests
Target version:
Start date:
2022-07-08
Due date:
% Done:

0%

Estimated time:
Difficulty:

Description

Observations

Most of incidents tested several times , mostly two times in some cases it can even be 3 times .
Example :
https://openqa.suse.de/tests/9010214
https://openqa.suse.de/tests/9009883

This is waste of resources because in this certain case only change between two runs is addition of RRID which brings 0 functionality so there is no good reason to run tests second time

Acceptance criteria

  • find a way to skip triggers for incidents w/o RRID

History

#1 Updated by asmorodskyi 7 months ago

  • Priority changed from Normal to High

increasing priority of the ticket , considering this critical because incidents also triggering Public Cloud tests so we loosing some real money for nothing

#2 Updated by jbaier_cz 7 months ago

I would be also nice to include at least the relevant part of the discussion: https://suse.slack.com/archives/C02CANHLANP/p1657279161559939

Long story short, there is nothing wrong with the bot-ng; this can be probably seen as a feature request to not schedule Public Cloud test for incidents without release request.

#3 Updated by okurz 7 months ago

  • Category set to Feature requests
  • Priority changed from High to Normal
  • Target version set to future

If Jan wouldn't have stated it, I would have it :) please include the relevant details from the discussion

#4 Updated by asmorodskyi 7 months ago

okurz wrote:

If Jan wouldn't have stated it, I would have it :) please include the relevant details from the discussion

Jan already gave URL to the discussion , from my POV most relevant part of it is that currently triggering happens when incident appears in the repo ignoring if it is ready for release or not ( no check for RRID ) which cause more than one triggering per incident because sometimes developers may add some things in later phases .

I personally disagree with treating this problem as Public Cloud specific one but again for me personally most important part if it will be fixed for Public Cloud and when

#5 Updated by osukup 7 months ago

It's feature and important ... It triggers job when repo is changed. --> see REPOHASH variable in vars.json

Start testing incidents in staging area was decided years ago as part of 'move left initiative'.

#6 Updated by coolo 7 months ago

Not sure "move left initiative" was "years ago", but 0.5 years is "years ago" too, I take it.

And let's ask this: has any openQA result of those left moving ever stopped incident managers to create a RR?

#7 Updated by okurz 7 months ago

coolo wrote:

And let's ask this: has any openQA result of those left moving ever stopped incident managers to create a RR?

Likely very hard to know if people have not opened an RR for any reason. What's your point?

With my current, limited understanding I see openQA tests in general as rather cheap with the exception of tests in public cloud so I agree with Jan when he states "feature request to not schedule Public Cloud test for incidents without release request."

#8 Updated by coolo 7 months ago

My point is to evalute cost and benefit of running (public cloud) tests as they are. If noone reviews them they are just wasted (money for public cloud, energy for the rest)

#9 Updated by asmorodskyi 7 months ago

osukup wrote:

It's feature and important ... It triggers job when repo is changed. --> see REPOHASH variable in vars.json

Start testing incidents in staging area was decided years ago as part of 'move left initiative'.

AFAIK at least in classical https://en.wikipedia.org/wiki/Shift-left_testing initiative together with moving left approach you also need to switch from more expensive testing approach to faster and cheaper one . And if you "moving left" but same test set as you have "on the right" you doing something wrong . And my idea is exactly about that - define smaller and bigger test suites and run smaller one on every commit blindly and bigger one only after we got RRID

#10 Updated by okurz 7 months ago

Sure, I agree. As a first step I recommend to remove public cloud tests that you don't consider critical from the current incident tests that retrigger that often.

Also available in: Atom PDF