Added by livdywan over 2 years ago. Updated over 2 years ago.
Description
The Ratio of multi-machine tests is alerting.
ratio_mm_failed 5.360
The alert turned green again for now. Maybe the investigation work on #101271 caused a higher fail-ratio. Still, this should be investigated in more detail, e.g. take a look for the specific reason why multi-machine tests failed in the past two days.
The alert was firing today again (but has now already went off again).
Looking through the failed jobs.
Setting due date based on mean cycle time of SUSE QE Tools
So far i'm not seeing any patterns, the reasons for why the tests failed are very diverse. Only recent-ish change is that there is more active test development for one of our newer products.
I'll take a closer look at two 24 hour time frames, 2 days ago vs 30 days ago (when the ratio was around 3).
Checked about 200 multi machine jobs and the only noticeable change has really been new products being tested (and the tests being in active development). So i'm fairly certain there have been no larger underlying issues here, and we should (at least temporarily) consider increasing the alert value.