Project

General

Profile

action #46502

[tools] Support "record_info" in serial_failure_detection

Added by okurz over 2 years ago. Updated 2 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Enhancement to existing tests
Target version:
Start date:
2019-01-22
Due date:
2021-05-21
% Done:

0%

Estimated time:
Difficulty:

Description

Motivation

See https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/6589#issuecomment-456419001 . As the serial failure detection supports only "soft" and "hard" now we can fail the job or use "soft" but for this we want to always have a bugref so probably we should support to just set an info popup without altering the test result.

Acceptance criteria

  • AC1: serial issue detection can be used with a notice popup in test details but without altering the test result

Related issues

Related to openQA Tests - action #45530: [aarch64] system_workarounds.pm triggers lib/known_bugs serial detection which abort whole test suiteClosed2018-12-24

History

#1 Updated by okurz over 2 years ago

  • Related to action #45530: [aarch64] system_workarounds.pm triggers lib/known_bugs serial detection which abort whole test suite added

#2 Updated by szarate over 1 year ago

We don't want to paint it green

#3 Updated by okurz over 1 year ago

szarate Not sure what you want to tell with your statement. Of course I am with you to not just mark "known bugs" by record_info instead of a soft-fail. Have you followed https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/6589#issuecomment-456419001 and the whole discussion after that? The problem was with "noteworthy conditions" that can not have a workable bug behind where people also added record_soft_failure without bug references. Do you consider this feature worthwhile to follow up with or would you prefer an alternative?

#4 Updated by tjyrinki_suse 9 months ago

  • Subject changed from [functional][u] Support "record_info" in serial_failure_detection to [qe-core][functional] Support "record_info" in serial_failure_detection

#5 Updated by okurz 3 months ago

As I was asked for further details, let me try to explain: With serial_failure_detection I mean the logic called in https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/6589/files#diff-45750ef51e5e8360e7ef27b51599ba4b396c661c6ef82d1fd76cecd3ebec195cR27 and the reason why we created https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/6589 should explain the idea as well: soft-fail also according to http://open.qa/docs/ means "known, non-critical issue", so it should only be used if we have a bugref. only this way we can ensure it is "known". Otherwise jobs will end up all yellow but no one can do anything about it, leading to alarm-fatigue

#6 Updated by VANASTASIADIS 3 months ago

So, to clarify: the code in https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/6589/files#diff-45750ef51e5e8360e7ef27b51599ba4b396c661c6ef82d1fd76cecd3ebec195cR27 and https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/6589/files only changes what is being pushed to serial failures. serial failures is then passed to the backend (basetest.pm) via main.pm, and it's used to fail or softfail according to it's type attribute.

In my understanding, what is being asked is to add a third type option - apart from the currently allowed soft and hard - and then in the backend where the actual parsing, regex matching and (soft)failing takes place, account for that third type using record_info to notify users and then proceeding without failing or softfailing.

Is that understanding correct?

#7 Updated by VANASTASIADIS 3 months ago

  • Assignee set to VANASTASIADIS

#8 Updated by VANASTASIADIS 3 months ago

  • Status changed from New to In Progress

#9 Updated by okurz 3 months ago

VANASTASIADIS wrote:

In my understanding, what is being asked is to add a third type option - apart from the currently allowed soft and hard - and then in the backend where the actual parsing, regex matching and (soft)failing takes place, account for that third type using record_info to notify users and then proceeding without failing or softfailing.

Is that understanding correct?

that sounds like a good idea, yes.

#10 Updated by okurz 3 months ago

Given that within the tools team we have a different focus for now and are exceeding our WIP limit I suggest you unassign again. Can you?

#11 Updated by cdywan 3 months ago

Apparently there was no comment here following my chat with Bill:

  • There's the implementation in basetest.pm in os-autoinst which Bill would have to tackle from a Tools pov
  • There's the distri part where the desired behavior can be implemented (qe-core)
  • Why do we distinguish hardhard vs. softfail vs. "not a failure, not a bug" the point of that third, new option seems unclear
  • We need a category that only provides more data for debugging even if by default it will not be used/needed

#12 Updated by okurz 3 months ago

  • Subject changed from [qe-core][functional] Support "record_info" in serial_failure_detection to [tools] Support "record_info" in serial_failure_detection
  • Target version changed from future to Ready

Yes, I understand that now that this is related to os-autoinst. Then it's good. So we will take it over to the SUSE QE Tools team backlog.

#13 Updated by openqa_review 3 months ago

  • Due date set to 2021-05-21

Setting due date based on mean cycle time of SUSE QE Tools

#14 Updated by VANASTASIADIS 2 months ago

  • Status changed from In Progress to Resolved

#15 Updated by okurz 2 months ago

thank you!

Also available in: Atom PDF