Project

General

Profile

action #46988

openQA Project - coordination #39719: [saga][epic] Detection of "known failures" for stable tests, easy test results review and easy tracking of known issues

[qe-core][functional] Detect known bugs from system journal

Added by okurz over 2 years ago. Updated 7 months ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
Enhancement to existing tests
Target version:
SUSE QA tests - Milestone 30
Start date:
2018-04-16
Due date:
% Done:

0%

Estimated time:
Difficulty:

Description

Motivation

See parent ticket #39719 . As test reviewers we want to have less work to label jobs for known issues that can be detected from the system journal which we also upload

Acceptance criteria

  • AC1: We have an easy way to reference bugs as soft-failures by looking at matching patterns in the system journal

Suggestions

  • Take a look into the file lib/known_bugs.pm as introduced with 4fac2c92c
  • Extend pattern matching functionality to look into the system journal as well, maybe just use a matching pattern in the same file but used in the generic post_fail_hook that uploads the systemd journal, e.g. in lib/opensusebasetest.pm:problem_detection
  • Use example https://openqa.suse.de/tests/2429429#step/firefox/5 for detecting the firefox segfaulting on ppc64le
  • Use force_soft_failure

Related issues

Related to openQA Project - action #45011: Allow detection of known failures at the autoinst-log.txtWorkable2018-12-11

Related to openQA Tests - action #60992: [qe-core][functional][sporadic] generic problem with select_console / workqueue lockup messageResolved2019-12-12

History

#1 Updated by okurz over 2 years ago

  • Project changed from openQA Project to openQA Tests
  • Category set to Enhancement to existing tests
  • Status changed from New to Workable
  • Priority changed from Normal to High
  • Target version set to Milestone 23

#2 Updated by okurz over 2 years ago

  • Due date set to 2019-03-12

Setting later due-date to push off reminders in parent ticket.

#3 Updated by jorauch over 2 years ago

  • Status changed from Workable to In Progress
  • Assignee set to jorauch

Taking a look

#4 Updated by jorauch over 2 years ago

We should add a function to scan journalctl and call that in the post_fail_hook.

#5 Updated by jorauch over 2 years ago

We could use zluos idea of printing it to the serial log and reuse the existing serial scan for patterns
https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/6702

#6 Updated by jorauch over 2 years ago

  • Status changed from In Progress to Workable

Actually no progress so far due to other tasks

#7 Updated by jorauch over 2 years ago

Proof of concept:
http://pinky.arch.suse.de/tests/66#step/prepare_test_data/24
Injected a test error to known_bugs and a "echo 'test_bug'" to prepare_test_data

#8 Updated by jorauch over 2 years ago

  • Status changed from Workable to In Progress

#9 Updated by jorauch over 2 years ago

  • Status changed from In Progress to Workable

#10 Updated by jorauch over 2 years ago

  • Assignee deleted (jorauch)

Unassigning as discussed in the daily

#11 Updated by okurz over 2 years ago

  • Related to action #45011: Allow detection of known failures at the autoinst-log.txt added

#12 Updated by okurz over 2 years ago

  • Due date changed from 2019-03-12 to 2019-06-30

#13 Updated by dheidler about 2 years ago

  • Assignee set to dheidler

#14 Updated by dheidler about 2 years ago

  • Status changed from Workable to In Progress

#16 Updated by dheidler about 2 years ago

  • Status changed from In Progress to Feedback

#17 Updated by mgriessmeier about 2 years ago

  • Target version changed from Milestone 23 to Milestone 24

moving to M24

#18 Updated by SLindoMansilla about 2 years ago

  • Status changed from Feedback to Workable

#19 Updated by SLindoMansilla about 2 years ago

  • Status changed from Workable to Feedback

This should be picked up for grooming to define the specification of force_soft_failure. (okurz has to be present)

(I tried to change the status to new, but it is not shown)

#20 Updated by mgriessmeier about 2 years ago

  • Target version changed from Milestone 24 to Milestone 25

#21 Updated by szarate about 2 years ago

  • Status changed from Feedback to New
  • Assignee deleted (dheidler)

We can discuss this with Oliver sometime this week...

#22 Updated by szarate about 2 years ago

  • Status changed from New to Feedback
  • Assignee set to SLindoMansilla

Assigning to Sergio until he comes back

#23 Updated by szarate about 2 years ago

  • Priority changed from High to Normal

#24 Updated by riafarov almost 2 years ago

  • Due date changed from 2019-06-30 to 2019-08-06

Sorry for the noise, I've set due date to some time in future in order not to receive e-mail notifications about tickets with due dates (get them from parent epic I am tracking).

#25 Updated by mgriessmeier almost 2 years ago

  • Target version changed from Milestone 25 to Milestone 26

#26 Updated by SLindoMansilla almost 2 years ago

  • Priority changed from Normal to Urgent

To discuss on grooming

#27 Updated by riafarov almost 2 years ago

  • Due date changed from 2019-08-06 to 2019-12-31

Same as before, fixing due date not to receive emails about tickets with due dates.

#28 Updated by mgriessmeier almost 2 years ago

  • Status changed from Feedback to New
  • Target version changed from Milestone 26 to Milestone 27

#29 Updated by SLindoMansilla almost 2 years ago

  • Assignee deleted (SLindoMansilla)

To be refined

#30 Updated by mgriessmeier almost 2 years ago

  • Target version changed from Milestone 27 to Milestone 28

#31 Updated by SLindoMansilla over 1 year ago

  • Status changed from New to Workable
  • Priority changed from Urgent to Normal
  • Estimated time set to 42.00 h

Santi will provide the bug number.
This PR is needed before start working on this: https://github.com/os-autoinst/os-autoinst/pull/1136

#32 Updated by mgriessmeier over 1 year ago

  • Target version changed from Milestone 28 to Milestone 30

needs to be discussed offline

#33 Updated by okurz over 1 year ago

  • Due date changed from 2019-12-31 to 2020-12-31

#34 Updated by SLindoMansilla about 1 year ago

  • Status changed from Workable to New
  • Assignee set to SLindoMansilla

Maybe we can use an observer in the test distribution side.

#35 Updated by SLindoMansilla about 1 year ago

  • Due date deleted (2020-12-31)

#36 Updated by szarate about 1 year ago

  • Start date set to 2018-04-16

due to changes in a related task: #46988

#37 Updated by szarate about 1 year ago

  • Assignee deleted (SLindoMansilla)
  • Start date deleted (2019-02-01)
  • Estimated time deleted (42.00 h)

SLindoMansilla wrote:

Maybe we can use an observer in the test distribution side.

An observer pattern could be introduced on the test distribution side, but also piping the journal to some serial console aswell could help here

Unasigning Sergio, (and removing estimated time) perhaps it can be picked up or groomed later on even more

#38 Updated by szarate 10 months ago

  • Related to action #60992: [qe-core][functional][sporadic] generic problem with select_console / workqueue lockup message added

#39 Updated by tjyrinki_suse 7 months ago

  • Subject changed from [functional][u] Detect known bugs from system journal to [qe-core][functional] Detect known bugs from system journal

Also available in: Atom PDF