Project

General

Profile

Actions

action #45011

open

Allow detection of known failures at the autoinst-log.txt

Added by szarate almost 6 years ago. Updated almost 4 years ago.

Status:
Workable
Priority:
Low
Assignee:
-
Category:
Feature requests
Target version:
Start date:
2018-12-11
Due date:
% Done:

0%

Estimated time:

Description

PRs https://github.com/os-autoinst/os-autoinst/pull/1014 and https://github.com/os-autoinst/os-autoinst/pull/932 laid the fundation to process failures that were happening outside the test scope, but they don't cover autoinst-log.txt for example.

I created https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/6315 as a starting point when I stumbled across the problem, time to put it into a real ticket.

User Story

As an Test engineer I would like to have the capability of detecting problems in the worker host or os-autoinst so that the test can be cancelled if necesary

AC:

  • Provide a mechanism to match on regex in autoinst-log.txt based on patterns defined in the test distribution (preferably not in the main.pm or test suites per se, but it's own domain)
  • Provide patterns defined in os-autoinst for backend specific stuff, e.g. the "key event queue full"-thingy -> look for that string in os-autoinst for existing code to handle that
  • Enhance existing documentation

Related issues 5 (3 open2 closed)

Related to openQA Tests - action #46988: [qe-core][functional] Detect known bugs from system journalNew2018-04-16

Actions
Related to openQA Project - action #52235: isotovideo -d brokenResolveddheidler2019-05-28

Actions
Related to openQA Project - coordination #39719: [saga][epic] Detection of "known failures" for stable tests, easy test results review and easy tracking of known issuesResolvedokurz2018-05-23

Actions
Related to openQA Project - coordination #102906: [saga][epic] Increased stability of tests with less "known failures", known incompletes handled automatically within openQANew2018-04-16

Actions
Related to openQA Project - coordination #102915: [saga][epic] Automated classification of failuresNew2015-12-18

Actions
Actions #1

Updated by szarate almost 6 years ago

  • Subject changed from [tools]][u] Allow detection of known failures at the autoinst-log.txt to [tools][u] Allow detection of known failures at the autoinst-log.txt
Actions #2

Updated by szarate almost 6 years ago

  • Related to coordination #39719: [saga][epic] Detection of "known failures" for stable tests, easy test results review and easy tracking of known issues added
Actions #3

Updated by szarate almost 6 years ago

  • Target version set to Milestone 21+
Actions #4

Updated by okurz almost 6 years ago

  • Subject changed from [tools][u] Allow detection of known failures at the autoinst-log.txt to [tools][functional][u] Allow detection of known failures at the autoinst-log.txt
  • Target version changed from Milestone 21+ to Milestone 23
Actions #5

Updated by agraul almost 6 years ago

  • Related to deleted (coordination #39719: [saga][epic] Detection of "known failures" for stable tests, easy test results review and easy tracking of known issues)
Actions #6

Updated by agraul almost 6 years ago

  • Blocks coordination #39719: [saga][epic] Detection of "known failures" for stable tests, easy test results review and easy tracking of known issues added
Actions #7

Updated by szarate over 5 years ago

  • Status changed from New to Workable
Actions #8

Updated by okurz over 5 years ago

  • Related to action #46988: [qe-core][functional] Detect known bugs from system journal added
Actions #9

Updated by SLindoMansilla over 5 years ago

Hi, foursixnine

It seems that #46988 was a duplicate of this ticket, but it was already picked. Could closed this as duplicate (merging content)?

Actions #10

Updated by szarate over 5 years ago

Sergio, that one talks about journal logs, which come from the SUT, this ticket talks about isotovideo generated logs (backend), which would be an enhancement, and would require different processing on autoinst-log.txt as the messages are different, and do not come from the serial console of the sut.

Actions #11

Updated by mgriessmeier over 5 years ago

  • Target version changed from Milestone 23 to Milestone 24

moving to M24

Actions #12

Updated by dheidler over 5 years ago

Would something like create_list_of_serial_failures in lib/known_bugs.pm be feasible?

Actions #13

Updated by dheidler over 5 years ago

  • Status changed from Workable to In Progress
  • Assignee set to dheidler
Actions #14

Updated by dheidler over 5 years ago

  • Status changed from In Progress to Feedback
Actions #15

Updated by mgriessmeier over 5 years ago

  • Target version changed from Milestone 24 to Milestone 25
Actions #16

Updated by szarate over 5 years ago

  • Status changed from Feedback to Resolved
Actions #17

Updated by okurz over 5 years ago

Actions #18

Updated by okurz over 5 years ago

  • Status changed from Resolved to Workable

There was some fallout, see #52235 and #52247 . I guess we should continue here.

As coolo suggested:

"there is a reason the worker writes this file - because we add additional infos to it. […] Still having 2 processes writing the same file is very dangerous. It is just stupid to open a file and then start another process to write to it. I veto this insanity - because in a couple of days you guys will file tickets for truncated log files and complain. I repeat what I said: if isotovideo wants to catch log output, this should be a Mojo::Log subclass that does not write. And leave autoinst-log.txt alone."

Actions #19

Updated by dheidler over 5 years ago

  • Assignee deleted (dheidler)
Actions #20

Updated by mgriessmeier over 5 years ago

  • Priority changed from Normal to High
  • Target version changed from Milestone 25 to Milestone 26
Actions #21

Updated by mgriessmeier about 5 years ago

  • Subject changed from [tools][functional][u] Allow detection of known failures at the autoinst-log.txt to [tools] Allow detection of known failures at the autoinst-log.txt
  • Target version changed from Milestone 26 to Milestone 27

In the u-team we don't have the resources to look into this right now, since it apparently requires a lot of effort to check backend specific stuff
feel free to discuss with me in person - for now removing u-tag

Actions #22

Updated by mgriessmeier about 5 years ago

  • Target version changed from Milestone 27 to Milestone 29
Actions #23

Updated by coolo almost 5 years ago

  • Priority changed from High to Normal
  • Target version deleted (Milestone 29)

Not picking into tools backlog

Actions #24

Updated by okurz almost 5 years ago

My approach, focussed on incompletes, are based on the scripts openqa-monitor-incompletes and openqa-label-known-issues triggered in a gitlab CI pipeline in https://gitlab.suse.de/openqa/auto-review/pipelines

Actions #25

Updated by okurz about 4 years ago

  • Subject changed from [tools] Allow detection of known failures at the autoinst-log.txt to Allow detection of known failures at the autoinst-log.txt
  • Priority changed from Normal to Low
  • Target version set to Ready

It's in scope for QA tools with the target version, removing additional "[tools]" tag.

Actions #26

Updated by okurz almost 4 years ago

  • Target version changed from Ready to future
Actions #27

Updated by okurz almost 3 years ago

  • Blocks deleted (coordination #39719: [saga][epic] Detection of "known failures" for stable tests, easy test results review and easy tracking of known issues)
Actions #28

Updated by okurz almost 3 years ago

  • Related to coordination #39719: [saga][epic] Detection of "known failures" for stable tests, easy test results review and easy tracking of known issues added
Actions #29

Updated by okurz almost 3 years ago

  • Related to coordination #102906: [saga][epic] Increased stability of tests with less "known failures", known incompletes handled automatically within openQA added
Actions #30

Updated by okurz almost 3 years ago

Actions

Also available in: Atom PDF