Project

General

Profile

Actions

coordination #102912

open

coordination #102906: [saga][epic] Increased stability of tests with less "known failures", known incompletes handled automatically within openQA

[epic] Simplify investigation of job failures - 2nd

Added by okurz about 3 years ago. Updated 5 months ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
Feature requests
Target version:
QA (public, currently private due to #173521) - future
Start date:
2018-04-16
Due date:
% Done:

12%

Estimated time:
(Total: 0.00 h)

Description

motivation

Make job failure investigation easier to save time and ensure we do not miss failures

ideas

  • all package changes, e.g. save rpm -qa in file and provide diff and/or changelog for the worker and the SUT. For openSUSE e.g. read out changelog diff from https://openqa.opensuse.org/snapshot-changes/opensuse/Tumbleweed/ , for SLE from http://xcdchk.suse.de/

  • diff of test schedule

  • if best needle candidate matches 0% it is most likely not a trivial needle issue

  • In settings table mark origin of settings and changed settings, e.g. for setting "foo" instead of the table row "foo | 1" one could have

    • "foo | 1 (testsuites table)" when the settings comes from the test suites database table, e.g. compared to job templates, machines, etc. . This would also help when we allow even more sources for settings, e.g. load job templates from test distributions in parallel to database tables
    • update the settings table from vars.json after job run to included changes but then show which settings changed since the job was initially created
    • "foo | 1 (+)" when the setting is new in the scenario, with the table row and/or "(+)" in green (as in common colored diffs) and on hover it shows the explanation that this was added, linked to the commit, showing which job it compares against
    • "foo | 1 (<->)" or similar when the setting changed against "last good" where it was e.g. 0, with "(<->)" being a link to the "last good" job, with the table row in different color

Subtasks 8 (7 open1 closed)

openQA Tests (public) - action #46988: [qe-core][functional] Detect known bugs from system journalNew2018-04-16

Actions
action #62441: openqa-worker systemd service can timeout when stoppingNew2020-01-21

Actions
action #63065: [gsoc] dynamic detection of error conditions from test resultsResolvedokurz2020-02-04

Actions
action #64935: Provide test results of "related results" on job detailsWorkable2020-03-27

Actions
action #69319: Surface relevant versions used by job workerNew2020-07-24

Actions
action #99066: ci: Use a git commit message style checker in more reposNew

Actions
action #103035: Present changes between packages on openQA worker machines in "investigation" - enable in production on o3New

Actions
action #163475: On-demand diff'ing of any result+log file from failed openQA jobsNew

Actions

Related issues 1 (0 open1 closed)

Copied from openQA Project (public) - coordination #19720: [epic] Simplify investigation of job failuresResolvedokurz2019-12-17

Actions
Actions #1

Updated by okurz about 3 years ago

Actions #2

Updated by okurz 5 months ago

  • Subtask #163475 added
Actions

Also available in: Atom PDF