Project

General

Profile

coordination #41057

coordination #102906: [saga][epic] Increased stability of tests with less "known failures", known incompletes handled automatically within openQA

[epic] Make reviewing results easier

Added by lnussel about 3 years ago. Updated 9 days ago.

Status:
Blocked
Priority:
Normal
Assignee:
Category:
Feature requests
Target version:
Start date:
2018-04-16
Due date:
% Done:

33%

Estimated time:
(Total: 0.00 h)
Difficulty:
hard

Description

Observation

openQA test in scenario opensuse-15.1-DVD-x86_64-update_Leap_15.0_gnome@64bit fails in
gnucash

So I was just staring at this gnucash failure without any hint what went wrong. So what I had to do is

  • to go to the previous results
  • observe that the last run was green
  • click on the last run and manually navigate to gnucash
  • have two browser windows open to visually compare the difference
  • see that gnucash seems to behave the same but still fails. Maybe some subtle font change somewhere?
  • look at the source code trying to understand what the test does
  • can't understand why the code didn't look for a needle. wondering if the code I am looking at is what actually ran
  • open a terminal to look at the git history of gnucash.pm
  • see that okurz reverted a previous change earlier today
  • restart all tests that use gnucash

I think there are opportunities to reduce the amount of manual steps involved, for example

  • tell me that the previous run was good and provide a link I can click on to compare, jump to the relevant test/needle right away
  • warn me that the code I am looking at is not what was run. provide link to show the diff

Bonus points

  • show links/diffs to other runs in the same build that also have gnucash failures to proof it's always the same
  • links/diffs to gnucash failures in other job groups. I suppose in this case the problem was spotted in TW already
  • single click solution to restart all affected tests at once

Subtasks

action #35017: Show more test failure details with the "test failed" text thumbnail popupsWorkable

action #91601: Add "return to top" button on openQA pages, e.g. job details, index, group overviewResolvedkraih

action #91602: Inform users about keyboard shortcuts in openQAWorkable


Related issues

Blocked by openQA Project - coordination #19720: [epic] Simplify investigation of job failuresResolved2019-12-17

History

#1 Updated by lnussel about 3 years ago

  • Project changed from openQA Tests to openQA Project
  • Description updated (diff)
  • Category changed from Bugs in existing tests to 124

#2 Updated by coolo about 3 years ago

  • Subject changed from test fails in gnucash to Make reviewing results easier

#3 Updated by coolo about 3 years ago

  • Subject changed from Make reviewing results easier to [EPIC] Make reviewing results easier
  • Difficulty set to hard

#4 Updated by okurz over 2 years ago

  • Category changed from 124 to Feature requests

#5 Updated by okurz about 2 years ago

#6 Updated by okurz over 1 year ago

  • Subject changed from [EPIC] Make reviewing results easier to [epic] Make reviewing results easier

#7 Updated by okurz over 1 year ago

  • Status changed from New to Blocked
  • Assignee set to okurz
  • Target version set to Ready

waiting for results of #19720 first

#8 Updated by szarate about 1 year ago

  • Tracker changed from action to coordination
  • Status changed from Blocked to New
  • Difficulty deleted (hard)

#10 Updated by okurz about 1 year ago

  • Status changed from New to Blocked
  • Difficulty set to hard

#11 Updated by okurz about 1 year ago

  • Parent task set to #39719

#12 Updated by cdywan 5 months ago

Can you confirm what this was blocking on? I see no reference and the subtask is in Future.

#13 Updated by okurz 5 months ago

In #41057#note-7 I made the statement that we should have #19720 first before continuing. It's the related issue that is also in "Ready". The subtask is in future, true.

#14 Updated by cdywan 5 months ago

#15 Updated by cdywan 5 months ago

#16 Updated by okurz 9 days ago

  • Parent task changed from #39719 to #102906

Also available in: Atom PDF