QA » openQA Project

Observation¶

openQA test in scenario opensuse-15.1-DVD-x86_64-update_Leap_15.0_gnome@64bit fails in
gnucash

So I was just staring at this gnucash failure without any hint what went wrong. So what I had to do is

• to go to the previous results
• observe that the last run was green
• click on the last run and manually navigate to gnucash
• have two browser windows open to visually compare the difference
• see that gnucash seems to behave the same but still fails. Maybe some subtle font change somewhere?
• look at the source code trying to understand what the test does
• can't understand why the code didn't look for a needle. wondering if the code I am looking at is what actually ran
• open a terminal to look at the git history of gnucash.pm
• see that okurz reverted a previous change earlier today
• restart all tests that use gnucash

I think there are opportunities to reduce the amount of manual steps involved, for example

• tell me that the previous run was good and provide a link I can click on to compare, jump to the relevant test/needle right away
• warn me that the code I am looking at is not what was run. provide link to show the diff

Bonus points

• show links/diffs to other runs in the same build that also have gnucash failures to proof it's always the same
• links/diffs to gnucash failures in other job groups. I suppose in this case the problem was spotted in TW already
• single click solution to restart all affected tests at once