Project

General

Profile

Actions
Copy link

action #69148

closed

regression: output of self-tests on circle CI is missing from log view, "tests" tab shows nothing (due to timeout)

Added by okurz almost 4 years ago. Updated almost 4 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
livdywan
Category:
Regressions/Crashes
Target version:
Ready
Start date:
2020-07-20
Due date:
% Done:

0%

Estimated time:

Description

Observation

https://app.circleci.com/pipelines/github/os-autoinst/openQA/3546/workflows/28188eaa-92c1-41f7-b32f-739f7c1fe038/jobs/33798/steps shows

[10:34:59] t/ui/10-tests_overview.t ............... ok   130578 ms ( 0.21 usr  0.02 sys + 29.77 cusr 87.83 csys = 117.83 CPU)
[10:37:10] t/ui/12-needle-edit.t .................. 3/? [10:37:10] t/ui/12-needle-edit.t .................. 4/? [10:37:10] t/ui/12-needle-edit.t .................. 5/? [10:37:10] t/ui/12-needle-edit.t .................. 6/?                                                          [10:37:10] t/ui/12-needle-edit.t .................. Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/6 subtests 
[10:41:45] t/ui/13-admin-no-login.t ............... ok   123534 ms ( 0.03 usr  0.01 sys + 27.36 cusr 89.84 csys = 117.24 CPU)

with no details which we had some time ago, probably around 2020-03 and earlier. The tests tab does not show any details either, probably because the job timed out

Acceptance criteria

  • AC1: The circleCI log not only shows the summary but details about the failure, i.e. every failure that was reported, though not the verbose test output

Problem

Clearly a regression as this worked in the past. Maybe due to our change to add the "--merge" parameter on prove?

Suggestions

  • Research in past logs to see if we can still find references of old jobs
  • Check how tests look locally
  • Play around with parameters, e.g. "--merge" which one we need and how they behave
  • Fix it :)
Actions
Copy link
 #1

Updated by okurz almost 4 years ago

  • Description updated (diff)
  • Status changed from New to Workable
Actions
Copy link
 #2

Updated by livdywan almost 4 years ago

  • Status changed from Workable to In Progress
  • Assignee set to livdywan

Nice to see I'm not the only one to see this anymore (I had been looking into this a while back).

A way to reproduce it easily: set the timeout to something small like 8 in the Makefile and run at least two short tests. Observe that there is no testplan regardless of results.

I'll attempt to fix it by making Prove aware of the fact that timeout kills it before it can record the results using --trap and -s SIGINT -k 5.

Actions
Copy link
 #4

Updated by okurz almost 4 years ago

PR merged, I want proof that it actually works though, so let some tests timeout on circle CI :)

Actions
Copy link
 #5

Updated by livdywan almost 4 years ago

  • Status changed from In Progress to Feedback
Actions
Copy link
 #6

Updated by okurz almost 4 years ago

  • Status changed from Feedback to Resolved

@cdywan did you do something for an actual proof? I guess not but 2 months have passed and we did not come back to this ticket so let's assume it's actually fixed for good

Actions
Copy link

Also available in: Atom PDF