Project

General

Profile

action #71464

Show output of failed tests within circleci log view, like on local prove calls, not only "Tests" tab

Added by okurz 4 months ago. Updated 3 months ago.

Status:
Resolved
Priority:
Low
Assignee:
Category:
Feature requests
Target version:
Start date:
2020-09-17
Due date:
% Done:

0%

Estimated time:
Difficulty:

Description

Observation

https://app.circleci.com/pipelines/github/os-autoinst/openQA/4187/workflows/3490f55a-1aea-4aad-860e-2d411fb95e1e/jobs/40008/steps shows failed tests like this:

[12:12:10] t/07-api_keys.t ........................................... ok     8238 ms ( 0.00 usr  0.00 sys +  7.54 cusr  0.40 csys =  7.94 CPU)
[12:12:18] t/09-job_clone.t .......................................... 1/?                                                                              [12:12:18] t/09-job_clone.t .......................................... Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/16 subtests 
[12:12:36] t/10-jobs.t ............................................... ok    19702 ms ( 0.07 usr  0.00 sys + 17.99 cusr  0.67 csys = 18.73 CPU)
…

Test Summary Report
-------------------
t/09-job_clone.t                                        (Wstat: 256 Tests: 16 Failed: 1)
  Failed test:  7
  Non-zero exit status: 1
t/20-stale-job-detection.t                              (Wstat: 256 Tests: 3 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
t/40-script_openqa-clone-custom-git-refspec.t           (Wstat: 0 Tests: 43 Failed: 0)
  TODO passed:   30, 32
Files=76, Tests=2120, 1141.99 wallclock secs ( 5.12 usr  0.42 sys + 900.28 cusr 101.21 csys = 1007.03 CPU)
Result: FAIL
make[2]: *** [Makefile:174: test-unit-and-integration] Error 1
make[2]: Leaving directory '/home/squamata/project'
make[1]: *** [Makefile:169: test-with-database] Error 2
make[1]: Leaving directory '/home/squamata/project'
make: *** [Makefile:137: test-t] Error 2

Exited with code exit status 2

CircleCI received exit code 2

so we see which test failed and also in the summary we get the information which individual test step failed but we do not have the output in particular. The tab
https://app.circleci.com/pipelines/github/os-autoinst/openQA/4187/workflows/3490f55a-1aea-4aad-860e-2d411fb95e1e/jobs/40008/tests shows the details, like:

not ok 7 - cannot clone again
[debug] [pid:202] Duplicating jobs: {
  99926 => {
    chained_children => [],
    chained_parents => [],
    directly_chained_children => [],
    directly_chained_parents => [],
    is_parent_or_initial_job => 1,
    ok => 0,
    parallel_children => [],
    parallel_parents => [],
    state => "done",
  },
}
[warn] [pid:202] Duplication rolled back after error: Unable to clone: Job 99926 already has clone 99982

Expected result

We should be able to see the complete output in case of failed tests in the log window of circleci as well.

Suggestion

https://github.com/os-autoinst/openQA/commit/f3b5dd95cc8b61b6a29e7502b877f8ee9d78059d already tried to address this but it seems this does not cover all cases. Needs to be reproduced and debugged.


Related issues

Related to openQA Project - action #66664: circleci jobs do not retry, are aborted with "Too long with no output (exceeded 10m0s): context deadline exceeded"Resolved2020-05-09

Related to openQA Project - coordination #71926: [epic] t/14-grutasks.t takes multiple minutes within circleCI, only 6s locally (but errors but still succeeds?), and no log output visible in circleCIWorkable2020-09-26

Related to openQA Project - action #71845: tests fail in circleci with "Non-zero wait status: <non-negative number>" but "All <X> subtests passed"Resolved2020-09-24

History

#1 Updated by okurz 4 months ago

  • Related to action #66664: circleci jobs do not retry, are aborted with "Too long with no output (exceeded 10m0s): context deadline exceeded" added

#2 Updated by okurz 4 months ago

  • Status changed from Workable to In Progress
  • Assignee set to okurz

#3 Updated by okurz 4 months ago

  • Related to coordination #71926: [epic] t/14-grutasks.t takes multiple minutes within circleCI, only 6s locally (but errors but still succeeds?), and no log output visible in circleCI added

#4 Updated by okurz 4 months ago

  • Related to action #71845: tests fail in circleci with "Non-zero wait status: <non-negative number>" but "All <X> subtests passed" added

#5 Updated by okurz 4 months ago

  • Status changed from In Progress to Feedback

#6 Updated by okurz 3 months ago

  • Status changed from Feedback to Resolved

PR merged and output is back \o/

Also available in: Atom PDF