Project

General

Profile

Actions
Copy link

action #73162

closed

t/01-test-utilities.t fails in "test would have failed"

Added by okurz about 4 years ago. Updated about 4 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
okurz
Category:
Regressions/Crashes
Target version:
Ready
Start date:
2020-10-09
Due date:
2020-10-25
% Done:

0%

Estimated time:

Description

Observation

[ 1108s]     #   Failed test 'test would have failed'
[ 1108s]     #   at ./t/01-test-utilities.t line 47.
[ 1108s] 
[ 1108s]     #   Failed test 'manual termination via stop_service does not trigger _fail_and_exit'
[ 1108s]     #   at ./t/01-test-utilities.t line 58.
[ 1108s]     #          got: '1'
[ 1108s]     #     expected: '0'
[ 1108s]     # Looks like you failed 2 tests of 5.
[ 1108s] 
[ 1108s] #   Failed test 'warnings in sub processes are fatal test failures'
[ 1108s] #   at ./t/01-test-utilities.t line 59.
[ 1108s] # Looks like you failed 1 test of 1.
[ 1108s] ./t/01-test-utilities.t ..................................... 
[ 1108s] # Subtest: warnings in sub processes are fatal test failures
[ 1108s]     # PID of test-process: 5808
[ 1108s]     ok 1 - warning logged
[ 1108s]     not ok 2 - test would have failed
[ 1108s]     # PID of test-process: 5813
[ 1108s]     ok 3 - message of test failure
[ 1108s]     ok 4 - exit code of test failure is non-zero
[ 1108s]     not ok 5 - manual termination via stop_service does not trigger _fail_and_exit
[ 1108s]     1..5
[ 1108s] not ok 1 - warnings in sub processes are fatal test failures

in https://build.opensuse.org/build/devel:openQA:TestGithub:PR-3456/openSUSE_Factory/x86_64/openQA/_log from https://github.com/os-autoinst/openQA/pull/3456

Checking reproducibility…

EDIT: happened again, e.g. in https://build.opensuse.org/package/live_build_log/devel:openQA/openQA/openSUSE_Factory/x86_64 even though at time of writing it was fine again

Acceptance criteria

  • AC1: t/01-test-utilities.t does not fail in OBS checks reproducibly (either fixed or excluded)

Suggestions

  • Wait until this happens again and take a look in the log for details
  • Try to reproduce failure in test OBS project
  • Fix or exclude
Actions
Copy link
 #1

Updated by okurz about 4 years ago

  • Due date set to 2020-10-18
  • Status changed from In Progress to Feedback
  • Priority changed from Normal to Low

I don't see it in build.opensuse.org/project/show/devel:openQA

Actions
Copy link
 #2

Updated by okurz about 4 years ago

  • Status changed from Feedback to Rejected

haven't seen this again

Actions
Copy link
 #3

Updated by okurz about 4 years ago

  • Description updated (diff)
  • Due date changed from 2020-10-18 to 2020-10-25
  • Status changed from Rejected to Feedback
  • Priority changed from Low to Normal
Actions
Copy link
 #4

Updated by okurz about 4 years ago

  • Status changed from Feedback to Workable
  • Assignee changed from okurz to mkittler

Failed again and found details in https://build.opensuse.org/public/build/devel:openQA:tested/openSUSE_Tumbleweed/x86_64/openQA/_log today:

[  904s]     #   Failed test 'test would have failed'
[  904s]     #   at ./t/01-test-utilities.t line 47.
[  904s] 
[  904s]     #   Failed test 'manual termination via stop_service does not trigger _fail_and_exit'
[  904s]     #   at ./t/01-test-utilities.t line 58.
[  904s]     #          got: '1'
[  904s]     #     expected: '0'
[  904s]     # Looks like you failed 2 tests of 5.
[  904s] 
[  904s] #   Failed test 'warnings in sub processes are fatal test failures'
[  904s] #   at ./t/01-test-utilities.t line 59.
[  904s] # Looks like you failed 1 test of 1.
[  905s] ./t/01-test-utilities.t ..................................... 
[  905s] # Subtest: warnings in sub processes are fatal test failures
[  905s]     # PID of test-process: 5508
[  905s]     ok 1 - warning logged
[  905s]     not ok 2 - test would have failed
[  905s]     # PID of test-process: 5513
[  905s]     ok 3 - message of test failure
[  905s]     ok 4 - exit code of test failure is non-zero
[  905s]     not ok 5 - manual termination via stop_service does not trigger _fail_and_exit
[  905s]     1..5
[  905s] not ok 1 - warnings in sub processes are fatal test failures
[  905s] 1..1
[  905s] Dubious, test returned 1 (wstat 256, 0x100)
[  905s] Failed 1/1 subtests

@mkittler do you think you could take a look into this?

Actions
Copy link
 #5

Updated by mkittler about 4 years ago

  • Status changed from Workable to In Progress
Actions
Copy link
 #6

Updated by mkittler about 4 years ago

PR (also contains explanation): https://github.com/os-autoinst/openQA/pull/3477

Actions
Copy link
 #7

Updated by okurz about 4 years ago

The PR is merged but unfortunately did not work, https://build.opensuse.org/public/build/devel:openQA/openSUSE_Tumbleweed/x86_64/openQA/_log running openQA-4.6.1603211536.4498c99a4 corresponding to https://github.com/os-autoinst/openQA/commit/4498c99a4376dd7a0e2a2a7ac0f7baf67112fc0c which is the commit of the above PR shows still:

[ 1128s] 
[ 1128s]     #   Failed test 'test would have failed'
[ 1128s]     #   at ./t/01-test-utilities.t line 49.
[ 1128s] 
[ 1128s]     #   Failed test 'manual termination via stop_service does not trigger _fail_and_exit'
[ 1128s]     #   at ./t/01-test-utilities.t line 61.
[ 1128s]     #          got: '1'
[ 1128s]     #     expected: '0'
[ 1128s]     # Looks like you failed 2 tests of 5.
[ 1128s] 
[ 1128s] #   Failed test 'warnings in sub processes are fatal test failures'
[ 1128s] #   at ./t/01-test-utilities.t line 62.
[ 1128s] # Looks like you failed 1 test of 1.
[ 1128s] ./t/01-test-utilities.t ..................................... 
[ 1128s] # Subtest: warnings in sub processes are fatal test failures
[ 1128s]     # PID of test-process-1: 5818
[ 1128s]     # waiting at most 70 seconds for SIGCHLD (sleep is supposed to be interrupted by SIGCHLD)
[ 1128s]     ok 1 - warning logged
[ 1128s]     not ok 2 - test would have failed
[ 1128s]     # PID of test-process-2: 5877
[ 1128s]     # waiting at most 70 seconds for SIGTERM (sleep is supposed to be interrupted by SIGTERM)
[ 1128s]     ok 3 - message of test failure
[ 1128s]     ok 4 - exit code of test failure is non-zero
[ 1128s]     not ok 5 - manual termination via stop_service does not trigger _fail_and_exit
[ 1128s]     1..5
[ 1128s] not ok 1 - warnings in sub processes are fatal test failures
[ 1128s] 1..1
[ 1128s] Dubious, test returned 1 (wstat 256, 0x100)
[ 1128s] Failed 1/1 subtests
Actions
Copy link
 #9

Updated by mkittler about 4 years ago

  • Status changed from In Progress to New
  • Assignee deleted (mkittler)

Then I'm out of ideas. Not sure how to test the code. If we would mock everything it would be quite useless because I want to test whether the code actually works in accordance to the OS signal handling. Maybe just keep the test for local development (where I've never run into the error). Since you've already removed the test from OBS it shouldn't prevent our builds anymore at least.

Actions
Copy link
 #10

Updated by okurz about 4 years ago

  • Status changed from New to Resolved
  • Assignee set to okurz

I think it's ok like it is now. My PR is merged, the test is excluded from OBS checks so not failing there anymore but we still run the test fine within circleCI checks just fine.

Actions
Copy link

Also available in: Atom PDF