Project

General

Profile

Actions
Copy link

action #53498

closed

[sporadic] openQA CI tests fail on master in flaky/unstable t/ui/26-jobs_restart.t

Added by okurz over 5 years ago. Updated almost 5 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
okurz
Category:
Regressions/Crashes
Target version:
Current Sprint
Start date:
2019-06-24
Due date:
% Done:

0%

Estimated time:

Description

https://travis-ci.org/os-autoinst/openQA/jobs/549427366#L642 mentions

# Tests were run but no plan was declared and done_testing() was not seen.
./t/ui/26-jobs_restart.t ............. 
Dubious, test returned 254 (wstat 65024, 0xfe00)
All 7 subtests passed 
Test Summary Report
-------------------
./t/ui/26-jobs_restart.t           (Wstat: 65024 Tests: 7 Failed: 0)
  Non-zero exit status: 254
  Parse errors: No plan found in TAP output

Related issues 3 (0 open — 3 closed)

Related to openQA Project (public) - action #59984: unstable test: t/05-scheduler-full.tResolvedokurz2019-11-18

Actions
Related to openQA Project (public) - action #60209: flaky/unstable test in OBS t/17-labels_carry_over.t Resolvedokurz2019-11-24

Actions
Related to openQA Project (public) - action #53771: unstable test: t/ui/15-comments.t in line 102: "got: 'Demo wrote less than a minute ago (last edited less than a minute ago)'" expected: 'Demo wrote less than a minute ago'Resolvedokurz2019-07-03

Actions
Actions
Copy link
 #1

Updated by okurz over 5 years ago

  • Subject changed from openQA travis CI tests fail on master in t/ui/26-jobs_restart.t to [sporadic] openQA travis CI tests fail on master in t/ui/26-jobs_restart.t

Using https://github.com/os-autoinst/openQA/pull/2131/
I ran

for i in {001..100} ;do echo "### $i" && make PROVE_ARGS=t/ui/26-jobs_restart.t test-with-database || break ; done

and could not reproduce the problem locally. Could be only reproducible in the potentially slower travis CI environment or only in the openSUSE Leap 42.3 container environment or only when running more runs.

Actions
Copy link
 #3

Updated by okurz about 5 years ago

  • Subject changed from [sporadic] openQA travis CI tests fail on master in t/ui/26-jobs_restart.t to [sporadic] openQA CI tests fail on master in flaky/unstable t/ui/26-jobs_restart.t
  • Status changed from New to Feedback
  • Assignee set to okurz
  • Target version set to Current Sprint

I already have a PR trying to reproduce flaky tests more reliably, see https://github.com/os-autoinst/openQA/pull/2496

Actions
Copy link
 #4

Updated by okurz almost 5 years ago

I rebased my branches and pushed an update to https://github.com/os-autoinst/openQA/pull/2496 . We can see how tests currently fare.

Actions
Copy link
 #5

Updated by okurz almost 5 years ago

  • Related to action #59984: unstable test: t/05-scheduler-full.t added
Actions
Copy link
 #6

Updated by okurz almost 5 years ago

  • Related to action #60209: flaky/unstable test in OBS t/17-labels_carry_over.t added
Actions
Copy link
 #7

Updated by okurz almost 5 years ago

  • Related to action #53771: unstable test: t/ui/15-comments.t in line 102: "got: 'Demo wrote less than a minute ago (last edited less than a minute ago)'" expected: 'Demo wrote less than a minute ago' added
Actions
Copy link
 #8

Updated by okurz almost 5 years ago

Extracted a part into https://github.com/os-autoinst/openQA/pull/2693 to have the "unstable" tests covered which is actually what this ticket is about. Rest is about scheduler- and other full-stack test.

Actions
Copy link
 #9

Updated by livdywan almost 5 years ago

gh#2693 got merged

Actions
Copy link
 #10

Updated by okurz almost 5 years ago

  • Status changed from Feedback to Resolved

right. So with this we currently do not have any tests in "unstable_tests.txt" anymore, rest is for full stack tests in other tickets.

Actions
Copy link

Also available in: Atom PDF