action #53498

[sporadic] openQA CI tests fail on master in flaky/unstable t/ui/26-jobs_restart.t

Added by okurz 10 months ago. Updated 2 months ago.

Status:ResolvedStart date:24/06/2019
Priority:NormalDue date:
Assignee:okurz% Done:

0%

Category:Concrete Bugs
Target version:Current Sprint
Difficulty:
Duration:

Description

https://travis-ci.org/os-autoinst/openQA/jobs/549427366#L642 mentions

# Tests were run but no plan was declared and done_testing() was not seen.
./t/ui/26-jobs_restart.t ............. 
Dubious, test returned 254 (wstat 65024, 0xfe00)
All 7 subtests passed 
Test Summary Report
-------------------
./t/ui/26-jobs_restart.t           (Wstat: 65024 Tests: 7 Failed: 0)
  Non-zero exit status: 254
  Parse errors: No plan found in TAP output

Related issues

Related to openQA Project - action #59984: unstable test: t/05-scheduler-full.t Resolved 18/11/2019
Related to openQA Project - action #60209: flaky/unstable test in OBS t/17-labels_carry_over.t Resolved 24/11/2019
Related to openQA Project - action #53771: unstable test: t/ui/15-comments.t in line 102: "got: 'Dem... Blocked 03/07/2019

History

#1 Updated by okurz 10 months ago

  • Subject changed from openQA travis CI tests fail on master in t/ui/26-jobs_restart.t to [sporadic] openQA travis CI tests fail on master in t/ui/26-jobs_restart.t

Using https://github.com/os-autoinst/openQA/pull/2131/
I ran

for i in {001..100} ;do echo "### $i" && make PROVE_ARGS=t/ui/26-jobs_restart.t test-with-database || break ; done

and could not reproduce the problem locally. Could be only reproducible in the potentially slower travis CI environment or only in the openSUSE Leap 42.3 container environment or only when running more runs.

#3 Updated by okurz 5 months ago

  • Subject changed from [sporadic] openQA travis CI tests fail on master in t/ui/26-jobs_restart.t to [sporadic] openQA CI tests fail on master in flaky/unstable t/ui/26-jobs_restart.t
  • Status changed from New to Feedback
  • Assignee set to okurz
  • Target version set to Current Sprint

I already have a PR trying to reproduce flaky tests more reliably, see https://github.com/os-autoinst/openQA/pull/2496

#4 Updated by okurz 2 months ago

I rebased my branches and pushed an update to https://github.com/os-autoinst/openQA/pull/2496 . We can see how tests currently fare.

#5 Updated by okurz 2 months ago

  • Related to action #59984: unstable test: t/05-scheduler-full.t added

#6 Updated by okurz 2 months ago

  • Related to action #60209: flaky/unstable test in OBS t/17-labels_carry_over.t added

#7 Updated by okurz 2 months ago

  • Related to action #53771: unstable test: t/ui/15-comments.t in line 102: "got: 'Demo wrote less than a minute ago (last edited less than a minute ago)'" expected: 'Demo wrote less than a minute ago' added

#8 Updated by okurz 2 months ago

Extracted a part into https://github.com/os-autoinst/openQA/pull/2693 to have the "unstable" tests covered which is actually what this ticket is about. Rest is about scheduler- and other full-stack test.

#9 Updated by cdywan 2 months ago

gh#2693 got merged

#10 Updated by okurz 2 months ago

  • Status changed from Feedback to Resolved

right. So with this we currently do not have any tests in "unstable_tests.txt" anymore, rest is for full stack tests in other tickets.

Also available in: Atom PDF