Project

General

Profile

action #37638

Flaky fullstack test: 'Test 3 is scheduled' at t/full-stack.t

Added by szarate almost 3 years ago. Updated 5 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
okurz
Category:
Concrete Bugs
Target version:
QA - future
Start date:
2018-06-21
Due date:
% Done:

0%

Estimated time:
Difficulty:

Description

We are already familiar with this problem :) 

not ok 47 - Test 3 is scheduled
#   Failed test 'Test 3 is scheduled'
#   at /opt/testing_area/openqa/t/lib/OpenQA/Test/FullstackUtils.pm line 85.
#                   'State: assigned
# Clone of 2
# Assigned worker: fb5ec5661fea:1'
#     doesn't match '(?^:State: scheduled)'
Uncaught exception from user code:
Error while executing command: no such element: Unable to locate element: {"method":"css selector","selector":"[id='cancel_running']"}
(Session info: headless chrome=66.0.3359.170)
(Driver info: chromedriver=2.36 (0),platform=Linux 4.4.0-101-generic x86_64) at /usr/lib/perl5/vendor_perl/5.18.2/Selenium/Remote/Driver.pm line 388.
at /usr/lib/perl5/vendor_perl/5.18.2/Selenium/Remote/Driver.pm line 345.
Selenium::Remote::Driver::find_element('Test::Selenium::Chrome=HASH(0x7d082d0)', 'cancel_running', 'id') called at /usr/lib/perl5/vendor_perl/5.18.2/Test/Selenium/Remote/Driver.pm line 345
Test::Selenium::Remote::Driver::_find_element_with_action('Test::Selenium::Chrome=HASH(0x7d082d0)', 'click_ok', 'cancel_running', 'id', 'Caught cancel') called at /usr/lib/perl5/vendor_perl/5.18.2/Test/Selenium/Remote/Driver.pm line 400
Test::Selenium::Remote::Driver::click_element_ok('Test::Selenium::Chrome=HASH(0x7d082d0)', 'cancel_running', 'id', 'Caught cancel') called at t/full-stack.t line 261

Related issues

Related to openQA Project - action #54032: flaky test: t/full-stack.t "not ok 44 - Cache tests", subtest: "not ok 17 - test 6 is passed" at /opt/testing_area/openqa/t/lib/OpenQA/Test/FullstackUtils.pm line 120.Resolved2019-07-09

Related to openQA Project - action #38276: Flaky 'test candidate list' at t/ui/18-tests-details.tResolved2018-07-06

Related to openQA Project - action #55460: flaky test: t/33-developer_mode.t - not ok 308 - Wait for (?^:\"set_pause_on_screen_mismatch\":\"assert_screen\") timed outResolved2019-08-13

Related to openQA Project - action #55457: flaky test: t/api/11-bugs.tResolved2019-08-13

Related to openQA Project - action #53858: flaky test: 25-cache-service.tResolved2019-07-04

Related to openQA Project - action #62015: jobs incomplete without logs as some workers are rejected (was: Scheduler does not work)Resolved2020-01-10

Related to openQA Project - action #71554: unstable/flaky/sporadic t/full-stack.t test failing in script waits on CircleCIResolved2020-09-19

History

#1 Updated by okurz almost 2 years ago

  • Category set to Concrete Bugs

#2 Updated by okurz over 1 year ago

  • Related to action #54032: flaky test: t/full-stack.t "not ok 44 - Cache tests", subtest: "not ok 17 - test 6 is passed" at /opt/testing_area/openqa/t/lib/OpenQA/Test/FullstackUtils.pm line 120. added

#3 Updated by okurz over 1 year ago

  • Related to action #38276: Flaky 'test candidate list' at t/ui/18-tests-details.t added

#4 Updated by okurz over 1 year ago

  • Status changed from New to Feedback
  • Assignee set to okurz

https://github.com/os-autoinst/openQA/pull/2259 to remove "travis_retry" again after test stabilization – and proper testing in the PR

#5 Updated by okurz over 1 year ago

  • Related to action #55460: flaky test: t/33-developer_mode.t - not ok 308 - Wait for (?^:\"set_pause_on_screen_mismatch\":\"assert_screen\") timed out added

#6 Updated by okurz over 1 year ago

#7 Updated by okurz over 1 year ago

#8 Updated by okurz over 1 year ago

Three issues found:

After these are fixed we can come back to this specific and reconduct a full test run and see if it still appears.

#9 Updated by okurz over 1 year ago

  • Status changed from Feedback to Workable
  • Assignee deleted (okurz)

We moved to circle CI, I did not yet look into the unstable tests on circle CI but we still have a list with "unstable" tests handled within circle CI as well.

#10 Updated by okurz about 1 year ago

https://circleci.com/gh/os-autoinst/openQA/10986?utm_campaign=workflow-failed&utm_medium=email&utm_source=notification shows

while :; do\
    [ $n -lt "3" ] || exit 1;\
    [ $n -eq 0 ] || echo Retrying...;\
    prove -l --harness TAP::Harness::JUnit t/full-stack.t && break;\
    n=$[$n+1];\
done
t/full-stack.t .. 40/? 
#   Failed test 'Test 4 crashed as expected'
#   at /home/squamata/project/t/lib/OpenQA/Test/FullstackUtils.pm line 120.
#                   'State: scheduled
# 50'
#     doesn't match '(?^:Result: incomplete)'
t/full-stack.t .. 41/? 
    #   Failed test 'job is running'
    #   at /home/squamata/project/t/lib/OpenQA/Test/FullstackUtils.pm line 120.
    #                   'State: scheduled
    # Clone of 3
    # 50'
    #     doesn't match '(?^:State: running)'

    #   Failed test 'findElement: no such element: Unable to locate element: {"method":"link text","selector":"Live View"}'
    #   at /home/squamata/project/t/lib/OpenQA/SeleniumTest.pm line 150.

    #   Failed test 'clickElement: stale element reference: element is not attached to the page document'
    #   at /home/squamata/project/t/lib/OpenQA/SeleniumTest.pm line 150.
make[1]: *** [Makefile:115: test-unit-and-integration] Hangup
make: *** [Makefile:128: test-with-database] Hangup

#11 Updated by okurz about 1 year ago

  • Related to action #62015: jobs incomplete without logs as some workers are rejected (was: Scheduler does not work) added

#12 Updated by okurz 7 months ago

  • Related to action #71554: unstable/flaky/sporadic t/full-stack.t test failing in script waits on CircleCI added

#13 Updated by okurz 5 months ago

  • Status changed from Workable to Resolved
  • Assignee set to okurz

I think we have this covered. I did a lot of tests for stability of t/full-stack.t and this seems to have gone, not sure which specific change though. In ac0ccca26 we have removed retries for t/full-stack.t

Also available in: Atom PDF