action #66718
closed
unstable/flaky/sporadic test: t/ui/27-plugin_obs_rsync_status_details.t
Added by okurz about 4 years ago.
Updated over 3 years ago.
Category:
Regressions/Crashes
Description
Observation¶
See https://app.circleci.com/pipelines/github/os-autoinst/openQA/2827/workflows/13672907-1ad3-48d0-a1b2-e210991ed1f8/jobs/26708/steps :
[22:54:20] t/ui/27-plugin_obs_rsync_status_details.t .. 19/? [22:54:20] t/ui/27-plugin_obs_rsync_status_details.t .. 24/?
# Failed test 'BatchedProj obs builds'
# at t/ui/27-plugin_obs_rsync_status_details.t line 181.
# got: '470.2, 469.1'
# expected: '4704, 4703, 470.2, 469.1'
[22:54:20] t/ui/27-plugin_obs_rsync_status_details.t .. 27/? [22:54:20] t/ui/27-plugin_obs_rsync_status_details.t .. 28/? [22:54:20] t/ui/27-plugin_obs_rsync_status_details.t .. …60/? # Looks like you failed 1 test of 60.
[22:54:20] t/ui/27-plugin_obs_rsync_status_details.t .. Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/60 subtests
t/ui/27-plugin_obs_rsync_status_details.t failed in 1/2 runs within a stability test
Acceptance criteria¶
- AC1: t/ui/27-plugin_obs_rsync_status_details.t is stable in 20/20 tries
- AC2: t/ui/27-plugin_obs_rsync_status_details.t is removed from .circleci/unstable_tests.txt
Suggestions¶
- Run
make test RETRY=20 STABILITY_TEST=1 TESTS=t/ui/27-plugin_obs_rsync_status_details.t locally to reproduce
- Identify causes of sporadic failures and fix in either production code or tests
- Prove stability locally and in CI
- Remove from .circleci/unstable_tests.txt again
- Target version set to Ready
- Priority changed from Normal to Low
I cannot reproduce this issue in my local.
I run make test RETRY=20 STABILITY_TEST=1 TESTS=t/ui/27-plugin_obs_rsync_status_details.t, every time passed.
I also tried for i in $(seq 1 20); do sudo docker run -it --rm -v $PWD:/opt/testing_area localtest bash -c 'make test TESTS=t/ui/27-plugin_obs_rsync_status_details.t' >> 2; done, all tests passed. I am not sure if run it in docket is the way to ensure Prove stability in CI.
BTW, I modified the timelimit in this file, because it always returns error then the timelimit is 16.
server stopped
real 0m48.786s
user 0m0.075s
sys 0m0.049s
- Target version changed from Ready to future
Thank you for checking. If it's this hard to reproduce let's keep it like it is for now and look into this sometime later.
Seems like PRs are hitting this again:
not ok 24 - BatchedProj obs builds
[debug] [xl23hAJV] POST "/admin/obs_rsync/BatchedProj/obs_builds_text"
[debug] [xl23hAJV] Routing to controller "OpenQA::Shared::Controller::Session" and action "ensure_operator"
[debug] [xl23hAJV] Routing to controller "OpenQA::WebAPI::Plugin::ObsRsync::Controller::Gru" and action "update_obs_builds_text"
[debug] [xl23hAJV] 200 OK (0.024276s, 41.193/s)
[debug] [C0j4SN9_] GET "/admin/obs_rsync/BatchedProj/obs_builds_text"
[debug] [C0j4SN9_] Routing to controller "OpenQA::Shared::Controller::Session" and action "ensure_operator"
[debug] [C0j4SN9_] Routing to controller "OpenQA::WebAPI::Plugin::ObsRsync::Controller::Gru" and action "get_obs_builds_text"
[debug] [C0j4SN9_] 200 OK (0.016182s, 61.797/s)
Seen this on https://github.com/os-autoinst/openQA/pull/3694 and https://github.com/os-autoinst/openQA/pull/3635
- Due date set to 2021-02-23
- Status changed from Workable to Feedback
- Assignee set to andriinikitin
- Target version changed from future to Ready
- Status changed from Feedback to In Progress
- Assignee changed from andriinikitin to okurz
I will conduct the stability tests and see if we can remove that from the unstable list
- Status changed from In Progress to Feedback
- Status changed from Feedback to Resolved
- Related to action #89935: t/ui/27-plugin_obs_rsync_status_details.t fails in circleCI, master branch even added
Also available in: Atom
PDF