action #66718
closedunstable/flaky/sporadic test: t/ui/27-plugin_obs_rsync_status_details.t
Description
Observation¶
[22:54:20] t/ui/27-plugin_obs_rsync_status_details.t .. 19/? [22:54:20] t/ui/27-plugin_obs_rsync_status_details.t .. 24/?
# Failed test 'BatchedProj obs builds'
# at t/ui/27-plugin_obs_rsync_status_details.t line 181.
# got: '470.2, 469.1'
# expected: '4704, 4703, 470.2, 469.1'
[22:54:20] t/ui/27-plugin_obs_rsync_status_details.t .. 27/? [22:54:20] t/ui/27-plugin_obs_rsync_status_details.t .. 28/? [22:54:20] t/ui/27-plugin_obs_rsync_status_details.t .. …60/? # Looks like you failed 1 test of 60.
[22:54:20] t/ui/27-plugin_obs_rsync_status_details.t .. Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/60 subtests
t/ui/27-plugin_obs_rsync_status_details.t failed in 1/2 runs within a stability test
Acceptance criteria¶
- AC1: t/ui/27-plugin_obs_rsync_status_details.t is stable in 20/20 tries
- AC2: t/ui/27-plugin_obs_rsync_status_details.t is removed from .circleci/unstable_tests.txt
Suggestions¶
- Run
make test RETRY=20 STABILITY_TEST=1 TESTS=t/ui/27-plugin_obs_rsync_status_details.t
locally to reproduce - Identify causes of sporadic failures and fix in either production code or tests
- Prove stability locally and in CI
- Remove from .circleci/unstable_tests.txt again
Updated by okurz over 4 years ago
proposed the move of the test to unstables in https://github.com/os-autoinst/openQA/pull/3081
Updated by Xiaojing_liu almost 4 years ago
I cannot reproduce this issue in my local.
I run make test RETRY=20 STABILITY_TEST=1 TESTS=t/ui/27-plugin_obs_rsync_status_details.t
, every time passed.
I also tried for i in $(seq 1 20); do sudo docker run -it --rm -v $PWD:/opt/testing_area localtest bash -c 'make test TESTS=t/ui/27-plugin_obs_rsync_status_details.t' >> 2; done
, all tests passed. I am not sure if run it in docket is the way to ensure Prove stability in CI
.
BTW, I modified the timelimit in this file, because it always returns error then the timelimit is 16
.
server stopped
real 0m48.786s
user 0m0.075s
sys 0m0.049s
Updated by okurz almost 4 years ago
- Target version changed from Ready to future
Thank you for checking. If it's this hard to reproduce let's keep it like it is for now and look into this sometime later.
Updated by livdywan over 3 years ago
Seems like PRs are hitting this again:
not ok 24 - BatchedProj obs builds
[debug] [xl23hAJV] POST "/admin/obs_rsync/BatchedProj/obs_builds_text"
[debug] [xl23hAJV] Routing to controller "OpenQA::Shared::Controller::Session" and action "ensure_operator"
[debug] [xl23hAJV] Routing to controller "OpenQA::WebAPI::Plugin::ObsRsync::Controller::Gru" and action "update_obs_builds_text"
[debug] [xl23hAJV] 200 OK (0.024276s, 41.193/s)
[debug] [C0j4SN9_] GET "/admin/obs_rsync/BatchedProj/obs_builds_text"
[debug] [C0j4SN9_] Routing to controller "OpenQA::Shared::Controller::Session" and action "ensure_operator"
[debug] [C0j4SN9_] Routing to controller "OpenQA::WebAPI::Plugin::ObsRsync::Controller::Gru" and action "get_obs_builds_text"
[debug] [C0j4SN9_] 200 OK (0.016182s, 61.797/s)
Seen this on https://github.com/os-autoinst/openQA/pull/3694 and https://github.com/os-autoinst/openQA/pull/3635
Updated by okurz over 3 years ago
- Due date set to 2021-02-23
- Status changed from Workable to Feedback
- Assignee set to andriinikitin
- Target version changed from future to Ready
In recent days we saw this even more. Gladly @andriinikitin had a good idea how to address this:
https://github.com/os-autoinst/openQA/pull/3732
Updated by okurz over 3 years ago
- Status changed from Feedback to In Progress
- Assignee changed from andriinikitin to okurz
I will conduct the stability tests and see if we can remove that from the unstable list
Updated by okurz over 3 years ago
Did a successful local run with
env runs=100 count_fail_ratio prove -l t/ui/27-plugin_obs_rsync_status_details.t
Testing in circleCI https://github.com/os-autoinst/openQA/pull/3752
Updated by okurz over 3 years ago
- Status changed from In Progress to Feedback
https://app.circleci.com/pipelines/github/os-autoinst/openQA/5721/workflows/996637f1-4195-4eb4-840f-5a3b573f9a6c/jobs/53970 shows 20/20 successful runs within circleCI. Updated PR.
Updated by okurz over 3 years ago
- Related to action #89935: t/ui/27-plugin_obs_rsync_status_details.t fails in circleCI, master branch even added