action #175485
closedcoordination #102906: [saga][epic] Increased stability of tests with less "known failures", known incompletes handled automatically within openQA
coordination #175515: [epic] incomplete jobs with "Failed to find an available port: Address already in use"
jobs incomplete with auto_review:"setup failure: isotovideo can not be started" - why did osd-deployment trigger even though openQA-in-openQA as already showing incompletes? size:S
0%
Description
Observation¶
We have a check in the GitLab pipeline to evaluate the openQA in openQA test state. Seemingly the pipeline was fine even though we know things failed in production as reported in #175464.
Acceptance critiera¶
- AC1: The OSD deployment is not executed when openQA-in-openQA tests fail
Suggestions¶
- The check in bash is obviously wrong
- The sleep probably overrides the return code?
- Consider splitting the lines or move this to a proper script https://gitlab.suse.de/openqa/osd-deployment/-/blob/ebf5bcdc339b4aee17933f99aa89217f329da386/.gitlab-ci.yml#L160
- The check was exhausting the retries when it ran on 2025-01-15 and wrongfully passed.
Rollback actions¶
- DONE enable notification emails again in https://gitlab.suse.de/openqa/osd-deployment/-/settings/integrations/pipelines_email/edit
Updated by okurz about 1 month ago
- Copied from action #175482: jobs incomplete with auto_review:"setup failure: isotovideo can not be started" - why did no tests prevent this to be deployed on both o3+osd? added
Updated by okurz about 1 month ago
- Copied to action #175488: jobs incomplete with auto_review:"setup failure: isotovideo can not be started" - unable to login with ssh key on root@kerosene.qe.nue2.suse.org added
Updated by mkittler about 1 month ago
- Subject changed from jobs incomplete with auto_review:"setup failure: isotovideo can not be started" - why did osd-deployment trigger even though openQA-in-openQA as already showing incompletes? to jobs incomplete with auto_review:"setup failure: isotovideo can not be started" - why did osd-deployment trigger even though openQA-in-openQA as already showing incompletes? size:S
- Description updated (diff)
- Status changed from New to Workable
Updated by okurz about 1 month ago
- Status changed from Workable to In Progress
- Assignee set to okurz
for i in {1..7} ; do false || (echo retry $i && sleep 0); done will return with success if we exhaust the retries. We could use for i in {1..7} ; do false || (echo retry $i && sleep 0; false); done or use "retry":
osc copypac home:okurz:container:ca curl-jq-ssh home:okurz:container:ca curl-jq-ssh-retry
Updated by okurz about 1 month ago
- Status changed from In Progress to Feedback
Updated by okurz about 1 month ago ยท Edited
- Status changed from Feedback to In Progress
merged. now some steps use the wrong image.
From https://build.suse.de/project/show/home:okurz:container removed images "curl" and "curl-jq" which were used (at least) in https://gitlab.suse.de/openqa/osd-deployment/ as I switched to https://build.opensuse.org/package/show/home:okurz:container:ca/curl-jq-ssh-retry instead
Updated by okurz about 1 month ago
- Description updated (diff)
- Status changed from In Progress to Resolved