action #157195
openImplement in our CI a query to get the latest job id related with latest image
0%
Description
Motivation¶
Migrations from Leap to SLE, we need set the setting:
HDD_1_URL: 'https://openqa.opensuse.org/tests/3976134/asset/hdd/opensuse-15.6-x86_64-623.2-textmode@64bit.qcow2'
HDD_1: opensuse-15.6-x86_64-623.2-textmode@64bit.qcow2
HDD_1_URL: 'https://openqa.opensuse.org/tests/3977957/asset/hdd/opensuse-15.6-x86_64-623.2-gnome@64bit.qcow2'
HDD_1: opensuse-15.6-x86_64-623.2-gnome@64bit.qcow2
HDD_1_URL: 'https://openqa.opensuse.org/tests/3977685/asset/hdd/opensuse-15.6-x86_64-623.2-kde@64bit.qcow2'
HDD_1: opensuse-15.6-x86_64-623.2-kde@64bit.qcow2
The job id may be deleted, we need update those id and image name according the latest image name.
Acceptance criteria¶
AC1: Implement in our CI a query to get the latest id and set the setting without manual intervention.
Updated by JERiveraMoya 9 months ago
- Description updated (diff)
- Parent task set to #151984
Updated by JERiveraMoya 9 months ago
- Tags set to qe-yam-apr-sprint
- Description updated (diff)
- Status changed from New to Workable
Updated by JERiveraMoya 9 months ago
Explanation why the dynamic values are set in test suite instead of yaml, just for completeness:
https://gitlab.suse.de/qe-yam/openqa-job-groups/-/merge_requests/107#note_604100
Updated by syrianidou_sofia 8 months ago
- Status changed from Workable to In Progress
- Assignee set to syrianidou_sofia
Updated by syrianidou_sofia 8 months ago
The script works and is able to replace the macro in the yaml job group with the latest image. Example: https://gitlab.suse.de/qe-yam/openqa-job-groups/-/jobs/2524652
Updated by JERiveraMoya 8 months ago
- Tags changed from qe-yam-apr-sprint to qe-yam-may-sprint
Updated by syrianidou_sofia 7 months ago · Edited
- Status changed from In Progress to Workable
After reviewing the solution proposed in draft: https://gitlab.suse.de/qe-yam/openqa-job-groups/-/merge_requests/163/diffs
we came to the conclusion that verifying the image existence would add complextity and difficulty maintaining the code of the CI. Additionally, using variable syntax like {{my_var}} should be avoided (in slack discussion: https://suse.slack.com/archives/C02CLSHS3SB/p1714467732188989 "A better choice would be ${{ ... }}. The problem is that { is special in YAML because it creates a flow mapping (it's basically just like in jSON). With putting $ at the start you avoid having to quote it". )
Proposed solutions :
1 Using a similar way as in the above draft, in order to replace a variable only for the image build, according to the latest leap build, each time the CI runs for the particular job group. Simplifying the bash script as:
get_latest_build() {
local latest_build
local leap_version
mapfile -t -n 2 latest_build < <(curl -L -sS "openqa.opensuse.org/group_overview/50.json" \
| jq -r '([.build_results[] | select(.version=="15.6") | .build] | sort | reverse)[]')
if [[ "${#latest_builds[@]}" -lt 1 ]]; then
warn "Unable to find latest Leap builds."
exit 1
fi
}
This would require a scheduled pipeline that will run the particular job once every week/biweekly, because we cannot guarantee that there will be a merge request editing the relevant job group often enough.
2 Another approach: Instead of adding a job in the CI jobs that run while merging or committing code, we can create just a scheduled pipeline that will call a script to edit directly the master branch files that include leap image dependent jobs and will merge the code itself. See bellow useful links:
https://stackoverflow.com/questions/66941947/gitalb-ci-cd-update-a-file-on-gitlab-repository-after-modification-on-gitlab-run
https://stackoverflow.com/questions/65816737/gitlab-ci-scheduled-pipeline-not-running-on-default-branch
https://docs.gitlab.com/ee/ci/pipelines/schedules.html
This solution is cleaner as it will not affect pipelines while we create our own merge requests and we will avoid adding variables in the leap image names in the job group yaml files. The master branch will contain for example:
HDD_1_URL: 'https://openqa.opensuse.org/tests/3976134/asset/hdd/opensuse-15.6-x86_64-623.2-textmode@64bit.qcow2'
HDD_1: opensuse-15.6-x86_64-623.2-textmode@64bit.qcow2
Instead of
HDD_1_URL: 'https://openqa.opensuse.org/tests/3976134/asset/hdd/opensuse-15.6-x86_64-${{latest_leap_build}}-textmode@64bit.qcow2'
HDD_1: opensuse-15.6-x86_64-${{latest_leap_build}}-textmode@64bit.qcow2
The build number will be updated every time the scheduled pipeline runs to the latest leap build number.
3 The fastest and simplest solution would be to add a cron job that would edit the osd test suite data base directly, in order to keep using test reference in our job group , but the possible complications are unknown.
Updated by JERiveraMoya 6 months ago
- Tags changed from qe-yam-may-sprint to qe-yam-jan-sprint
Updated by JERiveraMoya 6 months ago
- Tags changed from qe-yam-jan-sprint to qe-yam-jun-sprint
Updated by JERiveraMoya 6 months ago
- Tags changed from qe-yam-jun-sprint to qe-yam-jul-sprint
Updated by rainerkoenig 5 months ago
- Status changed from In Progress to New
Back to new after discussing it today in the call.
Updated by JERiveraMoya 5 months ago · Edited
Leo found a simple way to query the issue:
https://gitlab.suse.de/qe-yam/openqa-job-groups/-/merge_requests/163#note_643358
But still we need to think if we need such a thing in a job or in the ci, best places would be obs sync plugin.