Project

General

Profile

Actions

action #157195

open

Implement in our CI a query to get the latest job id related with latest image

Added by tinawang123 9 months ago. Updated about 2 months ago.

Status:
New
Priority:
Low
Assignee:
-
Target version:
-
Start date:
2024-03-14
Due date:
% Done:

0%

Estimated time:

Description

Motivation

Migrations from Leap to SLE, we need set the setting:
HDD_1_URL: 'https://openqa.opensuse.org/tests/3976134/asset/hdd/opensuse-15.6-x86_64-623.2-textmode@64bit.qcow2'
HDD_1: opensuse-15.6-x86_64-623.2-textmode@64bit.qcow2

      HDD_1_URL: 'https://openqa.opensuse.org/tests/3977957/asset/hdd/opensuse-15.6-x86_64-623.2-gnome@64bit.qcow2'
      HDD_1: opensuse-15.6-x86_64-623.2-gnome@64bit.qcow2

      HDD_1_URL: 'https://openqa.opensuse.org/tests/3977685/asset/hdd/opensuse-15.6-x86_64-623.2-kde@64bit.qcow2'
      HDD_1: opensuse-15.6-x86_64-623.2-kde@64bit.qcow2

The job id may be deleted, we need update those id and image name according the latest image name.

Acceptance criteria

AC1: Implement in our CI a query to get the latest id and set the setting without manual intervention.

Actions #1

Updated by JERiveraMoya 9 months ago

  • Description updated (diff)
  • Parent task set to #151984
Actions #2

Updated by JERiveraMoya 9 months ago

  • Tags set to qe-yam-apr-sprint
  • Description updated (diff)
  • Status changed from New to Workable
Actions #3

Updated by JERiveraMoya 9 months ago

Explanation why the dynamic values are set in test suite instead of yaml, just for completeness:
https://gitlab.suse.de/qe-yam/openqa-job-groups/-/merge_requests/107#note_604100

Actions #4

Updated by syrianidou_sofia 8 months ago

  • Status changed from Workable to In Progress
  • Assignee set to syrianidou_sofia
Actions #5

Updated by syrianidou_sofia 8 months ago

The script works and is able to replace the macro in the yaml job group with the latest image. Example: https://gitlab.suse.de/qe-yam/openqa-job-groups/-/jobs/2524652

Actions #6

Updated by JERiveraMoya 8 months ago

  • Tags changed from qe-yam-apr-sprint to qe-yam-may-sprint
Actions #7

Updated by JERiveraMoya 7 months ago

  • Priority changed from Normal to Low
Actions #8

Updated by syrianidou_sofia 7 months ago · Edited

  • Status changed from In Progress to Workable

After reviewing the solution proposed in draft: https://gitlab.suse.de/qe-yam/openqa-job-groups/-/merge_requests/163/diffs
we came to the conclusion that verifying the image existence would add complextity and difficulty maintaining the code of the CI. Additionally, using variable syntax like {{my_var}} should be avoided (in slack discussion: https://suse.slack.com/archives/C02CLSHS3SB/p1714467732188989 "A better choice would be ${{ ... }}. The problem is that { is special in YAML because it creates a flow mapping (it's basically just like in jSON). With putting $ at the start you avoid having to quote it". )

Proposed solutions :
1 Using a similar way as in the above draft, in order to replace a variable only for the image build, according to the latest leap build, each time the CI runs for the particular job group. Simplifying the bash script as:

get_latest_build() {
    local latest_build
    local leap_version
    mapfile -t -n 2 latest_build < <(curl -L -sS "openqa.opensuse.org/group_overview/50.json" \
         | jq -r '([.build_results[] | select(.version=="15.6") | .build] | sort | reverse)[]')
    if [[ "${#latest_builds[@]}" -lt 1 ]]; then
        warn "Unable to find latest Leap builds."
        exit 1
    fi
}

This would require a scheduled pipeline that will run the particular job once every week/biweekly, because we cannot guarantee that there will be a merge request editing the relevant job group often enough.

2 Another approach: Instead of adding a job in the CI jobs that run while merging or committing code, we can create just a scheduled pipeline that will call a script to edit directly the master branch files that include leap image dependent jobs and will merge the code itself. See bellow useful links:
https://stackoverflow.com/questions/66941947/gitalb-ci-cd-update-a-file-on-gitlab-repository-after-modification-on-gitlab-run
https://stackoverflow.com/questions/65816737/gitlab-ci-scheduled-pipeline-not-running-on-default-branch
https://docs.gitlab.com/ee/ci/pipelines/schedules.html

This solution is cleaner as it will not affect pipelines while we create our own merge requests and we will avoid adding variables in the leap image names in the job group yaml files. The master branch will contain for example:
HDD_1_URL: 'https://openqa.opensuse.org/tests/3976134/asset/hdd/opensuse-15.6-x86_64-623.2-textmode@64bit.qcow2'
HDD_1: opensuse-15.6-x86_64-623.2-textmode@64bit.qcow2
Instead of
HDD_1_URL: 'https://openqa.opensuse.org/tests/3976134/asset/hdd/opensuse-15.6-x86_64-${{latest_leap_build}}-textmode@64bit.qcow2'
HDD_1: opensuse-15.6-x86_64-${{latest_leap_build}}-textmode@64bit.qcow2

The build number will be updated every time the scheduled pipeline runs to the latest leap build number.

3 The fastest and simplest solution would be to add a cron job that would edit the osd test suite data base directly, in order to keep using test reference in our job group , but the possible complications are unknown.

Actions #9

Updated by syrianidou_sofia 7 months ago

  • Assignee deleted (syrianidou_sofia)
Actions #10

Updated by JERiveraMoya 7 months ago

  • Priority changed from Low to Normal
Actions #11

Updated by JERiveraMoya 6 months ago

  • Tags changed from qe-yam-may-sprint to qe-yam-jan-sprint
Actions #12

Updated by JERiveraMoya 6 months ago

  • Tags changed from qe-yam-jan-sprint to qe-yam-jun-sprint
Actions #13

Updated by JERiveraMoya 6 months ago

  • Tags changed from qe-yam-jun-sprint to qe-yam-jul-sprint
Actions #14

Updated by jfernandez 5 months ago

  • Assignee set to jfernandez
Actions #15

Updated by jfernandez 5 months ago

  • Status changed from Workable to In Progress
Actions #16

Updated by rainerkoenig 5 months ago

  • Status changed from In Progress to New

Back to new after discussing it today in the call.

Actions #17

Updated by rainerkoenig 5 months ago

  • Assignee deleted (jfernandez)
Actions #18

Updated by JERiveraMoya 5 months ago · Edited

Leo found a simple way to query the issue:
https://gitlab.suse.de/qe-yam/openqa-job-groups/-/merge_requests/163#note_643358
But still we need to think if we need such a thing in a job or in the ci, best places would be obs sync plugin.

Actions #19

Updated by JERiveraMoya 2 months ago

  • Priority changed from Normal to Low
Actions #20

Updated by JERiveraMoya about 2 months ago

  • Parent task deleted (#151984)
Actions

Also available in: Atom PDF