Project

General

Custom queries

Profile

Actions

action #109707

closed

[qe-core] Detect when a maintenance update has been already released before running the test

Added by szarate over 2 years ago. Updated over 2 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Refactor/Code Improvements
Target version:
-
Start date:
2022-06-09
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Difficulty:
Sprint:
QE-Core: August Sprint (Aug 03 - Aug 31)

Description

Observation

To be refined, general idea is: when cloning or restarting a job, we should have a warning telling us that said job will fail, because the update has already been released

openQA test in scenario sle-15-SP3-Server-DVD-Updates-x86_64-mau-webserver+15@pablo-herranz/os-autoinst-distri-opensuse#php_new_version@64bit fails in
add_update_test_repo

Test suite description

Testsuite maintained at https://gitlab.suse.de/qa-maintenance/qam-openqa-yml. SLE15+ version of mau-webserver

Reproducible

Fails since (at least) Build pablo-herranz/os-autoinst-distri-opensuse#php_new_version

Expected result

Last good: pablo-herranz/os-autoinst-distri-opensuse#php_new_version (or more recent)

Further details

Always latest result in this scenario: latest

Suggestions

  • If #109902 turns out to be a successful experiment, implemment something similar for this ticket.
  • Always remind to use the job
  • Check per incident, if repository in the settings are available before tests are running

Subtasks 1 (0 open1 closed)

action #112250: [QE-Core] zypper modify repo incorrect usage is muted in the patch_and_reboot testRejecteddvenkatachala2022-06-09

Actions

Related issues 3 (1 open2 closed)

Related to openQA Project (public) - action #112349: Provide an option to always clone the latest version of the scenario run by the jobNew2022-06-13

Actions
Related to QA (public) - action #111710: [qa-tools] [tools] remove usage of *_TEST_TEMPLATE vars in qem-bot and openQA media definition if favor of *_TEST_REPOS size:MResolvedosukup2022-05-272022-09-02

Actions
Related to openQA Project (public) - action #116788: Cloning of job with released test issues not workingResolveddvenkatachala2022-09-19

Actions
Actions #3

Updated by szarate over 2 years ago

  • Sprint set to QE-Core: May Sprint (May 11 - Jun 08)
  • Description updated (diff)
  • Start date deleted (2022-04-08)

Yet another example: https://suse.slack.com/archives/C02D16TCP99/p1650956217675289:

Job: https://openqa.suse.de/tests/8632058#step/patch_and_reboot/159 failed with:

2022-04-25 20:11:26 <5> linux(5692) [zypp] Exception.cc(log):166  - [TEST_24|http://dist.suse.de/ibs/SUSE:/Maintenance:/23830/SUSE_Updates_SLE-WSM_12-SP5_x86_64/] Repository type can't be determined.
2022-04-25 20:11:26 <5> linux(5692) [zypp] Exception.cc(log):166 
2022-04-25 20:11:26 <5> linux(5692) [zypp] Exception.cc(log):166 repos.cc(refresh_raw_metadata):395 CAUGHT:   [TEST_24|http://dist.suse.de/ibs/SUSE:/Maintenance:/23830/SUSE_Updates_SLE-WSM_12-SP5_x86_64/] Valid metadata not found at specified URL
2022-04-25 20:11:26 <5> linux(5692) [zypp] Exception.cc(log):166 History:
2022-04-25 20:11:26 <5> linux(5692) [zypp] Exception.cc(log):166  - [TEST_24|http://dist.suse.de/ibs/SUSE:/Maintenance:/23830/SUSE_Updates_SLE-WSM_12-SP5_x86_64/] Repository type can't be determined.
Actions #5

Updated by dvenkatachala over 2 years ago

  • Status changed from Blocked to Workable

Anna and I had discussion, she suggested that we can implement the changes in clone_job.pl and openqa-clone-custom-git-refspec.
I will make the change as per suggestion.

Actions #8

Updated by szarate over 2 years ago

Dee to update the ticket

Actions #9

Updated by dvenkatachala over 2 years ago

I have done following changes in Script/CloneJob.pm

  1. Added a new module to verify the maintenance updates.
  2. In this module, defined a hash: the key (_TEST_TEMPLATE) => corresponding value (_TEST_ISSUE).
  3. Later this module substitutes the "@INCIDENTNR@" of TEST_TEMPLATE URLs with each corresponding TEST_ISSUE (defined in job settings) to verify the maintenance updates.

Different scenario:
In a few job settings templates, for e.g, https://openqa.suse.de/tests/8911139#settings . I see TEST_TEMPLATE with no associated incidents. How can I address such settings? Do you have any suggestions?

Further observation:
While reviewing this job(https://openqa.suse.de/tests/8911139) and I noticed a false positive in the test : https://openqa.suse.de/tests/8911139#step/patch_and_reboot/15

Actions #10

Updated by szarate over 2 years ago

While we wait for an agreement on https://github.com/os-autoinst/openQA/pull/4693

dvenkatachala wrote:

I have done following changes in Script/CloneJob.pm

  1. Added a new module to verify the maintenance updates.
  2. In this module, defined a hash: the key (_TEST_TEMPLATE) => corresponding value (_TEST_ISSUE).
  3. Later this module substitutes the "@INCIDENTNR@" of TEST_TEMPLATE URLs with each corresponding TEST_ISSUE (defined in job settings) to verify the maintenance updates.

Different scenario:
In a few job settings templates, for e.g, https://openqa.suse.de/tests/8911139#settings . I see TEST_TEMPLATE with no associated incidents. How can I address such settings? Do you have any suggestions?

I think the logic here could go towards getting all the keys using a regex, so anything that matches _TEST_TEMPLATE should have a _TEST_ISSUES pair, so in this case we have LTSS... using a regex or grepping would help in the future to automatically add new products

Further observation:
While reviewing this job(https://openqa.suse.de/tests/8911139) and I noticed a false positive in the test : https://openqa.suse.de/tests/8911139#step/patch_and_reboot/15

Dee, could you catch this particular case?

Actions #14

Updated by dvenkatachala over 2 years ago

  • Tags changed from qe-core-april-sprint, qe-core-may-sprint, qe-core-june-sprint to qe-core-april-sprint, qe-core-may-sprint

szarate wrote:

While we wait for an agreement on https://github.com/os-autoinst/openQA/pull/4693

dvenkatachala wrote:

I have done following changes in Script/CloneJob.pm

  1. Added a new module to verify the maintenance updates.
  2. In this module, defined a hash: the key (_TEST_TEMPLATE) => corresponding value (_TEST_ISSUE).
  3. Later this module substitutes the "@INCIDENTNR@" of TEST_TEMPLATE URLs with each corresponding TEST_ISSUE (defined in job settings) to verify the maintenance updates.

Different scenario:
In a few job settings templates, for e.g, https://openqa.suse.de/tests/8911139#settings . I see TEST_TEMPLATE with no associated incidents. How can I address such settings? Do you have any suggestions?

I think the logic here could go towards getting all the keys using a regex, so anything that matches _TEST_TEMPLATE should have a _TEST_ISSUES pair, so in this case we have LTSS... using a regex or grepping would help in the future to automatically add new products

Further observation:
While reviewing this job(https://openqa.suse.de/tests/8911139) and I noticed a false positive in the test : https://openqa.suse.de/tests/8911139#step/patch_and_reboot/15

Dee, could you catch this particular case?

Santi, I am sorry I made wrong analysis here. There is no issue in the test.

Actions #16

Updated by szarate over 2 years ago

Hi!, can you provide an update of the current state of the ticket?

Actions #17

Updated by dvenkatachala over 2 years ago

I could not proceed much due to Hack week and review shifts. I will continue working on this ticket after review/bugbuster.

Actions #20

Updated by dvenkatachala over 2 years ago

The PR 4693 verifies the maintenance updates availability by constructing the Template_URL from the test variables _TEST_TEMPLATE and corresponding _TEST_ISSUES pair.

Next step is to look for the MAINT_TEST_REPO instead of the _TEST_TEMPLATE from the test variable to verify the maintenance updates after the implementation of the poo#111710.

Actions #22

Updated by dvenkatachala over 2 years ago

Output snippets from the verification runs,
Aggregated jobs:

  1. Current job 9289622 will fail, because the repositories for the below updates are unavailable [ "https://download.suse.de/ibs/SUSE:/Maintenance:/25166/SUSE_Updates_SLE-Product-SLES_15-LTSS_x86_64/", "https://download.suse.de/ibs/SUSE:/Maintenance:/25296/SUSE_Updates_SLE-Product-SLES_15-LTSS_x86_64/", ] at /usr/share/openQA/script/../lib/OpenQA/Script/CloneJobSUSE.pm line 37.

2.Current job 9289071 will fail, because the repositories for the below updates are unavailable
[
"https://download.suse.de/ibs/SUSE:/Maintenance:/24448/SUSE_Updates_SLE-Module-SAP-Applications_15-SP2_x86_64/",
"https://download.suse.de/ibs/SUSE:/Maintenance:/25299/SUSE_Updates_SLE-Module-SAP-Applications_15-SP2_x86_64/",
] at /usr/share/openQA/script/../lib/OpenQA/Script/CloneJobSUSE.pm line 37.

Incident Jobs:

  1. Current job 9223278 will fail, because the repositories for the below updates are unavailable
    [
    "http://download.suse.de/ibs/SUSE:/Maintenance:/25210/SUSE_Updates_SLE-Product-SLES_15-LTSS_x86_64",
    ] at /usr/share/openQA/script/../lib/OpenQA/Script/CloneJobSUSE.pm line 37.

  2. Current job 9109827 will fail, because the repositories for the below updates are unavailable
    [
    "http://download.suse.de/ibs/SUSE:/Maintenance:/24981/SUSE_Updates_SLE-Product-SLES_15-SP1-LTSS_x86_64",
    ] at /usr/share/openQA/script/../lib/OpenQA/Script/CloneJobSUSE.pm line 37.

Actions #23

Updated by dvenkatachala over 2 years ago

  • Status changed from In Progress to Resolved
Actions

Also available in: Atom PDF