action #109707
closed[qe-core] Detect when a maintenance update has been already released before running the test
Added by szarate over 2 years ago. Updated over 2 years ago.
100%
Description
Observation¶
To be refined, general idea is: when cloning or restarting a job, we should have a warning telling us that said job will fail, because the update has already been released
openQA test in scenario sle-15-SP3-Server-DVD-Updates-x86_64-mau-webserver+15@pablo-herranz/os-autoinst-distri-opensuse#php_new_version@64bit fails in
add_update_test_repo
Test suite description¶
Testsuite maintained at https://gitlab.suse.de/qa-maintenance/qam-openqa-yml. SLE15+ version of mau-webserver
Reproducible¶
Fails since (at least) Build pablo-herranz/os-autoinst-distri-opensuse#php_new_version
Expected result¶
Last good: pablo-herranz/os-autoinst-distri-opensuse#php_new_version (or more recent)
Further details¶
Always latest result in this scenario: latest
Suggestions¶
- If #109902 turns out to be a successful experiment, implemment something similar for this ticket.
- Always remind to use the job
- Check per incident, if repository in the settings are available before tests are running
Updated by szarate over 2 years ago
- Tags deleted (
to-be-refined, qe-core-april-sprint) - Description updated (diff)
Updated by szarate over 2 years ago
- Sprint set to QE-Core: May Sprint (May 11 - Jun 08)
- Description updated (diff)
- Start date deleted (
2022-04-08)
Yet another example: https://suse.slack.com/archives/C02D16TCP99/p1650956217675289:
Job: https://openqa.suse.de/tests/8632058#step/patch_and_reboot/159 failed with:
2022-04-25 20:11:26 <5> linux(5692) [zypp] Exception.cc(log):166 - [TEST_24|http://dist.suse.de/ibs/SUSE:/Maintenance:/23830/SUSE_Updates_SLE-WSM_12-SP5_x86_64/] Repository type can't be determined.
2022-04-25 20:11:26 <5> linux(5692) [zypp] Exception.cc(log):166
2022-04-25 20:11:26 <5> linux(5692) [zypp] Exception.cc(log):166 repos.cc(refresh_raw_metadata):395 CAUGHT: [TEST_24|http://dist.suse.de/ibs/SUSE:/Maintenance:/23830/SUSE_Updates_SLE-WSM_12-SP5_x86_64/] Valid metadata not found at specified URL
2022-04-25 20:11:26 <5> linux(5692) [zypp] Exception.cc(log):166 History:
2022-04-25 20:11:26 <5> linux(5692) [zypp] Exception.cc(log):166 - [TEST_24|http://dist.suse.de/ibs/SUSE:/Maintenance:/23830/SUSE_Updates_SLE-WSM_12-SP5_x86_64/] Repository type can't be determined.
Updated by dvenkatachala over 2 years ago
- Status changed from Blocked to Workable
Anna and I had discussion, she suggested that we can implement the changes in clone_job.pl and openqa-clone-custom-git-refspec.
I will make the change as per suggestion.
Updated by dvenkatachala over 2 years ago
- Status changed from Workable to In Progress
Updated by dvenkatachala over 2 years ago
I have done following changes in Script/CloneJob.pm
- Added a new module to verify the maintenance updates.
- In this module, defined a hash: the key (_TEST_TEMPLATE) => corresponding value (_TEST_ISSUE).
- Later this module substitutes the "@INCIDENTNR@" of TEST_TEMPLATE URLs with each corresponding TEST_ISSUE (defined in job settings) to verify the maintenance updates.
Different scenario:
In a few job settings templates, for e.g, https://openqa.suse.de/tests/8911139#settings . I see TEST_TEMPLATE with no associated incidents. How can I address such settings? Do you have any suggestions?
Further observation:
While reviewing this job(https://openqa.suse.de/tests/8911139) and I noticed a false positive in the test : https://openqa.suse.de/tests/8911139#step/patch_and_reboot/15
Updated by szarate over 2 years ago
While we wait for an agreement on https://github.com/os-autoinst/openQA/pull/4693
dvenkatachala wrote:
I have done following changes in Script/CloneJob.pm
- Added a new module to verify the maintenance updates.
- In this module, defined a hash: the key (_TEST_TEMPLATE) => corresponding value (_TEST_ISSUE).
- Later this module substitutes the "@INCIDENTNR@" of TEST_TEMPLATE URLs with each corresponding TEST_ISSUE (defined in job settings) to verify the maintenance updates.
Different scenario:
In a few job settings templates, for e.g, https://openqa.suse.de/tests/8911139#settings . I see TEST_TEMPLATE with no associated incidents. How can I address such settings? Do you have any suggestions?
I think the logic here could go towards getting all the keys using a regex, so anything that matches _TEST_TEMPLATE should have a _TEST_ISSUES pair, so in this case we have LTSS... using a regex or grepping would help in the future to automatically add new products
Further observation:
While reviewing this job(https://openqa.suse.de/tests/8911139) and I noticed a false positive in the test : https://openqa.suse.de/tests/8911139#step/patch_and_reboot/15
Dee, could you catch this particular case?
Updated by szarate over 2 years ago
- Sprint changed from QE-Core: May Sprint (May 11 - Jun 08) to QE-Core: June Sprint (Jun 08 - Jul 06)
Updated by dvenkatachala over 2 years ago
- Tags changed from qe-core-april-sprint, qe-core-may-sprint, qe-core-june-sprint to qe-core-april-sprint, qe-core-may-sprint
szarate wrote:
While we wait for an agreement on https://github.com/os-autoinst/openQA/pull/4693
dvenkatachala wrote:
I have done following changes in Script/CloneJob.pm
- Added a new module to verify the maintenance updates.
- In this module, defined a hash: the key (_TEST_TEMPLATE) => corresponding value (_TEST_ISSUE).
- Later this module substitutes the "@INCIDENTNR@" of TEST_TEMPLATE URLs with each corresponding TEST_ISSUE (defined in job settings) to verify the maintenance updates.
Different scenario:
In a few job settings templates, for e.g, https://openqa.suse.de/tests/8911139#settings . I see TEST_TEMPLATE with no associated incidents. How can I address such settings? Do you have any suggestions?I think the logic here could go towards getting all the keys using a regex, so anything that matches
_TEST_TEMPLATEshould have a_TEST_ISSUESpair, so in this case we have LTSS... using a regex or grepping would help in the future to automatically add new productsFurther observation:
While reviewing this job(https://openqa.suse.de/tests/8911139) and I noticed a false positive in the test : https://openqa.suse.de/tests/8911139#step/patch_and_reboot/15Dee, could you catch this particular case?
Santi, I am sorry I made wrong analysis here. There is no issue in the test.
Updated by szarate over 2 years ago
- Related to action #112349: Provide an option to always clone the latest version of the scenario run by the job added
Updated by szarate over 2 years ago
Hi!, can you provide an update of the current state of the ticket?
Updated by dvenkatachala over 2 years ago
I could not proceed much due to Hack week and review shifts. I will continue working on this ticket after review/bugbuster.
Updated by szarate over 2 years ago
- Sprint changed from QE-Core: June Sprint (Jun 08 - Jul 06) to QE-Core: July Sprint (Jul 06 - Aug 03)
Updated by dvenkatachala over 2 years ago
- Related to action #111710: [qa-tools] [tools] remove usage of *_TEST_TEMPLATE vars in qem-bot and openQA media definition if favor of *_TEST_REPOS size:M added
Updated by dvenkatachala over 2 years ago
The PR 4693 verifies the maintenance updates availability by constructing the Template_URL from the test variables _TEST_TEMPLATE and corresponding _TEST_ISSUES pair.
Next step is to look for the MAINT_TEST_REPO instead of the _TEST_TEMPLATE from the test variable to verify the maintenance updates after the implementation of the poo#111710.
Updated by szarate over 2 years ago
- Sprint changed from QE-Core: July Sprint (Jul 06 - Aug 03) to QE-Core: August Sprint (Aug 03 - Aug 31)
Updated by dvenkatachala over 2 years ago
Output snippets from the verification runs,
Aggregated jobs:
- Current job 9289622 will fail, because the repositories for the below updates are unavailable [ "https://download.suse.de/ibs/SUSE:/Maintenance:/25166/SUSE_Updates_SLE-Product-SLES_15-LTSS_x86_64/", "https://download.suse.de/ibs/SUSE:/Maintenance:/25296/SUSE_Updates_SLE-Product-SLES_15-LTSS_x86_64/", ] at /usr/share/openQA/script/../lib/OpenQA/Script/CloneJobSUSE.pm line 37.
2.Current job 9289071 will fail, because the repositories for the below updates are unavailable
[
"https://download.suse.de/ibs/SUSE:/Maintenance:/24448/SUSE_Updates_SLE-Module-SAP-Applications_15-SP2_x86_64/",
"https://download.suse.de/ibs/SUSE:/Maintenance:/25299/SUSE_Updates_SLE-Module-SAP-Applications_15-SP2_x86_64/",
] at /usr/share/openQA/script/../lib/OpenQA/Script/CloneJobSUSE.pm line 37.
Incident Jobs:
Current job 9223278 will fail, because the repositories for the below updates are unavailable
[
"http://download.suse.de/ibs/SUSE:/Maintenance:/25210/SUSE_Updates_SLE-Product-SLES_15-LTSS_x86_64",
] at /usr/share/openQA/script/../lib/OpenQA/Script/CloneJobSUSE.pm line 37.Current job 9109827 will fail, because the repositories for the below updates are unavailable
[
"http://download.suse.de/ibs/SUSE:/Maintenance:/24981/SUSE_Updates_SLE-Product-SLES_15-SP1-LTSS_x86_64",
] at /usr/share/openQA/script/../lib/OpenQA/Script/CloneJobSUSE.pm line 37.
Updated by dvenkatachala over 2 years ago
- Status changed from In Progress to Resolved
Updated by okurz about 2 years ago
- Related to action #116788: Cloning of job with released test issues not working added