action #39449
closedenable openQA maintenance testing
Description
openQA and the bot needs to be set up to also test maintenance updates for the new release
Updated by lnussel over 5 years ago
- Copied from action #25032: enable openQA maintenance testing added
Updated by lnussel over 5 years ago
- Due date changed from 2018-12-25 to 2019-03-15
- Start date changed from 2018-12-07 to 2019-03-11
Updated by lnussel about 5 years ago
- Due date changed from 2019-03-15 to 2019-03-22
Updated by okurz about 5 years ago
- Status changed from New to Feedback
First collected complete template dump with openqa-dump-templates --host https://openqa.opensuse.org --full > o3_templates_$(date +%F) . Then added scenarios with
old=15.0; new=15.1; for i in Incidents Updates ; do openqa-dump-templates --host https://openqa.opensuse.org --group "openSUSE Leap $old $i" --full | sed -e "s/$old/$new/g" > leap_${new}_$i && openqa-load-templates --host https://openqa.opensuse.org ./leap_${new}_$i ; done
This created the job groups visible on https://openqa.opensuse.org/admin/groups , reordered them next to the "15.0" and "42.3" update jobs by drag & drop.
Created a test maintenance request in
https://build.opensuse.org/request/show/684360
and bot adaptions in https://github.com/openSUSE/openSUSE-release-tools/pull/1903
Updated by lnussel about 5 years ago
So before your absence can you hand this over to someone to finish please?
Updated by okurz about 5 years ago
- Assignee changed from okurz to coolo
@coolo can you please take over?
Updated by lnussel about 5 years ago
- Status changed from Feedback to In Progress
- Assignee set to pluskalm
Updated by lnussel about 5 years ago
- Due date changed from 2019-03-22 to 2019-04-05
Updated by pcervinka about 5 years ago
Job groups were setup by Marcus last week with Ondrej's assistance.
I will check failures in setup_online_repos:
https://openqa.opensuse.org/tests/890548#step/setup_online_repos/2
Updated by lnussel about 5 years ago
- Assignee changed from pluskalm to pcervinka
Updated by pcervinka about 5 years ago
- Related to action #49628: [qam] test fails in setup_online_repos - missing configure online repositories added
Updated by pcervinka about 5 years ago
Created bug for missing dialog for online repositories: https://bugzilla.opensuse.org/show_bug.cgi?id=1130390
Updated by pcervinka about 5 years ago
So, I finished review of Leap 15.1 Updates and Incidents group. We are blocked by bsc#1130390, there are no other issues so far.
Updated by pcervinka about 5 years ago
- Status changed from In Progress to Blocked
Blocked by bsc#1130390.
Updated by pcervinka about 5 years ago
- Status changed from Blocked to In Progress
Checking results with latest iso.
Updated by pcervinka about 5 years ago
Latest iso with skelcd fix was released, but tests are still failing.
Step in online_repos https://openqa.opensuse.org/tests/904506#step/online_repos/1 was never designed to be clickable on "YES" button. It just matches needle online-repos-popup and continues by pressing "alt-n". In Leap15.0 it means usual Next, but in 15.1 it means "NO" button to activate online repositories.
[2019-04-11T07:47:19.148 CEST] [debug] >>> testapi::_handle_found_needle: found online-repos-popup-20190303, similarity 1.00 @ 49/259
[2019-04-11T07:47:19.148 CEST] [debug] /var/lib/openqa/cache/openqa1-opensuse/tests/opensuse/tests/installation/online_repos.pm:67 called testapi::wait_screen_change
[2019-04-11T07:47:19.148 CEST] [debug] <<< testapi::wait_screen_change(timeout=10, similarity_level=50)
[2019-04-11T07:47:19.149 CEST] [debug] /var/lib/openqa/cache/openqa1-opensuse/tests/opensuse/tests/installation/online_repos.pm:67 called testapi::send_key
[2019-04-11T07:47:19.149 CEST] [debug] <<< testapi::send_key(key='alt-n', do_wait=0)
In Leap 15.0 repositories are activated in setup_online_repos https://openqa.opensuse.org/tests/904522#step/setup_online_repos/1, by clicking on "Configure online repositories", but this is not present in Leap15.1.
It is product behavior change. Is it expected?
@lnussel what should be correct behavior and what should be done to fix it? Could be this change caught sooner on TW, if it had update repositories enabled by variables? I suppose, this scenario is not covered by usual TW runs.
If it is a test change, we will have to update behavior in online_repos and setup_online_repos. Or can we have "Configure online repositories" in the same place like in 15.0?
Updated by lnussel about 5 years ago
with the switch to the role selection dialog the behavior you see in 15.1 now is correct. So the test has to be adjusted.
Updated by pcervinka about 5 years ago
- Blocked by action #50309: [qam][blue][leap][opensuse] Update online_repos behavior to activate online repositories added
Updated by pcervinka about 5 years ago
- Status changed from In Progress to Blocked
Updated by pcervinka about 5 years ago
Created https://progress.opensuse.org/issues/50309, but don't have space to work on it.
Updated by pcervinka about 5 years ago
Reshuffled my priorities, I will work on poo#50309.
Updated by pcervinka about 5 years ago
Created bug https://bugzilla.opensuse.org/show_bug.cgi?id=1132609 for issue with repositories.
File http://download.opensuse.org/YaST/Repos/_openSUSE_Leap_15.1_Default.xml contains incorrect Leap version:
curl -s http://download.opensuse.org/YaST/Repos/_openSUSE_Leap_15.1_Default.xml | grep download.opensuse.org
<url>http://download.opensuse.org/distribution/leap/$releasever/repo/oss/</url>
<url>http://download.opensuse.org/distribution/leap/$releasever/repo/non-oss/</url>
<url>http://download.opensuse.org/source/distribution/leap/$releasever/repo/oss/</url>
<url>http://download.opensuse.org/debug/distribution/leap/$releasever/repo/oss/</url>
<url>http://download.opensuse.org/update/leap/$releasever/oss</url>
<url>http://download.opensuse.org/update/leap/$releasever/non-oss/</url>
<url>http://download.opensuse.org/debug/update/leap/$releasever/oss</url>
<url>http://download.opensuse.org/update/leap/$releasever-test/</url>
There are just variable names.
Updated by pcervinka about 5 years ago
Faulty behavior reported in bsc#1132609 was issue of build 447.1, latest build 450.2 is fine.
Updated by pcervinka about 5 years ago
- Blocked by action #50459: [qam][blue][leap] test fails in updates_packagekit_gpk - fail after update added
Updated by pcervinka about 5 years ago
- Blocked by deleted (action #50459: [qam][blue][leap] test fails in updates_packagekit_gpk - fail after update)
Updated by pcervinka about 5 years ago
- Status changed from Blocked to Feedback
Today we merged fix for setup of repositories and updated link to latest iso on ooo(thx to ludwig).
And we got first "green" Updates run of Leap 15.1:
https://openqa.opensuse.org/tests/overview?distri=opensuse&version=15.1&build=20190417-1&groupid=61
Here is the quick summary of issues:
bsc#1130390 - Bug in skelcd-control-openSUSE, missing popup about online repositories => product fixed
poo#50309 - Setup of online repositories was not ready for 15.1 => openQA test fixed
bsc#1132609 - $releasever was expanded by yast, but only issue of 447.1 build, build 450.2 is ok => bug closed as invalid
poo#50459 - failure in updates_packagekit_gpk, was only on ooo, couldn't reproduce on private instance, disappeared with latest iso => hmm/meh(?)
Single incident for hello package is green as well:
https://openqa.opensuse.org/tests/overview?distri=opensuse&version=15.1&build=%3A9859%3Ahello.1554960697&groupid=60
There is only problem with update-test-trivial, which fail at qam_verify_package_install:
https://openqa.opensuse.org/tests/overview?distri=opensuse&version=15.1&build=%3A8555%3Aupdate-test-trivial.1553528547&groupid=60
According to Marcus, we don't need to go deeper into this.
I will continue with reviewing of results for next few runs.
Updated by pcervinka about 5 years ago
Removed BETA=1 from medium 15.1 DVD-Updates and DVD-Incidents, latest iso failed with https://openqa.opensuse.org/tests/915581#step/welcome/9. Leap 15.1 is in RC phase now.
Updated by pcervinka about 5 years ago
- % Done changed from 30 to 90
Leap 15.1 Updates are green again https://openqa.opensuse.org/tests/overview?distri=opensuse&version=15.1&build=20190424-1&groupid=61
Updated by pcervinka about 5 years ago
- Status changed from Feedback to Resolved
- % Done changed from 90 to 100
There is no pending task(rechecked with Marcus).
Updated by lnussel over 4 years ago
- Copied to action #56330: enable openQA maintenance testing added
Updated by szarate over 3 years ago
- Related to action #72202: [qe-core] setup openSUSE Leap 15.3 openQA added