action #25032

enable openQA maintenance testing

Added by lnussel over 2 years ago. Updated almost 2 years ago.

Status:ResolvedStart date:01/04/2018
Priority:HighDue date:19/04/2018
Assignee:okurz% Done:

100%

Category:QA
Target version:Stabilized
Duration: 14

Description

openQA and the bot needs to be set up to also test maintenance updates for the new release


Related issues

Related to openQA Project - action #35377: dump-templates renders versions not always as string Rejected 23/04/2018
Related to openQA Tests - action #35694: [qam] test fails in welcome - Tests not adapted for "beta... Closed 30/04/2018
Related to openQA Tests - action #35604: [opensuse][functional][u] test fails in gnucash, not clos... Resolved 26/04/2018 17/07/2018
Related to openQA Tests - action #34717: [functional][u][easy][fast] adjust gnucash test per gnuca... Resolved 11/04/2018 19/06/2018
Related to openQA Tests - action #35685: [opensuse][functional][u][sporadic][medium] Various unsta... Resolved 30/04/2018 17/07/2018
Related to openQA Tests - action #35994: [functional][opensuse][y][leap15.0][fast] adapt maintenan... Resolved 08/05/2018 22/05/2018
Copied from openSUSE Leap 42.3 Release - action #16996: enable openQA maintenance testing Resolved 02/06/2017 16/06/2017
Copied to openSUSE Leap 15.1 - action #39449: enable openQA maintenance testing Resolved 11/03/2019 05/04/2019

History

#1 Updated by lnussel over 2 years ago

  • Copied from action #16996: enable openQA maintenance testing added

#2 Updated by lnussel about 2 years ago

  • Due date changed from 02/03/2018 to 15/04/2018
  • Start date changed from 16/02/2018 to 01/04/2018

#3 Updated by lnussel almost 2 years ago

  • Category changed from QA to QA
  • Assignee set to okurz

Not sure who would take care of this nowadays but it makes sense to look into setting this up early.

#4 Updated by lnussel almost 2 years ago

does this work meanwhile?

#5 Updated by okurz almost 2 years ago

I don't think anyone has done anything here yet. I wanted to talk to coolo personally, then I forgot about it, then I missed the opportunity again, then … sorry :(

Will address in daily jangouts call.

#6 Updated by lnussel almost 2 years ago

  • Due date changed from 15/04/2018 to 19/04/2018
  • Priority changed from Normal to High

#7 Updated by okurz almost 2 years ago

  • Status changed from New to Feedback

coolo is reworking openSUSE maintenance tests, plan to have leap 15.0 maintenance tests after 42.3 calmed down. SLE had a "new" approach since 2017-07 which will also be applied to openSUSE tests now. Maint.+QAM plans to drive the work on the tests to have them ready for SLE15+Leap15.0

#8 Updated by okurz almost 2 years ago

I created openQA job groups openSUSE Leap 15.0 Updates and … Incidents with the test plan taken over from Leap 42.3 for now:

for i in Incidents Updates ; do openqa-dump-templates --host https://openqa.opensuse.org --group "openSUSE Leap 42.3 $i" --full | sed -e 's/42\.3/15.0/g' > leap_15.0_$i && openqa-load-templates --host https://openqa.opensuse.org leap_15.0_$i ; done

And manually using the webui on https://openqa.opensuse.org/admin/groups put the job group in the right position as subgroup to openSUSE Leap behind the other openSUSE Leap 15.0 jobs.

Now, how to ensure the maintenance bot can make use of these? And where can I monitor the bot operating, e.g. tail logfile or screen session to attach to?

#9 Updated by okurz almost 2 years ago

  • Related to action #35377: dump-templates renders versions not always as string added

#11 Updated by lnussel almost 2 years ago

what is the status here?

#12 Updated by okurz almost 2 years ago

coolo and tacit setup the openqa-maintenance bot to work aligned with the SLE workflow. test-updates are triggered, openQA tests fail due to necessary test adaption. I understood that SUSE QAM confirmed that they would take responsibility for the necessary openQA test adaptions but I have not seen effects of this so far. I would adress the question again about the current state to coolo latest on next RM call, e.g. next Monday or Wednesday.

#13 Updated by okurz almost 2 years ago

  • Related to action #35694: [qam] test fails in welcome - Tests not adapted for "beta warning" on Leap 15 with updates added

#14 Updated by lnussel almost 2 years ago

what is the status here?

#15 Updated by okurz almost 2 years ago

Incidents and Updates tests are in place, tests can work in principle, e.g. see textmode passing. tacit wanted to clarify who are the responsibles persons from maintenance/QAM for adapting the openQA tests as well as regular test review. QSF (QA SLE functional) would stand ready for support but only for exceptional handling.

#16 Updated by lnussel almost 2 years ago

Looks like for install_with_updates the installation/setup_online_repos test needs to be called earlier. Could you just fix that? We really need that setup up ASAP.

other tests fail in gnucash due to low disk space and in applying online updates. Not sure why that happens given that the same test works in the main project.

The kde test fails in applying online updates due to outdated disk images which results in too many updates installed. Could you move the symlink on openqa.o.o forward please?

#17 Updated by lnussel almost 2 years ago

  • % Done changed from 0 to 70

#18 Updated by okurz almost 2 years ago

  • Status changed from Feedback to In Progress

Ok, let's see what we can do.

lnussel wrote:

The kde test fails in applying online updates due to outdated disk images which results in too many updates installed. Could you move the symlink on openqa.o.o forward please?

Done.

As root on o3:

cd /var/lib/openqa/share/factory/iso/fixed/
rm openSUSE-Leap-15.0-DVD-x86_64.iso
rm openSUSE-Leap-15.0-Build213.2-DVD-x86_64.iso
ln ../openSUSE-Leap-15.0-DVD-x86_64-Build235.2-Media.iso openSUSE-Leap-15.0-DVD-x86_64.iso

#19 Updated by okurz almost 2 years ago

  • Related to action #35604: [opensuse][functional][u] test fails in gnucash, not closing correctly added

#20 Updated by okurz almost 2 years ago

  • Related to action #34717: [functional][u][easy][fast] adjust gnucash test per gnucash 3.0 update added

#21 Updated by okurz almost 2 years ago

  • Related to action #35685: [opensuse][functional][u][sporadic][medium] Various unstable tests on o3 - user_gui_login + gnucash added

#22 Updated by okurz almost 2 years ago

  • Related to action #35994: [functional][opensuse][y][leap15.0][fast] adapt maintenance tests for install_with_updates - test fails in setup_online_repos added

#23 Updated by okurz almost 2 years ago

  • Status changed from In Progress to Resolved
  • % Done changed from 70 to 100

incident tests are fine, update tests are also all ok with https://openqa.opensuse.org/tests/overview?distri=opensuse&version=15.0&build=20180508-4&groupid=55 so I guess we are done here

#24 Updated by lnussel over 1 year ago

  • Copied to action #39449: enable openQA maintenance testing added

Also available in: Atom PDF