coordination #18602
closed[sles][functional][epic]review existing autoyast tests (was: test fails in repos because ip and tar is not installed)
100%
Description
Acceptance criteria¶
- AC1 all autoyast tests are in a job group which has a proper review workflow (e.g. the job group does not exist anymore, all tests must be either in "SLE functional", "test development" or within the existing "autoyast" job group but properly reviewed or deleted)
Problem¶
Also there are two kind of autoyast tests, autoyast tests self-contained within os-autoinst-distri-opensuse (1) and a "AY framework created by APAC colleagues" (2).
The following approach is suggested:
- DONE: find existing stable and passing jobs from job group Regression: Autoyast and move them to sles functional as long as they are of type (1)
- DONE: review remaining failing jobs from (1), triage them and either delete if unimportant or fix and move to sles functional job group
- DONE: delete all "aytests" as it seems there are no stakeholders in these
- find reviewers (more than one)
- discuss with QA PM (mawerner) how test results should be communication and also who can act as "coordinator"
- document review workflow changes within https://wiki.microfocus.net/index.php/RD-OPS_QA/openQA_review#Functional.2Bautoyast.2Bstaging
As a reminder: The tests were added for a reason (which I don't know) and by topic they belong to sles functional (they are not about kernel, virtualization, migration, etc.) so it's in the domain of sles functional. https://wiki.microfocus.net/index.php?title=RD-OPS_QA/openQA_review#SLES.2BHA.2Bautoyast.2Bstaging also mentions the responsibility covering autoyast
Further details¶
Always latest result in this scenario: latest
old information¶
Observation¶
openQA test in scenario sle-12-SP3-Server-DVD-x86_64-autoyast_aytests_sles12_minimal+supportserver@64bit fails in
repos
Need to pre-install ip and tar binaries
Reproducible¶
Fails since (at least) Build 0333 (current job)
Expected result¶
Last good: 0328 (or more recent)
Updated by asmorodskyi over 7 years ago
it appears that test belongs to group of auto-yast tests created with some custom framework AY used for test auto-yast feature . But because general approach is to use regular testing of auto-yast via openqa was decided to moved this test into https://openqa.suse.de/group_overview/60
Updated by okurz over 7 years ago
- Subject changed from test fails in repos because ip and tar is not installed to [sles][functional]review existing autoyast tests (was: test fails in repos because ip and tar is not installed)
- Description updated (diff)
- Status changed from New to In Progress
- Assignee changed from asmorodskyi to okurz
Actually there was a misunderstanding regarding the move of the test as this does not solve anything.
Regarding the "ip command not found" a solution might be to just delete the script lines which don't work because they also previously did not work but the test sitll continued and eventually passed.
Moved passing tests to SLES functional, now waiting for next build to provide a cleaner picture on the remaining ones.
Updated by asmorodskyi over 7 years ago
I suggesting as very first step delete all "aytest" and only afterwards all other things. In this case we will not waste time on "aytests" which will be deleted in any case
Updated by okurz over 7 years ago
agreed. There is https://github.com/yast/autoyast-integration-test and these tests seem to also use http://download.opensuse.org/repositories/YaST:/Head so I guess we don't need to do that.
Updated by okurz over 7 years ago
seems like I forget "autoyast-supportserver" somehow? Strange. Jobs incomplete now in sles functional. Added autoyast-supportserver to sles functional and will try to trigger these tests again.
Updated by okurz over 7 years ago
- Priority changed from Normal to Low
We have quite some autoyast tests moved to the sles functional group now and we are getting a better understanding of them. I would like to track this still over the next weeks.
Updated by mkravec over 7 years ago
- Priority changed from Low to High
Setting priority to high because of duscussions on YaST call
Updated by mkravec over 7 years ago
Updated by mkravec over 7 years ago
- % Done changed from 0 to 100
Jobs restarted. Tar should be installed if "ip a" does not fail.
Updated by okurz over 7 years ago
- Related to action #20064: [sles][functional][yast] Bring peace and order to autoyast profiles added
Updated by mkravec over 7 years ago
- Status changed from In Progress to Resolved
- Assignee changed from okurz to mkravec
Updated by okurz over 7 years ago
- Description updated (diff)
- Status changed from Resolved to In Progress
- Assignee changed from mkravec to okurz
- Priority changed from High to Normal
Thanks for your work to fix the failures but there is still something missing: IMHO the task is not done. Would you mind if I reopen the ticket and assign back to me with a comment what is necessary to be done? The original issue is done but we expanded that ticket to "review autoyast group" which is not done yet. I updated acceptance criteria.
Updated by riafarov over 7 years ago
After couple of discussions we have following ideas:
1) Move all autoyast to separate job group. I do review of this group on daily basis. And we find another person to do meta review and take care of it when I'm not in the office.
2) Move/update all profiles in aytests repo and keep them there. Get rid of support server where possible and add precondition step to modify profiles before installation to do variables expansion, etc. We use same framework for that using same openQA test to run different verification scripts.
Updated by okurz over 7 years ago
- Related to action #23722: [sle][functional][sle15] Update autoyast profiles to be compatible with SLE15 added
Updated by okurz over 7 years ago
- Description updated (diff)
- Category changed from Bugs in existing tests to Enhancement to existing tests
Updated description. So now it's mainly about organisational changes pending.
Updated by okurz about 7 years ago
- Status changed from Feedback to In Progress
- Assignee deleted (
okurz) - Target version changed from Milestone 12 to Milestone 11
discussed with QA SLE functional PO
Updated by okurz about 7 years ago
- Subject changed from [sles][functional]review existing autoyast tests (was: test fails in repos because ip and tar is not installed) to [sles][functional][epic]review existing autoyast tests (was: test fails in repos because ip and tar is not installed)
Updated by okurz about 7 years ago
- Status changed from In Progress to Resolved
- Assignee set to okurz
Updated by okurz about 7 years ago
- Due date changed from 2017-10-11 to 2017-11-08
due to changes in a related task
Updated by riafarov about 7 years ago
- Due date changed from 2017-11-08 to 2017-10-11
due to changes in a related task
Updated by szarate about 4 years ago
- Tracker changed from action to coordination
Updated by szarate about 4 years ago
See for the reason of tracker change: http://mailman.suse.de/mailman/private/qa-sle/2020-October/002722.html