coordination #18602
closed
[sles][functional][epic]review existing autoyast tests (was: test fails in repos because ip and tar is not installed)
Added by asmorodskyi over 7 years ago.
Updated about 4 years ago.
Category:
Enhancement to existing tests
Estimated time:
(Total: 0.00 h)
Description
Acceptance criteria¶
- AC1 all autoyast tests are in a job group which has a proper review workflow (e.g. the job group does not exist anymore, all tests must be either in "SLE functional", "test development" or within the existing "autoyast" job group but properly reviewed or deleted)
Problem¶
Also there are two kind of autoyast tests, autoyast tests self-contained within os-autoinst-distri-opensuse (1) and a "AY framework created by APAC colleagues" (2).
The following approach is suggested:
- DONE: find existing stable and passing jobs from job group Regression: Autoyast and move them to sles functional as long as they are of type (1)
- DONE: review remaining failing jobs from (1), triage them and either delete if unimportant or fix and move to sles functional job group
- DONE: delete all "aytests" as it seems there are no stakeholders in these
- find reviewers (more than one)
- discuss with QA PM (mawerner) how test results should be communication and also who can act as "coordinator"
- document review workflow changes within https://wiki.microfocus.net/index.php/RD-OPS_QA/openQA_review#Functional.2Bautoyast.2Bstaging
As a reminder: The tests were added for a reason (which I don't know) and by topic they belong to sles functional (they are not about kernel, virtualization, migration, etc.) so it's in the domain of sles functional. https://wiki.microfocus.net/index.php?title=RD-OPS_QA/openQA_review#SLES.2BHA.2Bautoyast.2Bstaging also mentions the responsibility covering autoyast
Further details¶
Always latest result in this scenario: latest
old information¶
Observation¶
openQA test in scenario sle-12-SP3-Server-DVD-x86_64-autoyast_aytests_sles12_minimal+supportserver@64bit fails in
repos
Need to pre-install ip and tar binaries
Reproducible¶
Fails since (at least) Build 0333 (current job)
Expected result¶
Last good: 0328 (or more recent)
- Assignee set to asmorodskyi
it appears that test belongs to group of auto-yast tests created with some custom framework AY used for test auto-yast feature . But because general approach is to use regular testing of auto-yast via openqa was decided to moved this test into https://openqa.suse.de/group_overview/60
- Subject changed from test fails in repos because ip and tar is not installed to [sles][functional]review existing autoyast tests (was: test fails in repos because ip and tar is not installed)
- Description updated (diff)
- Status changed from New to In Progress
- Assignee changed from asmorodskyi to okurz
Actually there was a misunderstanding regarding the move of the test as this does not solve anything.
Regarding the "ip command not found" a solution might be to just delete the script lines which don't work because they also previously did not work but the test sitll continued and eventually passed.
Moved passing tests to SLES functional, now waiting for next build to provide a cleaner picture on the remaining ones.
I suggesting as very first step delete all "aytest" and only afterwards all other things. In this case we will not waste time on "aytests" which will be deleted in any case
seems like I forget "autoyast-supportserver" somehow? Strange. Jobs incomplete now in sles functional. Added autoyast-supportserver to sles functional and will try to trigger these tests again.
- Priority changed from Normal to Low
We have quite some autoyast tests moved to the sles functional group now and we are getting a better understanding of them. I would like to track this still over the next weeks.
- Priority changed from Low to High
Setting priority to high because of duscussions on YaST call
- % Done changed from 0 to 100
Jobs restarted. Tar should be installed if "ip a" does not fail.
- Related to action #20064: [sles][functional][yast] Bring peace and order to autoyast profiles added
- Status changed from In Progress to Resolved
- Assignee changed from okurz to mkravec
- Description updated (diff)
- Status changed from Resolved to In Progress
- Assignee changed from mkravec to okurz
- Priority changed from High to Normal
Thanks for your work to fix the failures but there is still something missing: IMHO the task is not done. Would you mind if I reopen the ticket and assign back to me with a comment what is necessary to be done? The original issue is done but we expanded that ticket to "review autoyast group" which is not done yet. I updated acceptance criteria.
After couple of discussions we have following ideas:
1) Move all autoyast to separate job group. I do review of this group on daily basis. And we find another person to do meta review and take care of it when I'm not in the office.
2) Move/update all profiles in aytests repo and keep them there. Get rid of support server where possible and add precondition step to modify profiles before installation to do variables expansion, etc. We use same framework for that using same openQA test to run different verification scripts.
- Status changed from In Progress to Feedback
- Assignee changed from okurz to riafarov
- Related to action #23722: [sle][functional][sle15] Update autoyast profiles to be compatible with SLE15 added
- Assignee changed from riafarov to okurz
- Description updated (diff)
- Category changed from Bugs in existing tests to Enhancement to existing tests
Updated description. So now it's mainly about organisational changes pending.
- Target version set to Milestone 12
- Status changed from Feedback to In Progress
- Assignee deleted (
okurz)
- Target version changed from Milestone 12 to Milestone 11
discussed with QA SLE functional PO
- Subject changed from [sles][functional]review existing autoyast tests (was: test fails in repos because ip and tar is not installed) to [sles][functional][epic]review existing autoyast tests (was: test fails in repos because ip and tar is not installed)
- Status changed from In Progress to Resolved
- Assignee set to okurz
So with #25604 the ACs are covered, what's coming next is covered in related tickets, e.g. cleanup of test suite names and adaptions to SLE15, e.g. see #23722
Until further notice I myself will keep the role of coordinating the efforts around autoyast and do meta-review of the review reports.
- Due date changed from 2017-10-11 to 2017-11-08
due to changes in a related task
- Due date changed from 2017-11-08 to 2017-10-11
due to changes in a related task
- Tracker changed from action to coordination
Also available in: Atom
PDF