action #17042
closedcoordination #15108: [sle][functional][u][epic] Modules (Installation + migration)
[sles][functional][modules] Modules installation
0%
Description
goal¶
Make sure that each module installs correctly.
acceptance criteria¶
- AC1: One module is tested on every SLES build using proxy-SCC
- AC2: Multiple modules are tested
tasks¶
- set
SCC_ADDONS=<module_name1>[,<module_name2>]
accordingly (SCC regurl should only register SLES, not point to any module) - after the module(s) are registered and enabled packages should be installed from the modules, e.g. by selecting the appropriate pattern(s)
further details¶
As SLE service packs and therefore also builds within service packs are meant to be ABI compatible modules can be simply installed over SCC while registering the latest SLES build. For all modules that are already on SCC we just install them over SCC (That's it. No need to sync anything anywhere).
Enabling modules does not install anything from it automatically. patterns or packages need to be selected, e.g. after enabling "hpcm" then "zypper search hpc" should return something. Also the repository can be explicitly selected with "zypper lr" and then all packages and patterns from that repo can be listed with "zypper search -r ".
So far module installation was never done besides textmode during online migration but these are actually "post-validation" tests as described in #15800.
Files
Updated by maritawerner almost 8 years ago
- Related to coordination #15108: [sle][functional][u][epic] Modules (Installation + migration) added
Updated by maritawerner almost 8 years ago
okurz: I think that that should be covered by the sle functional team? Do you agree? Any feedback is welcome.
Updated by okurz almost 8 years ago
- Related to deleted (coordination #15108: [sle][functional][u][epic] Modules (Installation + migration))
Updated by okurz almost 8 years ago
- Subject changed from [sles][functional] Modules Installation to [sles][functional] Modules Installation
- Category set to New test
Updated by maritawerner almost 8 years ago
Modules for SP3 are not enabled in SCC yet. I wait for Anja/Frederic to get back to us.
Updated by dgutu almost 8 years ago
Modules are available while registering the system on SCC-Proxy.
Only Certifications module is missing. Will give it a try on normal SCC.
Updated by dgutu almost 8 years ago
- Certification modules is not available for our scc key.
- Legacy Module is failing to download license, emailed Arteom about this.
Updated by dgutu almost 8 years ago
Legacy and Web Scripting needs license to be synced, rsync should be adapted.
Other modules like, Public Cloud, ToolChain, Container, Advances System Management Modules are fine.
@Marita, what about Certification Module, is it on PRD as available one?
Updated by maritawerner almost 8 years ago
Andreas Stieger, the security PJM, confirmed to me that atm there is no certifications module for SP2 and for SP3. They will develop a new module but he expects that to be available in July. So we can skip all installation/Migration scenarios with the certifications module.
Updated by dgutu almost 8 years ago
Thx for the info Marita.
As per today this one is running in test development group with 4 modules from 6.
https://openqa.suse.de/tests/808525
I'm working on rsync.pl to enable the remaining 2 modules to this testmodule.
Updated by dgutu over 7 years ago
The rsync.pl syncs everything related to the latest SP and staging but modules are related to GA,
because of that I'm still working on it.
Updated by maritawerner over 7 years ago
Dumitru, could you please add more details into the ticket description or split the ticket. Please check PRD section 1.4.2.1.
Updated by asmorodskyi over 7 years ago
- Related to action #9464: [sle][functional][u] Modules - Advanced Systems Management added
Updated by asmorodskyi over 7 years ago
- Related to coordination #9466: [sle][functional][epic][medium][u]Modules - Containers: Test the SLE module for containers added
Updated by asmorodskyi over 7 years ago
- Related to coordination #9470: [sle][functional][u][sle12][medium][epic]Modules - Web and Scripting added
Updated by asmorodskyi over 7 years ago
- Related to deleted (coordination #9470: [sle][functional][u][sle12][medium][epic]Modules - Web and Scripting)
Updated by asmorodskyi over 7 years ago
- Related to deleted (coordination #9466: [sle][functional][epic][medium][u]Modules - Containers: Test the SLE module for containers)
Updated by asmorodskyi over 7 years ago
- Related to deleted (action #9464: [sle][functional][u] Modules - Advanced Systems Management)
Updated by dgutu over 7 years ago
- Subject changed from [sles][functional] Modules Installation to [sles][functional][modules] Modules installation
Updated by okurz over 7 years ago
What I did as an experiment:
- on osd:
rsync -aHP rsync://dist.suse.de/ibs/SUSE/Products/SLE-Module-Adv-Systems-Management/12/x86_64/product/ /var/lib/openqa/share/factory/repo/SLE-12-Module-Adv-Systems-Management-POOL-x86_64-Build20170412-Media1/
- openqa_clone_job_osd 871704 _GROUP=0 SCC_ADDONS=asmm SCC_URL=http://Server-0326.Module-Adv-Systems-Management-20170412.proxy.scc.suse.de -> https://openqa.suse.de/tests/874996#live, also see https://openqa.suse.de/tests/874996/file/video.ogv
- installed with module enabled, then manually installed all packages from that module
so that works.
Updated by dgutu over 7 years ago
Thank you for the try but I found some caveats.
- When module is selected by using registration you need to get this module as a product on the installation settings page and respective patterns.
- When I tried to use regurl and include the once you have used I got the module in the product list but the installer gives a warning that the packages can't be installed. Continue to work on that.
Updated by dgutu over 7 years ago
I found that there are 2 types sources for modules.
http://dist.suse.de/ibs/SUSE:/SLE-12:/GA/images/repo/SLE-12-Module*
Offering:
- correct structure of the folder and files so we can sync this to ftp:/openqa.suse.de
Doesn't offer:
- It contains only 4 modules from 6 (asmm, lgm, pcm and wsm - present), (tcm and contm - missing)
- very limited number of packages and old version
http://dist.suse.de/ibs/SUSE/Updates/
Offering:
- Latest and updated packages for all modules
Doesn't offer:
- Missing license files for lgm and wsm modules
- Folder and file organized like repo and not like ISO
I'm confused now...
Updated by maritawerner over 7 years ago
Answer from Stefan Behlert:
The first link is the pool channel added to the ibs for GA. At that time only 4 modules existed. The newer modules must have other channels that were added for SP1/SP2.
The second channel is the update channel for all modules were you can find all updates for all modules. Licences are not in there because they are in the pool channel.
If you want to know more here maybe ask a colleague from QAM. Mantienance are the experts around channels, etc.
Updated by okurz over 7 years ago
- Description updated (diff)
- Status changed from New to In Progress
I guess we got confused in before and the approach is way easier than thought. Updated description accordingly.
$ openqa_clone_job_osd 916988 _GROUP=0 SCC_ADDONS=asmm TEST=gnome+sdk+proxy_SCC+asmm_okurz
-> sle-12-SP3-Server-DVD-x86_64-Build0367-gnome+sdk+proxy_SCC+asmm_okurz
Just in case if we would need to test modules which are not yet released over SCC or when we would need to test a specific version of modules then we would need to sync them to osd and "fake" the repositories. This has been mentioned in #12908#note-4 and it seems coolo confused me or meant something different.
Updated by okurz over 7 years ago
aha, and https://openqa.suse.de/tests/919019#step/consoletest_setup/32 failed now so … proxy-SCC issue?
Updated by okurz over 7 years ago
https://github.com/SUSE/scc-qa-proxy/issues/14 is the feature request. I asked the SCC team if someone can take a look.
Updated by okurz over 7 years ago
In case this is not feasible to do within proxy-SCC then we need to follow the approach of syncing everything to osd for testing but I am wondering about the "pool/product" and "update" part. https://openqa.suse.de/tests/919019#step/consoletest_setup/32 shows that when specifying only SLES in the regurl both "…-Pool" and "…-Updates" now point to the same directory on osd.
For the repos there are seemingly multiple versions:
- e.g. http://dist.suse.de/ibs/SUSE:/SLE-12:/GA/images/repo/ - looks like the repo of the module at the point in time when SLE 12 GA was released
- http://dist.suse.de/ibs/SUSE/Products/SLE-Module-Adv-Systems-Management/12/x86_64/product/ - probably the "last development release" of the module before SLE 12 GA (it's one build below the upper one)
- http://dist.suse.de/ibs/SUSE/Updates/SLE-Module-Adv-Systems-Management/12/x86_64/update/ - updates including all recent packages of the module
I want to crosscheck how it looks like when we specify the module in the regurl as well but osd is currently down.
Updated by okurz over 7 years ago
- File canvas.png canvas.png added
$ openqa_clone_job_osd 916988 _GROUP=0 SCC_ADDONS=asmm TEST=gnome+sdk+proxy_SCC+asmm_okurz SCC_URL=http://Server-0367.Module-Adv-Systems-Management-20170506.proxy.scc.suse.de
Created job #920359: sle-12-SP3-Server-DVD-x86_64-Build0367-gnome+sdk+proxy_SCC_without_addon@64bit
-> https://openqa.suse.de/t920359
From the following screenshot
we can see how the repositories look like after rewriting from proxy-SCC when also the module is specified in SCC_URL: http://openqa.suse.de/assets/repo/SLE-12-Module-Adv-Systems-Management-POOL-x86_64-Build20170506-Media1/ but this is the same URL for both the Pool and Updates repository. I see this as another problem because when both repository URL yield the same path we can not easily sync both Pool and Updates but need to merge them. So even more messy.
As an experiment nevertheless I synced both Pool and Updates into the target path on osd:
for path in SUSE:/SLE-12:/GA/images/repo/SLE-12-Module-Adv-Systems-Management-POOL-x86_64-Build0096-Media1/ SUSE/Updates/SLE-Module-Adv-Systems-Management/12/x86_64/update/ ; do rsync -aHP rsync://dist.suse.de/ibs/$path /var/lib/openqa/share/factory/repo/SLE-12-Module-Adv-Systems-Management-POOL-x86_64-Build20170506-Media1/ ; done
yielding ftp://openqa.suse.de/SLE-12-Module-Adv-Systems-Management-POOL-x86_64-Build20170506-Media1/ , a messy merged directory. If this is the approach to go then for modules which where not existant at the time of GA we might need to find a corresponding pool directory first from a later SP or a products directory elsewhere. Retriggered above's openQA job
-> https://openqa.suse.de/tests/920360#live
to see if the repository can be at least refreshed properly.
That failed with
$ curl -s https://openqa.suse.de/tests/920360/file/start_install-y2logs.tar.bz2 | tar -xjf - YaST2/badlist -O
2 packages failed
nothing provides product_flavor(sle-module-adv-systems-management) needed by product:sle-module-adv-systems-management-12-0.x86_64
nothing provides product_flavor(sle-module-adv-systems-management) needed by product:sle-module-adv-systems-management-12-0.x86_64
so apparently the pool directory is incomplete and does not provide the product properly anymore. Do we really need to reverse engineer the update channels management infrastructure here?
Updated by okurz over 7 years ago
Updated by okurz over 7 years ago
- Status changed from In Progress to Feedback
- Assignee changed from dgutu to okurz
There is a pull request open for scc-qa-proxy to only rewrite specified products. I think that is the right approach and we should wait for the PR to be be active on proxy-SCC before continuing.
Updated by okurz over 7 years ago
The PR to proxy-SCC was merged. Let's see if it already has an effect -> https://openqa.suse.de/tests/925867#live
https://openqa.suse.de/tests/925867#step/zypper_lr/6 shows that the module repositories are not rewritten and the ones from SCC are used which is what we wanted here.
Next step should be wait for https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/2848 then install packages from repo.
Updated by dgutu over 7 years ago
Modules are installalable. I have added a testsuite named gnome+allmodule_ProxySCC first for x86_64,ppc64,s390x and separate one for aarch64.
The last arch only modules are available.
Updated by dgutu over 7 years ago
- Status changed from Feedback to In Progress
- Assignee changed from okurz to dgutu
Updated by okurz over 7 years ago
- Target version changed from Milestone 7 to Milestone 8
please update with current status and plan
Updated by dgutu over 7 years ago
- Status changed from In Progress to Resolved
Since Build0374 all modules are enabled through proxy-SCC.
Sincer Build0398 I've added to be installed as a pattern, only this 2 patterns actually are available:
From ASMM - Machinery and Puppet. CFEngine has 2 entries with the same name so I've created a bug for that
FROM PCM - Google Cloud Platform and OpenStack
There are separate progress ticker which indicates what modules needs to be properly tested, not only installation of packages.