coordination #15108: [sle][functional][u][epic] Modules (Installation + migration)
[sles][functional][modules] Modules installation
Make sure that each module installs correctly.
- AC1: One module is tested on every SLES build using proxy-SCC
- AC2: Multiple modules are tested
SCC_ADDONS=<module_name1>[,<module_name2>]accordingly (SCC regurl should only register SLES, not point to any module)
- after the module(s) are registered and enabled packages should be installed from the modules, e.g. by selecting the appropriate pattern(s)
As SLE service packs and therefore also builds within service packs are meant to be ABI compatible modules can be simply installed over SCC while registering the latest SLES build. For all modules that are already on SCC we just install them over SCC (That's it. No need to sync anything anywhere).
Enabling modules does not install anything from it automatically. patterns or packages need to be selected, e.g. after enabling "hpcm" then "zypper search hpc" should return something. Also the repository can be explicitly selected with "zypper lr" and then all packages and patterns from that repo can be listed with "zypper search -r ".
So far module installation was never done besides textmode during online migration but these are actually "post-validation" tests as described in #15800.
#11 Updated by maritawerner over 4 years ago
Andreas Stieger, the security PJM, confirmed to me that atm there is no certifications module for SP2 and for SP3. They will develop a new module but he expects that to be available in July. So we can skip all installation/Migration scenarios with the certifications module.
#24 Updated by okurz over 4 years ago
What I did as an experiment:
- on osd:
rsync -aHP rsync://dist.suse.de/ibs/SUSE/Products/SLE-Module-Adv-Systems-Management/12/x86_64/product/ /var/lib/openqa/share/factory/repo/SLE-12-Module-Adv-Systems-Management-POOL-x86_64-Build20170412-Media1/
- openqa_clone_job_osd 871704 _GROUP=0 SCC_ADDONS=asmm SCC_URL=http://Server-0326.Module-Adv-Systems-Management-20170412.proxy.scc.suse.de -> https://openqa.suse.de/tests/874996#live, also see https://openqa.suse.de/tests/874996/file/video.ogv
- installed with module enabled, then manually installed all packages from that module
so that works.
#25 Updated by dgutu over 4 years ago
Thank you for the try but I found some caveats.
- When module is selected by using registration you need to get this module as a product on the installation settings page and respective patterns.
- When I tried to use regurl and include the once you have used I got the module in the product list but the installer gives a warning that the packages can't be installed. Continue to work on that.
#26 Updated by dgutu over 4 years ago
I found that there are 2 types sources for modules.
- correct structure of the folder and files so we can sync this to ftp:/openqa.suse.de
- It contains only 4 modules from 6 (asmm, lgm, pcm and wsm - present), (tcm and contm - missing)
- very limited number of packages and old version
- Latest and updated packages for all modules
- Missing license files for lgm and wsm modules
- Folder and file organized like repo and not like ISO
I'm confused now...
#27 Updated by maritawerner over 4 years ago
Answer from Stefan Behlert:
The first link is the pool channel added to the ibs for GA. At that time only 4 modules existed. The newer modules must have other channels that were added for SP1/SP2.
The second channel is the update channel for all modules were you can find all updates for all modules. Licences are not in there because they are in the pool channel.
If you want to know more here maybe ask a colleague from QAM. Mantienance are the experts around channels, etc.
#28 Updated by okurz over 4 years ago
- Description updated (diff)
- Status changed from New to In Progress
I guess we got confused in before and the approach is way easier than thought. Updated description accordingly.
$ openqa_clone_job_osd 916988 _GROUP=0 SCC_ADDONS=asmm TEST=gnome+sdk+proxy_SCC+asmm_okurz
Just in case if we would need to test modules which are not yet released over SCC or when we would need to test a specific version of modules then we would need to sync them to osd and "fake" the repositories. This has been mentioned in #12908#note-4 and it seems coolo confused me or meant something different.
#29 Updated by okurz over 4 years ago
aha, and https://openqa.suse.de/tests/919019#step/consoletest_setup/32 failed now so … proxy-SCC issue?
#31 Updated by okurz over 4 years ago
In case this is not feasible to do within proxy-SCC then we need to follow the approach of syncing everything to osd for testing but I am wondering about the "pool/product" and "update" part. https://openqa.suse.de/tests/919019#step/consoletest_setup/32 shows that when specifying only SLES in the regurl both "…-Pool" and "…-Updates" now point to the same directory on osd.
For the repos there are seemingly multiple versions:
- e.g. http://dist.suse.de/ibs/SUSE:/SLE-12:/GA/images/repo/ - looks like the repo of the module at the point in time when SLE 12 GA was released
- http://dist.suse.de/ibs/SUSE/Products/SLE-Module-Adv-Systems-Management/12/x86_64/product/ - probably the "last development release" of the module before SLE 12 GA (it's one build below the upper one)
- http://dist.suse.de/ibs/SUSE/Updates/SLE-Module-Adv-Systems-Management/12/x86_64/update/ - updates including all recent packages of the module
I want to crosscheck how it looks like when we specify the module in the regurl as well but osd is currently down.
#32 Updated by okurz over 4 years ago
$ openqa_clone_job_osd 916988 _GROUP=0 SCC_ADDONS=asmm TEST=gnome+sdk+proxy_SCC+asmm_okurz SCC_URL=http://Server-0367.Module-Adv-Systems-Management-20170506.proxy.scc.suse.de Created job #920359: sle-12-SP3-Server-DVD-x86_64-Build0367-gnome+sdk+proxy_SCC_without_addon@64bit
From the following screenshot
we can see how the repositories look like after rewriting from proxy-SCC when also the module is specified in SCC_URL: http://openqa.suse.de/assets/repo/SLE-12-Module-Adv-Systems-Management-POOL-x86_64-Build20170506-Media1/ but this is the same URL for both the Pool and Updates repository. I see this as another problem because when both repository URL yield the same path we can not easily sync both Pool and Updates but need to merge them. So even more messy.
As an experiment nevertheless I synced both Pool and Updates into the target path on osd:
for path in SUSE:/SLE-12:/GA/images/repo/SLE-12-Module-Adv-Systems-Management-POOL-x86_64-Build0096-Media1/ SUSE/Updates/SLE-Module-Adv-Systems-Management/12/x86_64/update/ ; do rsync -aHP rsync://dist.suse.de/ibs/$path /var/lib/openqa/share/factory/repo/SLE-12-Module-Adv-Systems-Management-POOL-x86_64-Build20170506-Media1/ ; done
yielding ftp://openqa.suse.de/SLE-12-Module-Adv-Systems-Management-POOL-x86_64-Build20170506-Media1/ , a messy merged directory. If this is the approach to go then for modules which where not existant at the time of GA we might need to find a corresponding pool directory first from a later SP or a products directory elsewhere. Retriggered above's openQA job
to see if the repository can be at least refreshed properly.
That failed with
$ curl -s https://openqa.suse.de/tests/920360/file/start_install-y2logs.tar.bz2 | tar -xjf - YaST2/badlist -O 2 packages failed nothing provides product_flavor(sle-module-adv-systems-management) needed by product:sle-module-adv-systems-management-12-0.x86_64 nothing provides product_flavor(sle-module-adv-systems-management) needed by product:sle-module-adv-systems-management-12-0.x86_64
so apparently the pool directory is incomplete and does not provide the product properly anymore. Do we really need to reverse engineer the update channels management infrastructure here?
#34 Updated by okurz over 4 years ago
- Status changed from In Progress to Feedback
- Assignee changed from dgutu to okurz
#35 Updated by okurz over 4 years ago
The PR to proxy-SCC was merged. Let's see if it already has an effect -> https://openqa.suse.de/tests/925867#live
https://openqa.suse.de/tests/925867#step/zypper_lr/6 shows that the module repositories are not rewritten and the ones from SCC are used which is what we wanted here.
Next step should be wait for https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/2848 then install packages from repo.
#39 Updated by dgutu over 4 years ago
- Status changed from In Progress to Resolved
Since Build0374 all modules are enabled through proxy-SCC.
Sincer Build0398 I've added to be installed as a pattern, only this 2 patterns actually are available:
From ASMM - Machinery and Puppet. CFEngine has 2 entries with the same name so I've created a bug for that
FROM PCM - Google Cloud Platform and OpenStack
There are separate progress ticker which indicates what modules needs to be properly tested, not only installation of packages.