Project

General

Profile

Actions
Copy link

action #17042

closed

coordination #15108: [sle][functional][u][epic] Modules (Installation + migration)

[sles][functional][modules] Modules installation

Added by maritawerner almost 8 years ago. Updated over 7 years ago.

Status:
Resolved
Priority:
High
Assignee:
dgutu
Category:
New test
Target version:
openQA Project (public) - Milestone 8
Start date:
2017-02-14
Due date:
% Done:

0%

Estimated time:
Difficulty:

Description

goal

Make sure that each module installs correctly.

acceptance criteria

  • AC1: One module is tested on every SLES build using proxy-SCC
  • AC2: Multiple modules are tested

tasks

  • set SCC_ADDONS=<module_name1>[,<module_name2>] accordingly (SCC regurl should only register SLES, not point to any module)
  • after the module(s) are registered and enabled packages should be installed from the modules, e.g. by selecting the appropriate pattern(s)

further details

As SLE service packs and therefore also builds within service packs are meant to be ABI compatible modules can be simply installed over SCC while registering the latest SLES build. For all modules that are already on SCC we just install them over SCC (That's it. No need to sync anything anywhere).
Enabling modules does not install anything from it automatically. patterns or packages need to be selected, e.g. after enabling "hpcm" then "zypper search hpc" should return something. Also the repository can be explicitly selected with "zypper lr" and then all packages and patterns from that repo can be listed with "zypper search -r ".
So far module installation was never done besides textmode during online migration but these are actually "post-validation" tests as described in #15800.

Files

Download all files
canvas.png (40.2 KB) canvas.png screenshot of openQA test run showing how the repository URLs look like after rewriting okurz, 2017-05-06 07:33
openQA_rewritten_sle_module_paths.png (15.3 KB) openQA_rewritten_sle_module_paths.png okurz, 2017-05-06 09:08
Actions
Copy link
 #1

Updated by maritawerner almost 8 years ago

  • Related to coordination #15108: [sle][functional][u][epic] Modules (Installation + migration) added
Actions
Copy link
 #2

Updated by maritawerner almost 8 years ago

okurz: I think that that should be covered by the sle functional team? Do you agree? Any feedback is welcome.

Actions
Copy link
 #3

Updated by okurz almost 8 years ago

  • Related to deleted (coordination #15108: [sle][functional][u][epic] Modules (Installation + migration))
Actions
Copy link
 #4

Updated by okurz almost 8 years ago

  • Parent task set to #15108
Actions
Copy link
 #5

Updated by okurz almost 8 years ago

  • Subject changed from [sles][functional] Modules Installation to [sles][functional] Modules Installation
  • Category set to New test
Actions
Copy link
 #6

Updated by maritawerner almost 8 years ago

Modules for SP3 are not enabled in SCC yet. I wait for Anja/Frederic to get back to us.

Actions
Copy link
 #7

Updated by dgutu almost 8 years ago

  • Assignee set to dgutu
Actions
Copy link
 #8

Updated by dgutu almost 8 years ago

Modules are available while registering the system on SCC-Proxy.
Only Certifications module is missing. Will give it a try on normal SCC.

Actions
Copy link
 #9

Updated by dgutu almost 8 years ago

  1. Certification modules is not available for our scc key.
  2. Legacy Module is failing to download license, emailed Arteom about this.
Actions
Copy link
 #10

Updated by dgutu almost 8 years ago

Legacy and Web Scripting needs license to be synced, rsync should be adapted.
Other modules like, Public Cloud, ToolChain, Container, Advances System Management Modules are fine.
@Marita, what about Certification Module, is it on PRD as available one?

Actions
Copy link
 #11

Updated by maritawerner almost 8 years ago

Andreas Stieger, the security PJM, confirmed to me that atm there is no certifications module for SP2 and for SP3. They will develop a new module but he expects that to be available in July. So we can skip all installation/Migration scenarios with the certifications module.

Actions
Copy link
 #12

Updated by dgutu almost 8 years ago

Thx for the info Marita.
As per today this one is running in test development group with 4 modules from 6.
https://openqa.suse.de/tests/808525
I'm working on rsync.pl to enable the remaining 2 modules to this testmodule.

Actions
Copy link
 #13

Updated by dgutu almost 8 years ago

The rsync.pl syncs everything related to the latest SP and staging but modules are related to GA,
because of that I'm still working on it.

Actions
Copy link
 #14

Updated by maritawerner over 7 years ago

Dumitru, could you please add more details into the ticket description or split the ticket. Please check PRD section 1.4.2.1.

Actions
Copy link
 #15

Updated by asmorodskyi over 7 years ago

  • Related to action #9464: [sle][functional][u] Modules - Advanced Systems Management added
Actions
Copy link
 #16

Updated by asmorodskyi over 7 years ago

  • Related to coordination #9466: [sle][functional][epic][medium][u]Modules - Containers: Test the SLE module for containers added
Actions
Copy link
 #17

Updated by asmorodskyi over 7 years ago

  • Related to coordination #9470: [sle][functional][u][sle12][medium][epic]Modules - Web and Scripting added
Actions
Copy link
 #18

Updated by asmorodskyi over 7 years ago

  • Related to deleted (coordination #9470: [sle][functional][u][sle12][medium][epic]Modules - Web and Scripting)
Actions
Copy link
 #19

Updated by asmorodskyi over 7 years ago

  • Related to deleted (coordination #9466: [sle][functional][epic][medium][u]Modules - Containers: Test the SLE module for containers)
Actions
Copy link
 #20

Updated by asmorodskyi over 7 years ago

  • Related to deleted (action #9464: [sle][functional][u] Modules - Advanced Systems Management)
Actions
Copy link
 #21

Updated by okurz over 7 years ago

  • Target version set to Milestone 7
Actions
Copy link
 #22

Updated by dgutu over 7 years ago

  • Subject changed from [sles][functional] Modules Installation to [sles][functional][modules] Modules installation
Actions
Copy link
 #23

Updated by okurz over 7 years ago

  • Description updated (diff)
Actions
Copy link
 #24

Updated by okurz over 7 years ago

What I did as an experiment:

so that works.

Actions
Copy link
 #25

Updated by dgutu over 7 years ago

Thank you for the try but I found some caveats.

  1. When module is selected by using registration you need to get this module as a product on the installation settings page and respective patterns.
  2. When I tried to use regurl and include the once you have used I got the module in the product list but the installer gives a warning that the packages can't be installed. Continue to work on that.
Actions
Copy link
 #26

Updated by dgutu over 7 years ago

I found that there are 2 types sources for modules.

http://dist.suse.de/ibs/SUSE:/SLE-12:/GA/images/repo/SLE-12-Module*
Offering:

  • correct structure of the folder and files so we can sync this to ftp:/openqa.suse.de

Doesn't offer:

  • It contains only 4 modules from 6 (asmm, lgm, pcm and wsm - present), (tcm and contm - missing)
  • very limited number of packages and old version

http://dist.suse.de/ibs/SUSE/Updates/
Offering:

  • Latest and updated packages for all modules

Doesn't offer:

  • Missing license files for lgm and wsm modules
  • Folder and file organized like repo and not like ISO

I'm confused now...

Actions
Copy link
 #27

Updated by maritawerner over 7 years ago

Answer from Stefan Behlert:

The first link is the pool channel added to the ibs for GA. At that time only 4 modules existed. The newer modules must have other channels that were added for SP1/SP2.
The second channel is the update channel for all modules were you can find all updates for all modules. Licences are not in there because they are in the pool channel.

If you want to know more here maybe ask a colleague from QAM. Mantienance are the experts around channels, etc.

Actions
Copy link
 #28

Updated by okurz over 7 years ago

  • Description updated (diff)
  • Status changed from New to In Progress

I guess we got confused in before and the approach is way easier than thought. Updated description accordingly.

$ openqa_clone_job_osd 916988 _GROUP=0 SCC_ADDONS=asmm TEST=gnome+sdk+proxy_SCC+asmm_okurz

-> sle-12-SP3-Server-DVD-x86_64-Build0367-gnome+sdk+proxy_SCC+asmm_okurz

Just in case if we would need to test modules which are not yet released over SCC or when we would need to test a specific version of modules then we would need to sync them to osd and "fake" the repositories. This has been mentioned in #12908#note-4 and it seems coolo confused me or meant something different.

Actions
Copy link
 #29

Updated by okurz over 7 years ago

aha, and https://openqa.suse.de/tests/919019#step/consoletest_setup/32 failed now so … proxy-SCC issue?

Actions
Copy link
 #30

Updated by okurz over 7 years ago

https://github.com/SUSE/scc-qa-proxy/issues/14 is the feature request. I asked the SCC team if someone can take a look.

Actions
Copy link
 #31

Updated by okurz over 7 years ago

In case this is not feasible to do within proxy-SCC then we need to follow the approach of syncing everything to osd for testing but I am wondering about the "pool/product" and "update" part. https://openqa.suse.de/tests/919019#step/consoletest_setup/32 shows that when specifying only SLES in the regurl both "…-Pool" and "…-Updates" now point to the same directory on osd.

For the repos there are seemingly multiple versions:

I want to crosscheck how it looks like when we specify the module in the regurl as well but osd is currently down.

Actions
Copy link
 #32

Updated by okurz over 7 years ago 

$ openqa_clone_job_osd 916988 _GROUP=0 SCC_ADDONS=asmm TEST=gnome+sdk+proxy_SCC+asmm_okurz SCC_URL=http://Server-0367.Module-Adv-Systems-Management-20170506.proxy.scc.suse.de
Created job #920359: sle-12-SP3-Server-DVD-x86_64-Build0367-gnome+sdk+proxy_SCC_without_addon@64bit

-> https://openqa.suse.de/t920359

From the following screenshot
the screenshot
we can see how the repositories look like after rewriting from proxy-SCC when also the module is specified in SCC_URL: http://openqa.suse.de/assets/repo/SLE-12-Module-Adv-Systems-Management-POOL-x86_64-Build20170506-Media1/ but this is the same URL for both the Pool and Updates repository. I see this as another problem because when both repository URL yield the same path we can not easily sync both Pool and Updates but need to merge them. So even more messy.

As an experiment nevertheless I synced both Pool and Updates into the target path on osd:

for path in SUSE:/SLE-12:/GA/images/repo/SLE-12-Module-Adv-Systems-Management-POOL-x86_64-Build0096-Media1/ SUSE/Updates/SLE-Module-Adv-Systems-Management/12/x86_64/update/ ; do rsync -aHP rsync://dist.suse.de/ibs/$path /var/lib/openqa/share/factory/repo/SLE-12-Module-Adv-Systems-Management-POOL-x86_64-Build20170506-Media1/ ; done

yielding ftp://openqa.suse.de/SLE-12-Module-Adv-Systems-Management-POOL-x86_64-Build20170506-Media1/ , a messy merged directory. If this is the approach to go then for modules which where not existant at the time of GA we might need to find a corresponding pool directory first from a later SP or a products directory elsewhere. Retriggered above's openQA job

-> https://openqa.suse.de/tests/920360#live

to see if the repository can be at least refreshed properly.

That failed with

$ curl -s https://openqa.suse.de/tests/920360/file/start_install-y2logs.tar.bz2 | tar -xjf - YaST2/badlist -O
2 packages failed
nothing provides product_flavor(sle-module-adv-systems-management) needed by product:sle-module-adv-systems-management-12-0.x86_64
nothing provides product_flavor(sle-module-adv-systems-management) needed by product:sle-module-adv-systems-management-12-0.x86_64

so apparently the pool directory is incomplete and does not provide the product properly anymore. Do we really need to reverse engineer the update channels management infrastructure here?

Actions
Copy link
 #34

Updated by okurz over 7 years ago

  • Status changed from In Progress to Feedback
  • Assignee changed from dgutu to okurz

There is a pull request open for scc-qa-proxy to only rewrite specified products. I think that is the right approach and we should wait for the PR to be be active on proxy-SCC before continuing.

Actions
Copy link
 #35

Updated by okurz over 7 years ago

The PR to proxy-SCC was merged. Let's see if it already has an effect -> https://openqa.suse.de/tests/925867#live

https://openqa.suse.de/tests/925867#step/zypper_lr/6 shows that the module repositories are not rewritten and the ones from SCC are used which is what we wanted here.

Next step should be wait for https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/2848 then install packages from repo.

Actions
Copy link
 #36

Updated by dgutu over 7 years ago

Modules are installalable. I have added a testsuite named gnome+allmodule_ProxySCC first for x86_64,ppc64,s390x and separate one for aarch64.
The last arch only modules are available.

Actions
Copy link
 #37

Updated by dgutu over 7 years ago

  • Status changed from Feedback to In Progress
  • Assignee changed from okurz to dgutu
Actions
Copy link
 #38

Updated by okurz over 7 years ago

  • Target version changed from Milestone 7 to Milestone 8

please update with current status and plan

Actions
Copy link
 #39

Updated by dgutu over 7 years ago

  • Status changed from In Progress to Resolved

Since Build0374 all modules are enabled through proxy-SCC.
Sincer Build0398 I've added to be installed as a pattern, only this 2 patterns actually are available:
From ASMM - Machinery and Puppet. CFEngine has 2 entries with the same name so I've created a bug for that
FROM PCM - Google Cloud Platform and OpenStack

There are separate progress ticker which indicates what modules needs to be properly tested, not only installation of packages.

Actions
Copy link

Also available in: Atom PDF