Project

General

Profile

action #12370

toolchain/crash - test uses incorrect

Added by RBrownSUSE over 5 years ago. Updated almost 5 years ago.

Status:
Resolved
Priority:
Urgent
Assignee:
michalnowak
Category:
Bugs in existing tests
Target version:
-
Start date:
2016-06-16
Due date:
% Done:

100%

Estimated time:
Difficulty:

Description

toolchain/crash currently uses the following repo (as an example) for debuginfo packages:
http://download.suse.de/ibs/SUSE/Products/SLE-SERVER/$version/$arch/product_debug/

This is wrong, as that repo has the debuginfo for the last released milestone, but openQA needs to test the next build we might want to release.

As the debuginfo must match the version of the packages in the main repo, this is leading to the test to fail

Proposed solution:

Test should be using something like

http://download.suse.de/ibs/SUSE:/SLE-12-SP2:/GA/images/repo/SLE-$version-Server-POOL-$arch-Build$BUILD_SLE-Media3/

However, we cannot rely on it being pulled from download.suse.de

Therefore rsync.pl should be modified to sync the -Server-POOL--Media3 repo (Debug POOL), just like we already do for the -Server-POOL--Media1 (Product POOL) repo

Then the test should be using that openQA hosted repo, not the one from download.suse.de which cannot be trusted to be static and present when needed.

Michal's already on the case, this poo# exists purely for tracking purposes in the openQA dashboard


Related issues

Related to openQA Infrastructure - action #12506: openqa-scripts on osd/o3 should be deployed automatically (CD)Resolved2016-06-26

History

#1 Updated by michalnowak over 5 years ago

Sync debuginfo RPMs for SLE-12 Server (https://gitlab.suse.de/openqa/scripts/merge_requests/34)
[WIP] Use provided repo with debuginfo RPMs (https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/1479)

Before we have a proper and verified fix I suggest to disable the crash.pm test.

#2 Updated by michalnowak over 5 years ago

  • Status changed from New to Feedback

Links above host a fix pending review.

#3 Updated by okurz over 5 years ago

The PRs are merged and should be live in current tests, but they seem to still fail, e.g. in https://openqa.suse.de/tests/455375#step/crash/22 Looks like something is missing in the repo path, please check.

#4 Updated by michalnowak over 5 years ago

https://openqa.suse.de/tests/455375#settings: Variable REPO_SLES_DEBUG is not present in the job. Also I don't see any SLE-12-SP2-Server-POOL-x86_64-Build####-Media3 on ftp.openqa.de nor in openqa.suse.de assets. Where can I see log from rsync.pl run, and how it was executed?

#5 Updated by okurz over 5 years ago

You can't see the logs from the rsync.pl runs yourself easily. You can ask to add your ssh key in https://gitlab.suse.de/openqa/salt-pillars-openqa but currently the gitlab server does not even show up content within repos. You have to ask admins of osd to help.

#6 Updated by okurz over 5 years ago

I checked on osd: /opt/openqa-scripts hasn't been updated so it can't work. So the main problem I see know is missing CD for scripts. I am trying to get in touch with coolo/sysrich.

#7 Updated by okurz over 5 years ago

  • Related to action #12506: openqa-scripts on osd/o3 should be deployed automatically (CD) added

#8 Updated by okurz over 5 years ago

I updated the scripts repo now manually and created a related issue to improve this in future.

#9 Updated by michalnowak over 5 years ago

Thanks Oliver. Looking at toolchain_zypper test from build 1640 at openqa.s.d the change was not in effect, will wait for next build.

#10 Updated by michalnowak over 5 years ago

The changes are live: https://openqa.suse.de/tests/457581#step/crash/29. However installing kernel-default-base-debuginfo and kernel-default-debuginfo fails because such packages seems not to be defined in the metadata of the respective repo:

# cat /etc/zypp/repos.d/XXX.repo 
[XXX]
name=xxx
enabled=1
autorefresh=1
baseurl=ftp://openqa.suse.de/SLE-12-SP2-Server-POOL-x86_64-Build1641-Media3/
type=rpm-md
gpgcheck=0
# zypper se -s --repo XXX kernel-default
Loading repository data...
Reading installed packages...

S | Name                 | Type    | Version     | Arch   | Repository
--+----------------------+---------+-------------+--------+-----------
v | kernel-default       | package | 4.4.13-47.4 | x86_64 | xxx       
  | kernel-default-base  | package | 4.4.13-47.4 | x86_64 | xxx       
  | kernel-default-devel | package | 4.4.13-47.4 | x86_64 | xxx       

If I install package from that repo:

# zypper -v in --repo XXX kernelshark
Verbosity: 1
Non-option program arguments: 'kernelshark' 
Initializing Target
Checking whether to refresh metadata for xxx
Loading repository data...
Reading installed packages...
Force resolution: No
Selecting 'kernelshark-2.0.4-3.95.x86_64' from repository 'xxx' for installation.
Resolving package dependencies...
Force resolution: No

The following NEW package is going to be installed:
  kernelshark  2.0.4-3.95


1 new package to install.
Overall download size: 1.4 MiB. Already cached: 0 B. After the operation, additional 2.4 MiB will be used.
Continue? [y/n/? shows all options] (y): 
committing
Retrieving package kernelshark-2.0.4-3.95.x86_64                                                                                                          (1/1),   1.4 MiB (  2.4 MiB unpacked)
Retrieving: kernelshark-2.0.4-3.95.x86_64.rpm ...........................................................................................................................................[done]
Checking for file conflicts: ............................................................................................................................................................[done]
(1/1) Installing: kernelshark-2.0.4-3.95.x86_64 .........................................................................................................................................[done]
CommitResult  (total 1, done 1, error 0, skipped 0, updateMessages 0)

I get the ordinary binary, i.e. non-debuginfo package:

# rpmquery -l kernelshark-2.0.4-3.95.x86_64
/usr/bin/kernelshark
/usr/bin/trace-graph
/usr/bin/trace-view
...

But I don't see there (ftp://openqa.suse.de/SLE-12-SP2-Server-POOL-x86_64-Build1641-Media3/x86_64/) non-debuginfo packages.

#11 Updated by okurz about 5 years ago

  • Status changed from Feedback to In Progress
  • Priority changed from High to Urgent

I am sorry I can't help with understanding which packages are needed or how they can be provided.

Increasing prio as this happens often enough on osd as well as o3 to be regarded as urgent. Please fix.

Recent example: https://openqa.suse.de/tests/464769

#12 Updated by michalnowak about 5 years ago

This PR https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/1518 should fix/workaround it.

Also talked to bugfinder:

<bugfinder> I guess this is the old zypper misfeature ...
<bugfinder> if the path ends in "3" it tries to get metadata by replacing the last digit at the end by a "1" ...
<bugfinder> that's why we create a "DEBUG" symlink on dist
<bugfinder> like http://dist.nue.suse.com/install/SLP/SLE-12-SP1-Server-LATEST/x86_64/DEBUG/
<bugfinder> or with directory names like "product_debug" as we do this for the official pool channels:
<bugfinder> http://dist.nue.suse.com/ibs/SUSE/Products/SLE-SERVER/12-SP2/x86_64/product_debug/

#13 Updated by okurz about 5 years ago

thanks for sharing this. So the "workaround" might actually be because of yet another case where openQA might need to do some symlinking as well as is done when stuff is published from ibs to dist.

#14 Updated by michalnowak about 5 years ago

Next week I am on vacation PR#1518 should fix the problem in the mean time. Once I am back I'll look into proper solution via renaming the repo.

#15 Updated by michalnowak about 5 years ago

  • Status changed from In Progress to Resolved
  • % Done changed from 0 to 100

The test now uses correct repository thanks to change in libzypp: https://openqa.suse.de/tests/overview?distri=sle&version=12-SP2&build=2002&groupid=25 (see the 'toolchain*' test suites). I believe this issue is fixed. Closing.

#16 Updated by mkravec almost 5 years ago

  • Status changed from Resolved to In Progress

Test died:
Could not retrieve required variable REPO_SLES_DEBUG at /usr/lib/os-autoinst/testapi.pm line 452.

https://openqa.suse.de/tests/598582#step/crash/19
https://openqa.suse.de/tests/599179#step/crash/14

#17 Updated by michalnowak almost 5 years ago

  • Status changed from In Progress to Resolved

Also available in: Atom PDF