Project

General

Profile

Actions
Copy link

action #34732

closed

coordination #17436: [tools][functional][y][epic] tests must not have access to published repos, e.g. download.o.o on Tumbleweed -> remove all references to download.o.o from repos during installation and afterwards

[functiona][y][opensuse] make zypper_info use mirrored source repo, do not remove official update repos

Added by riafarov over 6 years ago. Updated almost 6 years ago.

Status:
Resolved
Priority:
High
Assignee:
riafarov
Category:
Bugs in existing tests
Target version:
SUSE QA (private) - Milestone 22
Start date:
2018-04-11
Due date:
2019-01-15
% Done:

0%

Estimated time:
8.00 h
Difficulty:

Description

Motivation

We have moved forward in https://progress.opensuse.org/issues/32038
Now we sync oss, oss-debug and non-oss repos where available. And use them in the tests.

We have identified 2 scenarios, where it doesn't work that well. One is zypper_info, which we now schedule in console tests instead of extra tests and run before replace the repos, so it uses official source repos.

Second issue if with update repos, as we remove them, it breaks scenarios which touch packagekit. As of now it's mainly yast control center, where we get an error about missing update repos. Other scenarios are excluded as have different flow (e.g. url is given in the installer, etc.).

We assume that simple filtering will work when syncing. So we need to reevaluate if that is not the case.

Suggestions

To solve zypper_info, we need a feature in rsync.pl to sync only defined packages from the repo to save space.
For update repos, we actually though not to remove them when cleaning official repos.

Acceptance criteria

  1. Mirrored repos are used when using packages from source oss repo
  2. zypper_info tets suite works with partly synced repo

Related issues 1 (0 open — 1 closed)

Blocks qe-yam - action #20206: [functional][y][yast][easy] Add yast2_ui_devel scenario (was: please remove yast2_cmdline from 42.3 test plan)ResolvedJERiveraMoya2017-07-02

Actions
Actions
Copy link
 #1

Updated by okurz over 6 years ago

  • Subject changed from [functiona][opensuse] make zypper_info use mirrored source repo, do not remove official update repos to [functiona][u][opensuse] make zypper_info use mirrored source repo, do not remove official update repos
  • Due date set to 2018-06-19
  • Category set to Bugs in existing tests
  • Target version set to Milestone 17
Actions
Copy link
 #2

Updated by riafarov over 6 years ago

  • Subject changed from [functiona][u][opensuse] make zypper_info use mirrored source repo, do not remove official update repos to [functiona][y][opensuse] make zypper_info use mirrored source repo, do not remove official update repos
Actions
Copy link
 #3

Updated by riafarov over 6 years ago

  • Due date deleted (2018-06-19)
  • Target version changed from Milestone 17 to Milestone 21+

Moving, as requires a lot of changes

Actions
Copy link
 #4

Updated by riafarov over 6 years ago

  • Due date set to 2018-12-18

Setting due date, so parent epic stops sending me reminders

Actions
Copy link
 #5

Updated by okurz over 6 years ago

  • Target version changed from Milestone 21+ to Milestone 21+
Actions
Copy link
 #6

Updated by mloviska over 6 years ago

  • Is duplicate of action #20206: [functional][y][yast][easy] Add yast2_ui_devel scenario (was: please remove yast2_cmdline from 42.3 test plan) added
Actions
Copy link
 #7

Updated by mloviska over 6 years ago

  • Is duplicate of deleted (action #20206: [functional][y][yast][easy] Add yast2_ui_devel scenario (was: please remove yast2_cmdline from 42.3 test plan))
Actions
Copy link
 #8

Updated by mloviska over 6 years ago

  • Blocks action #20206: [functional][y][yast][easy] Add yast2_ui_devel scenario (was: please remove yast2_cmdline from 42.3 test plan) added
Actions
Copy link
 #9

Updated by riafarov over 6 years ago

rsync.pl feature to sync required parts of the repo is relevant not only for source, but also for debug-info repos.

Actions
Copy link
 #10

Updated by okurz about 6 years ago

  • Priority changed from Normal to High
  • Target version changed from Milestone 21+ to Milestone 21

I know understood that the OSS repo is about 70GB whereas the debuginfo repo we sync has 75GB (!) which is mostly unused. This can be easily seen on http://openqa.opensuse.org/admin/assets and also explains why we have troubles with the premature deletion of repos e.g. lately on openSUSE Tumbleweed ppc64le and aarch64. So making sure that we only sync selectively the packages we need for testing would help in this regard.

Actions
Copy link
 #11

Updated by riafarov about 6 years ago

  • Status changed from New to Workable
Actions
Copy link
 #12

Updated by riafarov about 6 years ago

  • Description updated (diff)
  • Estimated time set to 8.00 h
Actions
Copy link
 #13

Updated by JERiveraMoya about 6 years ago

  • Assignee set to JERiveraMoya
Actions
Copy link
 #14

Updated by JERiveraMoya about 6 years ago

  • Status changed from Workable to In Progress

Result from POC/prototype suggested by @riafarov as starting point to evaluate filtering of packages:
Using a small repo, i.e.: ftp://openqa.suse.de/SLE-12-SP4-RT-POOL-x86_64-Build0074-Media1/ downloaded in my local machine, all packages in x86_64/ and noarch/ were removed except one for each folder.

  • zypper ar of that repo via url creating a simple python web server -> successful
  • zypper in of both packages kept on previous mentioned folder -> successful (I didn't perform the installation as I have some unrelated difficulties with networking in VM when using openVPN) but if zypper is displaying well what is going to be installed in my system (TW) despite the deletion of those many packages I guess it should suffice to move forward.
Actions
Copy link
 #15

Updated by JERiveraMoya about 6 years ago

I solved my issue with VM when using openVPN via feature of Virtual Manager to "Add Connection" and installing virtualization tools in a remote machine with ssh access.
I re-tested performing the installation and I didn't see any difference, it is possible to add the repo and install the packages and as far the dependencies of the package to be installed can be retrieved from this repo or other repos we are fine.

Actions
Copy link
 #17

Updated by riafarov about 6 years ago

  • Due date changed from 2018-12-18 to 2019-01-15
  • Status changed from In Progress to Workable
  • Assignee deleted (JERiveraMoya)
  • Target version changed from Milestone 21 to Milestone 22

There is small part remaining, so let's try to resolve during the week. If not - next year =)

Actions
Copy link
 #18

Updated by JERiveraMoya almost 6 years ago

I was learning how to debug the script and at least I feel more comfortable now although it still a long path for me to understand what is doing.
I noticed that before running _reposync, function update_current_repo is executed copying all the files without filter. At the end there is a .new folder under repos/ in correct sync but other folders exists with the whole content. It looks like this:

SLE-15-SP1-Installer-DVD-x86_64-Build128.1-Media1
SLE-15-SP1-Module-Basesystem-POOL-x86_64-Build128.1-Media1
SLE-15-SP1-Module-Basesystem-POOL-x86_64-Build128.1-Media1.new
SLE-15-SP1-Module-Basesystem-POOL-x86_64-CURRENT-Media1 -> SLE-15-SP1-Module-Basesystem-POOL-x86_64-Build128.1-Media1
SLE-15-SP1-Module-Basesystem-POOL-x86_64-Media1-CURRENT

Let's figure out what is the purpose of those functions and discuss how we can continue when you're back @riafarov. Thanks.

Actions
Copy link
 #19

Updated by riafarov almost 6 years ago

  • Assignee set to riafarov
Actions
Copy link
 #20

Updated by riafarov almost 6 years ago

  • Status changed from Workable to Feedback
Actions
Copy link
 #21

Updated by riafarov almost 6 years ago

  • Status changed from Feedback to Resolved
Actions
Copy link

Also available in: Atom PDF