action #34732
closedcoordination #17436: [tools][functional][y][epic] tests must not have access to published repos, e.g. download.o.o on Tumbleweed -> remove all references to download.o.o from repos during installation and afterwards
[functiona][y][opensuse] make zypper_info use mirrored source repo, do not remove official update repos
0%
Description
Motivation¶
We have moved forward in https://progress.opensuse.org/issues/32038
Now we sync oss, oss-debug and non-oss repos where available. And use them in the tests.
We have identified 2 scenarios, where it doesn't work that well. One is zypper_info, which we now schedule in console tests instead of extra tests and run before replace the repos, so it uses official source repos.
Second issue if with update repos, as we remove them, it breaks scenarios which touch packagekit. As of now it's mainly yast control center, where we get an error about missing update repos. Other scenarios are excluded as have different flow (e.g. url is given in the installer, etc.).
We assume that simple filtering will work when syncing. So we need to reevaluate if that is not the case.
Suggestions¶
To solve zypper_info, we need a feature in rsync.pl to sync only defined packages from the repo to save space.
For update repos, we actually though not to remove them when cleaning official repos.
Acceptance criteria¶
- Mirrored repos are used when using packages from source oss repo
- zypper_info tets suite works with partly synced repo
Updated by okurz about 6 years ago
- Subject changed from [functiona][opensuse] make zypper_info use mirrored source repo, do not remove official update repos to [functiona][u][opensuse] make zypper_info use mirrored source repo, do not remove official update repos
- Due date set to 2018-06-19
- Category set to Bugs in existing tests
- Target version set to Milestone 17
Updated by riafarov almost 6 years ago
- Subject changed from [functiona][u][opensuse] make zypper_info use mirrored source repo, do not remove official update repos to [functiona][y][opensuse] make zypper_info use mirrored source repo, do not remove official update repos
Updated by riafarov almost 6 years ago
- Due date deleted (
2018-06-19) - Target version changed from Milestone 17 to Milestone 21+
Moving, as requires a lot of changes
Updated by riafarov almost 6 years ago
- Due date set to 2018-12-18
Setting due date, so parent epic stops sending me reminders
Updated by okurz almost 6 years ago
- Target version changed from Milestone 21+ to Milestone 21+
Updated by mloviska almost 6 years ago
- Is duplicate of action #20206: [functional][y][yast][easy] Add yast2_ui_devel scenario (was: please remove yast2_cmdline from 42.3 test plan) added
Updated by mloviska almost 6 years ago
- Is duplicate of deleted (action #20206: [functional][y][yast][easy] Add yast2_ui_devel scenario (was: please remove yast2_cmdline from 42.3 test plan))
Updated by mloviska almost 6 years ago
- Blocks action #20206: [functional][y][yast][easy] Add yast2_ui_devel scenario (was: please remove yast2_cmdline from 42.3 test plan) added
Updated by riafarov over 5 years ago
rsync.pl feature to sync required parts of the repo is relevant not only for source, but also for debug-info repos.
Updated by okurz over 5 years ago
- Priority changed from Normal to High
- Target version changed from Milestone 21+ to Milestone 21
I know understood that the OSS repo is about 70GB whereas the debuginfo repo we sync has 75GB (!) which is mostly unused. This can be easily seen on http://openqa.opensuse.org/admin/assets and also explains why we have troubles with the premature deletion of repos e.g. lately on openSUSE Tumbleweed ppc64le and aarch64. So making sure that we only sync selectively the packages we need for testing would help in this regard.
Updated by riafarov over 5 years ago
- Description updated (diff)
- Estimated time set to 8.00 h
Updated by JERiveraMoya over 5 years ago
- Status changed from Workable to In Progress
Result from POC/prototype suggested by @riafarov as starting point to evaluate filtering of packages:
Using a small repo, i.e.: ftp://openqa.suse.de/SLE-12-SP4-RT-POOL-x86_64-Build0074-Media1/ downloaded in my local machine, all packages in x86_64/ and noarch/ were removed except one for each folder.
zypper arof that repo via url creating a simple python web server -> successfulzypper inof both packages kept on previous mentioned folder -> successful (I didn't perform the installation as I have some unrelated difficulties with networking in VM when using openVPN) but if zypper is displaying well what is going to be installed in my system (TW) despite the deletion of those many packages I guess it should suffice to move forward.
Updated by JERiveraMoya over 5 years ago
I solved my issue with VM when using openVPN via feature of Virtual Manager to "Add Connection" and installing virtualization tools in a remote machine with ssh access.
I re-tested performing the installation and I didn't see any difference, it is possible to add the repo and install the packages and as far the dependencies of the package to be installed can be retrieved from this repo or other repos we are fine.
Updated by JERiveraMoya over 5 years ago
Updated by riafarov over 5 years ago
- Due date changed from 2018-12-18 to 2019-01-15
- Status changed from In Progress to Workable
- Assignee deleted (
JERiveraMoya) - Target version changed from Milestone 21 to Milestone 22
There is small part remaining, so let's try to resolve during the week. If not - next year =)
Updated by JERiveraMoya over 5 years ago
I was learning how to debug the script and at least I feel more comfortable now although it still a long path for me to understand what is doing.
I noticed that before running _reposync, function update_current_repo is executed copying all the files without filter. At the end there is a .new folder under repos/ in correct sync but other folders exists with the whole content. It looks like this:
SLE-15-SP1-Installer-DVD-x86_64-Build128.1-Media1
SLE-15-SP1-Module-Basesystem-POOL-x86_64-Build128.1-Media1
SLE-15-SP1-Module-Basesystem-POOL-x86_64-Build128.1-Media1.new
SLE-15-SP1-Module-Basesystem-POOL-x86_64-CURRENT-Media1 -> SLE-15-SP1-Module-Basesystem-POOL-x86_64-Build128.1-Media1
SLE-15-SP1-Module-Basesystem-POOL-x86_64-Media1-CURRENT
Let's figure out what is the purpose of those functions and discuss how we can continue when you're back @riafarov. Thanks.
Updated by riafarov over 5 years ago
- Assignee set to riafarov
https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/6439/files got merged, https://gitlab.suse.de/openqa/scripts/merge_requests/281 is still not merged and not deployed yet too.
Updated by riafarov over 5 years ago
- Status changed from Workable to Feedback
Updated by riafarov over 5 years ago
- Status changed from Feedback to Resolved