action #69583: [y] Create repository for 'self_update' feature - qe-yam - openSUSE Project Management Tool

Custom queries

openQA Infrastructure Project
openqa-review - Closed tickets last updated by openqa-review, last 30 days
QA roadmap long-term
QA SLE functional
QA SLE Functional - closed in last 14 days
QA SLE Functional - High, need to be refined
QA SLE Functional - over cycle time median
QA SLE u
QA SLE y
QA tools (tag not necessary in openQA and subprojects)
QA tools tag (tag not necessary in openQA and subprojects; excluding tickets in "Ready" version as they are already on the backlog)
QAC - Backlog
QE tools team - backlog (ready issues)
QE tools team - backlog (w/o infra)
QE tools team - backlog SLA high
QE tools team - backlog SLA immediate
QE tools team - backlog SLA no immediate/urgent in feedback/blocked
QE tools team - backlog SLA normal
QE tools team - backlog SLA urgent
QE tools team - backlog SLO high
QE tools team - backlog SLO normal
QE tools team - backlog SLO urgent
QE tools team - backlog, high-level view (epics and higher)
QE tools team - backlog, non-reactive work, needs parent
QE tools team - backlog, top-level view (all sagas)
QE tools team - closed within last 14 days
QE tools team - closed within last 60 days
QE tools team - closed yesterday
QE Tools Team - Collaborative Session
QE tools team - due date forecast
QE tools team - exceeding due-date
QE tools team - infrastructure backlog
QE tools team - next - sorted by update time
QE tools team - next issues
QE tools team - non-estimated (unblocked) issues (w/o infra)
QE tools team - ready issues - Workable
QE tools team - ready, not assigned/blocked/low
QE tools team - update forecast
QE tools team - updated by priority
QE tools team - what members of the team are working on - Feedback (not-low)
QE Tools Team Backlog By Assignee
QE Yam backlog
QE Yam blocked issues
QE Yam closed yesterday
QE Yam issues IN_PROGRESS
QE Yam new action tickets per day
QE Yam new actions
QE Yam prio high, no updates last 4 weeks
QE Yam prio low & normal, no update for 1 year
QE Yam recjected yesterday
QE Yam resolved yesterday
QE Yam workable actions
soft failure tickets in progress
softfailure tickets resolved
softfailure tickets workable
Tools Team Retrospective
Tools Team Retrospective (not estimated or assigned)

Actions

Copy link

action #69583

closed

coordination #42854: [functional][y][epic] Introduce relevant installation validation test modules

[y] Create repository for 'self_update' feature

Added by oorlov almost 4 years ago. Updated over 3 years ago.

Status:

Resolved

Priority:

Normal

Assignee:

ybonatakis

Target version:

SUSE QA - SLE 15 SP3

Start date:

2020-08-04

Due date:

2020-08-25

% Done:

Estimated time:

8.00 h

Description

Motivation¶

'self_update=1' can be specified in boot parameters to update installer or 'self_update=0' to skip the updating.
The test suites yast_self_update and yast_no_self_update do not have any validation, they just installs the system.
The goal of parent ticket is to add such validation for the test suites. Please, see the parent ticket for more details.

We could use package like vim.

Task¶

In case when self_update=1 was specified in boot parameters, but there are no updates in repo, installer does not download anything and the /downloads directory is empty. So that it is impossible to verify that the feature works correctly.

In order to have the same expected result on each test run, it is decided to create a repo for this specific purpose, where the updates will exist at any time, regardless of the current product version.

Acceptance criteria¶

Repository with the installer update is added;
The repository is accessible during installation and its updates do not cause any issues in the installed product.

Files

ftp_repo_self_update.png (61.1 KB) ftp_repo_self_update.png

ybonatakis, 2020-08-20 15:30

History
Notes
Property changes

Actions

Copy link

Updated by riafarov almost 4 years ago

Description updated (diff)
Status changed from New to Workable
Estimated time set to 8.00 h

Actions

Copy link

Updated by ybonatakis over 3 years ago

Status changed from Workable to In Progress
Assignee set to ybonatakis

Actions

Copy link

Updated by ybonatakis over 3 years ago

File ftp_repo_self_update.png ftp_repo_self_update.png added

Steps to create repo and publish it:

We need a directory where we can save all the rpms that we want our repository to serve. To create the structure of the repo we just run createrepo_c. This will generate the metadata(or even the db) based on the packages which it will find in the repo. After the execution you should find the repodata folder with repomd.xml

i found vim-8.2.1412-631.1.x86_64.rpm from https://software.opensuse.org/download.html?project=editors&package=vim and i execute

# createrepo_c -v --update --no-database .

The --update parameter checks the metadata to decide if there is a change for each package. If the repo is new is in no use. The --no-database will not create a db. based on the documentation the db is for yum.

View screenshot...View screenshot...

The above can take place local. Then we have to upload the folder in OSD.

Next step is to sign the repo. This is important otherwise the system will complain during the installation ( http://aquarius.suse.cz/tests/3122#step/welcome/7). The public and the private key should located on the OSD(or wherever the repo is hold). One caveat is that we have to sign it every time we change a package in the repo.

We sign the repository and export the keys in the repo:
from OSD inside the folder that we copied

sudo gpg -a --detach-sign --default-key qayast repodata/repomd.xml
sudo gpg -a --export qayast > repodata/repomd.xml.key #This does not work actually because of the permission on the dir but the `gpg` cmd still stands to see the key and workaround the problem.

Now the test needs some editing because we still have to accept the key in the welcome module. We already have handle_untrusted_gpg_key which we can reuse, i guess.
Also the test fails so i need to investigate why the update repo is not used. installed package is vim-8.0.1568-631.5.6.1.x86_64.rpm .The expected one from the repo is vim-8.2.1412-631.1.x86_64.rpm

Notes:

~~it is supposed that the yast should be restarted[0]. it doesnt~~.
Also on the same link it says that "The installer self-update is only available if you use the GM images of the Unified Installer and Packages ISOs.". Does this mean that it doesnt work for alpha/beta?

[0] https://documentation.suse.com/sles/15-SP1/html/SLES-all/cha-install.html#sec-yast-install-self-update-apply-updates

Actions

Copy link

Updated by ybonatakis over 3 years ago

So the rpm repository[0] works. When i open the editor (it found as vim-nox11 instead vim) the version of the updated package showed up. rpm and zypper is not in any use. zypper is not installed in this point and rpm does not carry a db. In additional --version argument did not work.

Based on the above, the validations was not easy. Lesson learned is that YaST blindly installs all packages it finds in the self-update. YaST holds packages' version in /.packages.root. We can use this to check the expected version in inst-sys. However this is not updated with the updated versions retrieved by the provided rpm repository in the self_update. After talking to the devs, they let me know that they could provide a file similar to /.packages.root that it will have this information. bug ticket has been filed for it[1]

In additional, another point for testing that self_update works, is that we do not need any full featured package, we can create any dummy package with just some README file and check it is later present in the inst-sys. that would avoid any dependency problems and we would not need to execute anything on the system. this is came up as some problems that i had with the vim's behavior can be due to some dependencies. I have to find its dependencies, add them in the repo and try again but i dont think it is necessary for this ticket.

Finally the repo works passing ftp or http. But i did not managed to make it work with https. I tried with ssl.certs=0 and i add suse_ptf_key.asc in the repository thinking that this would fix the communication for the SSL but i had no success.

i have push changes for the test to use the repo[2] and the required needle[3]

I am putting this ticket in feedback as i have to provide documentation in our gitlab repo and create follow up tickets.

[0] https://openqa.suse.de/assets/repo/SLE-15-x86_64-self_update/
[1] https://bugzilla.suse.com/show_bug.cgi?id=1175614
[2] https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/10864
[3] https://gitlab.suse.de/openqa/os-autoinst-needles-sles/-/merge_requests/1418

Actions

Copy link