coordination #36958
closed[functional][y][epic] Check checksum of image downloaded from OBS to worker
100%
Description
It happened in https://bugzilla.suse.com/show_bug.cgi?id=1096416 that image was downloaded incorrectly to the Hyper-V host resulting in all Hyper-V tests to fail in a non-obvious way (looked like a product bug).
I don't know what/if any checksum is checked when qemu backend is used but in case of svirt backend there's no checksum validation at all.
In case of svirt backend the checksum has to be done in the virtualizaion host not the worker (sic). And perhaps test code is the right place for the check.
I am hearing that OBS generates SHA checksums for each build, we may need to get them with the image when rsync.pl is triggered.
Updated by okurz almost 6 years ago
[7 Jun 2018 13:34:46] okurz: mnowak: isn't openqa checking image integrity before starting to test them ?
[7 Jun 2018 13:35:49] fcrozat: unfortunately not in all cases, e.g. when copying from worker to hyperv host I guess
[7 Jun 2018 13:37:45] fcrozat, not in this case (don't know if in any case actually). well for that we would need some authoritative checksum of the build
[7 Jun 2018 13:38:15] there are checksums, in the download tree on IBS
[7 Jun 2018 13:38:38] and the iso also contains the checksum, embedded, IIRC
[7 Jun 2018 13:39:32] How can the iso contain its checksum?
[7 Jun 2018 13:39:48] mnowak: we always generate .sha256 files next to the images
[7 Jun 2018 13:40:41] fvogt: because the program validating it skips the part where the sum is stored ;)
[7 Jun 2018 13:52:40] fcrozat: you need to use checkmedia to validate the checksum - not really generically usable
[7 Jun 2018 13:52:55] coolo: thanks for the info
[7 Jun 2018 13:53:29] at least checkmedia support sha256+ :)
[7 Jun 2018 13:53:35] coolo: fcrozat the .sha256 files can be used with the classic sha256sum application
[7 Jun 2018 13:54:31] eg:
[7 Jun 2018 13:54:35] sha256sum -c openSUSE-Tumbleweed-NET-x86_64-Build260.1-Media.iso.sha256
[7 Jun 2018 13:54:36] openSUSE-Tumbleweed-NET-x86_64-Build260.1-Media.iso: OK
Updated by okurz almost 6 years ago
- Subject changed from [functional] Check checksum of image downloaded from OBS to worker to [functional][y] Check checksum of image downloaded from OBS to worker
- Target version set to Milestone 18
Updated by okurz almost 6 years ago
- Target version changed from Milestone 18 to Milestone 18
Updated by okurz over 5 years ago
- Target version changed from Milestone 18 to Milestone 19
Updated by riafarov over 5 years ago
- Subject changed from [functional][y] Check checksum of image downloaded from OBS to worker to [functional][y][epic] Check checksum of image downloaded from OBS to worker
Updated by okurz over 5 years ago
- Target version changed from Milestone 19 to Milestone 20
Updated by riafarov over 5 years ago
- Status changed from New to Feedback
- Assignee set to riafarov
Updated by okurz over 5 years ago
So by now I guess we have the biggest part solved. What I see as missing now is to add a check to all more complicated setup then just qemu jobs on workers, e.g. the originally mentioned hyper-v ones. However I guess mnowak can use the provided checksum in the test settings easily with e.g. a one-line change or we wait for the next case of failed jobs and then see to add what is missing.
IMHO ticket can be set to "Resolved" now.
Updated by riafarov over 5 years ago
- Due date changed from 2018-10-23 to 2019-01-02
due to changes in a related task
Updated by okurz over 5 years ago
- Target version changed from Milestone 20 to Milestone 22
okurz wrote:
IMHO ticket can be set to "Resolved" now.
I learned that the checksum is also not actively used in the svirt based tests only the test module was updated to support it in general.
Updated by riafarov over 5 years ago
- Due date changed from 2019-01-02 to 2019-01-29
due to changes in a related task
Updated by riafarov over 5 years ago
- Due date changed from 2019-01-29 to 2019-02-12
due to changes in a related task
Updated by riafarov about 5 years ago
- Due date changed from 2019-02-12 to 2019-02-26
due to changes in a related task
Updated by okurz about 5 years ago
- Status changed from Feedback to Blocked
- Target version changed from Milestone 22 to Milestone 23
okurz wrote:
"Blocked" by subtask?
answering myself: I guess, yes, that is a better approach :)
Updated by riafarov about 5 years ago
- Due date changed from 2019-02-26 to 2019-03-12
due to changes in a related task
Updated by riafarov about 5 years ago
- Status changed from Blocked to Resolved
I believe we can resolve this one, unless we want to start from scratch and do image validation in caching. Tricky part there is to get value provided by rsync, and I guess it will be better to do those independently on openQA side, meaning, calculating image checksum before publishing and then comparing when download to the worker. @okurz, please reopen in case of any objections or if you would like to try with caching as a part of this ticket.
Updated by szarate over 3 years ago
- Tracker changed from action to coordination
Updated by szarate over 3 years ago
See for the reason of tracker change: http://mailman.suse.de/mailman/private/qa-sle/2020-October/002722.html