Project

General

Profile

Actions

action #17760

closed

Upload .chksum files together with assets

Added by oholecek about 7 years ago. Updated over 6 years ago.

Status:
Rejected
Priority:
Normal
Assignee:
-
Category:
Feature requests
Target version:
-
Start date:
2017-03-16
Due date:
% Done:

0%

Estimated time:

Description

For asset caching we need some way of ensuring integrity and that the asset is up to date. At the same time we don't want WebUI to spend time computing checksums.

To solve this we can let worker to upload assets checksum in a separate file as is usual for i.e. distribution of isos.

e.g. - create and upload install_hdd.qcow2 and install_hdd.qcow2.chksum files.

WebUI then must enable distribution of both files.


Related issues 1 (0 open1 closed)

Precedes openQA Project - action #17574: [tools]Add caching/syncing of assetsResolvedszarate2017-04-29

Actions
Actions #1

Updated by szarate about 7 years ago

Who should compute the checksum for the isos?. WebUI? Rsync.pl?

Actions #2

Updated by oholecek about 7 years ago

Whoever does that for isos i.e. in here http://download.opensuse.org/tumbleweed/iso/ I assume that rsync.pl will need to be adapted to sync them as well for oS, and some similar adaptation for SLE.

Actions #3

Updated by szarate about 7 years ago

  • Precedes action #17574: [tools]Add caching/syncing of assets added
Actions #4

Updated by coolo about 7 years ago

  • Assignee set to szarate

For ISOs it's not soo important, but if we retrigger a job creating a HDD we need to make sure the children get the new HDD. So the worker would download the .chksum first unconditionaly and without caching and then verify what's in the cache.

As HDD checksums are calculated on the worker anyway, I guess I would just do it in the ISO controller when a new ISO is initially posted - unless the caller passes a chksum with it. But that means we need to support multiple checksums IMO - IBS creates sha256 because it's for security, but for HDDs we only care for integrity, so crc would be enough.

So a generic .chksum file with sha256:0fbef344d7ba1ec7da97443b9a1e5b2c2e8312657e701de349a99aee81c9079b or cksum:18307193911 - that would be enough to know if a cache hit is good enough. Or do we want to recalculate the chksum on worker download as well? Then I suggest we ignore the sha256 and calculate CRC in any case.

Actions #5

Updated by coolo about 7 years ago

  • Subject changed from Upload .chksum files together with assets to [tools] Upload .chksum files together with assets
  • Target version set to Milestone 6
Actions #6

Updated by szarate almost 7 years ago

  • Assignee deleted (szarate)
  • Target version deleted (Milestone 6)
Actions #7

Updated by szarate almost 7 years ago

Setting to future, as we don't really need this now (superseeded by etag + check of filesize by the cache)

Actions #8

Updated by coolo over 6 years ago

  • Subject changed from [tools] Upload .chksum files together with assets to Upload .chksum files together with assets
  • Status changed from New to Rejected

So let's assume we don't need this at the moment - for the future we have the more generic #13646

Actions

Also available in: Atom PDF