Project

General

Profile

action #88347

upload_asset function fails for larger files

Added by psladek 6 months ago. Updated 2 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Concrete Bugs
Target version:
Start date:
2021-01-29
Due date:
% Done:

0%

Estimated time:
Difficulty:
easy

Description

Observation

When the upload_asset() function is called in SUT on files larger than (exactly) 256kiB, they are uploaded into pool/X/assets_public (or pool/X/assets_private) with 0600 permissions (and not 0644 as for smaller files) with _openqa_worker user. This is than unreadable from server side for geekotest user and upload fails with :
'...Failed receiving asset: Can't copy file "/var/lib/openqa/pool/1/assets_public/testfile" to "/var/lib/openqa/share/factory/tmp/public/testfile.CHUNKS/testfile": Permission denied at /usr/share/openqa/script/../lib/OpenQA/Schema/Result/Jobs.pm...'
in worker log.
This problem appears only if LOCAL_UPLOAD (very useful feature otherwise) is set.

Steps to reproduce

  • Crosscheck original description when tests upload files larger than 256kiB

Problem

It turns out that this can be fixed by adding
'chmod 0644, $file;'
line in
lib/OpenQA/Worker/Job.pm
in either
'_upload_asset' sub in 'upload_local.prepare' => sub {' block,
or in
'_stop_step_4_upload' sub below 'next unless -f $file;' line.
This fixes permissions just before the final upload. If fix like this is ok, than the issue can probably be considered resolved. However as it seems that it can be fixed on more places, I am not sure about finding best solution, since I am not at all expert on the client or server upload code. It should probably be fixed already at the moment the file is placed in the pool/X/assets_public directory, or in some even more appropriate places.

Suggestion

  • Understand the current code and where it distinguishes different flows based on the file size (likely related to upload chunk size)
  • Fix in the same way regardless of the file size

History

#1 Updated by okurz 6 months ago

  • Description updated (diff)
  • Status changed from New to Workable

Hi, thanks for the report. And thanks for proposing the "easy solution" as well as suggestions to check deeper. I updated the ticket description following the template from https://progress.opensuse.org/projects/openqav3/wiki/#Defects

#2 Updated by kraih 5 months ago

I think the proposed fix makes sense. When i first proposed the fast local upload feature i was a little worried about permission issues like this, since the code only does a chmod 644 on the webui side currently.

#3 Updated by kraih 5 months ago

And looking at the code, upload_local.prepare seems the right place to the chmod.

#4 Updated by okurz 5 months ago

  • Status changed from Workable to Feedback
  • Assignee set to psladek

#5 Updated by okurz 5 months ago

merged

#6 Updated by okurz 2 months ago

  • Status changed from Feedback to Resolved

Assuming fixed

Also available in: Atom PDF