action #43511
closed[functional][u] Asset cache does not support ASSET_ assets
0%
Description
Observation¶
In job this job there's an extra asset being defined, and Gru has already code to handle "external downloads" however the caching does not handle these, causing the job to die.
ASSET_1_URL https://w3.nue.suse.com/~slindomansilla/tmp/bootloader_poo40670.pm
SCHEDULE bootloader_poo40670,tests/installation/welcome
error on bootloader_poo40670.pm: Can't locate ../../../pool/1/bootloader_poo40670.pm in @INC (you may need to install the ..::..::..::pool::1::bootloader_poo40670 module) (@INC contains: ../../../pool/1 /var/lib/openqa/cache/tests/sle/lib /var/lib/openqa/pool/1/blib/arch /var/lib/openqa/pool/1/blib/lib /usr/lib/os-autoinst /usr/lib/perl5/site_perl/5.18.2/ppc64le-linux-thread-multi /usr/lib/perl5/site_perl/5.18.2 /usr/lib/perl5/vendor_perl/5.18.2/ppc64le-linux-thread-multi /usr/lib/perl5/vendor_perl/5.18.2 /usr/lib/perl5/5.18.2/ppc64le-linux-thread-multi /usr/lib/perl5/5.18.2 /usr/lib/perl5/site_perl .) at (eval 739) line 1.
In a more optimal scenario, the webUI shows a list of assets to download, and the cache just takes care of said list, except the repos.
Acceptance criteria¶
- AC1: A job scheduled with a downloadable override test module can run successfully on osd
Suggestions¶
- Hook into the detect_asset_keys method of OpenQA::Worker::Engines::isotovideo and add handling for the ASSET_*_URL.
Updated by okurz over 5 years ago
- Subject changed from [y][u] Asset cache does not support 'other' assets to [functional][u] Asset cache does not support 'other' assets
- Description updated (diff)
- Assignee set to szarate
- Target version set to Milestone 21
@szarate please clarify the suggestions to make it workable
Updated by szarate over 5 years ago
- Status changed from New to In Progress
After looking at changes to the cache, I think this can be done independantly. as we already support KERNEL and INITRD. and the handling for asset urls didn't change...
Working on this today.
Updated by szarate over 5 years ago
- Status changed from In Progress to Feedback
Updated by okurz over 5 years ago
PR merged, we should be able to test this tomorrow on o3 assuming that the auto-deployment of workers does its job.
Updated by AdamWill over 5 years ago
I think the attempted fix here was wrong and would not have worked. I think https://github.com/os-autoinst/openQA/pull/1907 should fix it. If someone can check this out, that'd be great (I don't have worker caching enabled on the Fedora instances ATM, so I can't test it out Most Interesting Man In The World-style):
Updated by AdamWill over 5 years ago
- Subject changed from [functional][u] Asset cache does not support 'other' assets to [functional][u] Asset cache does not support ASSET_ assets
Updated by szarate over 5 years ago
- Assignee deleted (
szarate)
More information can be found on: https://github.com/os-autoinst/openQA/pull/1907#issuecomment-444394975
Long story short: loadtests needs to support loading test scripts out of tree (out of $CASEDIR), for this either the find_script or the load_tests itself needs to be modified in the os_autoinst project, by altering the behaviour if the SCHEDULE backend variable is set.
Updated by okurz over 5 years ago
- Status changed from Feedback to Workable
Also as agreed we set back to "Workable", no use to have it in "Feedback" without an assignee.
szarate wrote:
More information can be found on: https://github.com/os-autoinst/openQA/pull/1907#issuecomment-444394975
Long story short: loadtests needs to support loading test scripts out of tree (out of $CASEDIR), for this either the find_script or the load_tests itself needs to be modified in the os_autoinst project, by altering the behaviour if the SCHEDULE backend variable is set.
But that is already working, see https://github.com/os-autoinst/os-autoinst/pull/1019 . The problem is only with cache it seems.
Updated by szarate over 5 years ago
- Assignee set to szarate
I m taking this one back for the time being :)
Updated by szarate over 5 years ago
- Status changed from Workable to In Progress
I'm still looking at bits and pieces, but the asset isn't really downloaded for the time being to the pool nor is linked in the test. Initially cache wouldn't look at other assets except for kernel and initrd, downloadable assets were never meant to be part of cache (it came after)...
So for the time being I guess it's matter of registering a downloaded asset as part of a job (which somehow already is), and ensure the cache properly links it. Perhaps in the meantime, explain a bit better how it's all connected
Updated by okurz over 5 years ago
- Target version changed from Milestone 21 to Milestone 22
Updated by szarate over 5 years ago
- Status changed from In Progress to Workable
I will drop this one for now. I'm taking too long figuring out how to add unit tests for the download part in gru (Note that I mean both things together ;) not the unit tests that are in 14-grutasks.t)
Updated by szarate over 5 years ago
- Related to action #45038: [functional][y][leap] test fails in installation - autoyast profile can not be retrieved, no network? added
Updated by szarate about 5 years ago
- Status changed from Workable to In Progress
Updated by szarate about 5 years ago
- Related to action #46094: [sle][functional][u] test fails in installation - missing assets or wrong link added
Updated by szarate about 5 years ago
- Related to deleted (action #46094: [sle][functional][u] test fails in installation - missing assets or wrong link)
Updated by szarate about 5 years ago
- Blocks action #46094: [sle][functional][u] test fails in installation - missing assets or wrong link added
Updated by szarate about 5 years ago
- Status changed from In Progress to Feedback
I'm disabling for the time being changes done by https://github.com/os-autoinst/openQA/pull/1907 and https://github.com/os-autoinst/openQA/pull/1853 since it looks like the dynamic scheduling of test modules conflicts a bit with jobs that depend on ASSET_1_URL.
Updated by okurz about 5 years ago
- Target version changed from Milestone 22 to Milestone 23
Updated by okurz about 5 years ago
- Blocks action #36994: Dynamic test flow definition + override added
Updated by okurz about 5 years ago
- Status changed from Feedback to Resolved
PR was deployed to osd and looks good, e.g. see https://openqa.suse.de/tests/2506696 . I think we are done here. Followup work can be done in #36994 . Thanks for your work.