Project

General

Profile

action #90293

coordination #58184: [saga][epic][use case] full version control awareness within openQA, e.g. user forks and branches, fully versioned test schedules and configuration settings

coordination #80372: [epic] Cleanup vars.json as initial information container between openQA worker and isotovideo

coordination #67723: [epic] Remote openQA worker fails to run tests from openqa-clone-custom-git-refspec

Optional relative paths for CASEDIR and others to be not bound to specific workers

Added by okurz 4 months ago. Updated 2 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Feature requests
Target version:
Start date:
2021-03-18
Due date:
% Done:

0%

Estimated time:
20.00 h
Difficulty:

Description

Motivation

See #67723

Acceptance criteria

  • AC1: openQA and os-autoinst can use relative paths to reference the test distribution directory and needles so that jobs can be cloned between different environments without problems

Related issues

Copied from openQA Project - action #90290: Relative paths for CASEDIR and others as default to be not bound to specific workersResolved2021-03-18

History

#1 Updated by okurz 4 months ago

  • Copied from action #90290: Relative paths for CASEDIR and others as default to be not bound to specific workers added

#2 Updated by okurz 4 months ago

  • Assignee set to Xiaojing_liu

https://github.com/os-autoinst/openQA/pull/3799 is merged. As https://github.com/os-autoinst/os-autoinst/pull/1634 already includes a fix for all the cases where we assume we had problems I suggest after deployment to enable the feature on o3 completely, e.g. within the worker config of all our production workers, then wait a week, then enable on OSD, then wait a week, then go on with other features for which we brought in this change initially.

#3 Updated by okurz 4 months ago

  • Status changed from Workable to In Progress

#4 Updated by cdywan 4 months ago

okurz wrote:

https://github.com/os-autoinst/openQA/pull/3799 is merged. As https://github.com/os-autoinst/os-autoinst/pull/1634 already includes a fix for all the cases where we assume we had problems I suggest after deployment to enable the feature on o3 completely, e.g. within the worker config of all our production workers, then wait a week, then enable on OSD, then wait a week, then go on with other features for which we brought in this change initially.

As ENABLE_RELATIVE_PATH is a job setting, do you mean to add it to the worker settings of each worker via the web UI? I was thinking per job group 🤔️

@Xiaojing_liu Please document what you decide to enable

#5 Updated by okurz 4 months ago

cdywan wrote:

As ENABLE_RELATIVE_PATH is a job setting, do you mean to add it to the worker settings of each worker via the web UI? I was thinking per job group 🤔️

So which job groups did you think about then? We want the new behaviour to be default. And given the variable precedence it should be possible to set ENABLE_RELATIVE_PATH=1 in the worker config and where necessary we can set +ENABLE_RELATIVE_PATH=0 e.g. in specific test suites or job groups where we identify problems.

#6 Updated by Xiaojing_liu 4 months ago

cdywan wrote:

okurz wrote:

https://github.com/os-autoinst/openQA/pull/3799 is merged. As https://github.com/os-autoinst/os-autoinst/pull/1634 already includes a fix for all the cases where we assume we had problems I suggest after deployment to enable the feature on o3 completely, e.g. within the worker config of all our production workers, then wait a week, then enable on OSD, then wait a week, then go on with other features for which we brought in this change initially.

As ENABLE_RELATIVE_PATH is a job setting, do you mean to add it to the worker settings of each worker via the web UI? I was thinking per job group 🤔️

@Xiaojing_liu Please document what you decide to enable

I think we shouldn't add it to the worker settings. Just as cdywan said, ENABLE_RELATIVE_PATH is a job setting, we don't add worker's setting into the job's setting, so maybe this way doesn't make sense.
My intent was that adding this param when doing isos post, but I don't know where the command is called when a new build coming.

#7 Updated by openqa_review 4 months ago

  • Due date set to 2021-04-02

Setting due date based on mean cycle time of SUSE QE Tools

#8 Updated by okurz 4 months ago

I think we shouldn't add it to the worker settings. Just as cdywan said, ENABLE_RELATIVE_PATH is a job setting, we don't add worker's setting into the job's setting, so maybe this way doesn't make sense.

Well, everything is a job setting. The worker settings end up in job settings as well, isn't it?

My intent was that adding this param when doing isos post, but I don't know where the command is called when a new build coming.

It comes from https://github.com/os-autoinst/openqa-trigger-from-obs . Don't make that too complicated. Our intention is to have that behaviour enabled by default anyway. So unless you have a better idea I suggest to just enable it everywhere and then we can exclude it where needed until we have all the special cases fixed as well. If you want to go step-wise, how about adding it in the media. But likely these are already also too many to be easily changeable.

#9 Updated by Xiaojing_liu 4 months ago

okurz wrote:

I think we shouldn't add it to the worker settings. Just as cdywan said, ENABLE_RELATIVE_PATH is a job setting, we don't add worker's setting into the job's setting, so maybe this way doesn't make sense.

Well, everything is a job setting. The worker settings end up in job settings as well, isn't it?

My intent was that adding this param when doing isos post, but I don't know where the command is called when a new build coming.

It comes from https://github.com/os-autoinst/openqa-trigger-from-obs . Don't make that too complicated. Our intention is to have that behaviour enabled by default anyway. So unless you have a better idea I suggest to just enable it everywhere and then we can exclude it where needed until we have all the special cases fixed as well. If you want to go step-wise, how about adding it in the media. But likely these are already also too many to be easily changeable.

I see. I wasn't sure if adding the settings in the worker configuration file worked. And you are right, it works. So I think we can enable this by adding setting in worker's files.

#10 Updated by Xiaojing_liu 4 months ago

Xiaojing_liu wrote:

okurz wrote:

I think we shouldn't add it to the worker settings. Just as cdywan said, ENABLE_RELATIVE_PATH is a job setting, we don't add worker's setting into the job's setting, so maybe this way doesn't make sense.

Well, everything is a job setting. The worker settings end up in job settings as well, isn't it?

My intent was that adding this param when doing isos post, but I don't know where the command is called when a new build coming.

It comes from https://github.com/os-autoinst/openqa-trigger-from-obs . Don't make that too complicated. Our intention is to have that behaviour enabled by default anyway. So unless you have a better idea I suggest to just enable it everywhere and then we can exclude it where needed until we have all the special cases fixed as well. If you want to go step-wise, how about adding it in the media. But likely these are already also too many to be easily changeable.

I see. I wasn't sure if adding the settings in the worker configuration file worked. And you are right, it works. So I think we can enable this by adding setting in worker's files.

Adding the setting in worker configuration works, but there is another problem. If we add the setting in worker file, we cannot change it unless we change the value in the worker file. Because we handle the job's setting first, then adding the worker's setting into it. So the worker's setting will override the same job settings. okurz how about adding this to machines' settings? the number of machines is less than the media.

#11 Updated by okurz 4 months ago

This all sounds way too complicated. Again, as the goal should be to have the relative paths the default behaviour I now suggest that we follow our suggestions but in a slightly different order. So create a PR that turns around the feature flag to make relative paths the default unless disabled by a feature flag. Then again, wherever we hit problems in tests the test maintainers can choose which place is the most suitable to go back to absolute paths, be it individual test suites, job groups, machines or workers.

#13 Updated by Xiaojing_liu 4 months ago

  • Due date changed from 2021-04-02 to 2021-04-09

The pr has been merged. Change the due date because of hackweek, and also wait for feedback after the pr is deployed on o3 and OSD.

#14 Updated by Xiaojing_liu 4 months ago

  • Status changed from In Progress to Resolved
  • Estimated time set to 8.00 h

The pr has been deployed on o3 for some days and no special job failed.
The LTP job is passed with "CASEDIR" : "opensuse". See: https://openqa.opensuse.org/tests/1692489#
Cloned a job with ABSOLUTE_TEST_CONFIG_PATHS=1, it works well: See: https://openqa.opensuse.org/tests/1693609/file/vars.json

So I set this ticket Resolved.

#15 Updated by Xiaojing_liu 4 months ago

  • Estimated time changed from 8.00 h to 16.00 h

#16 Updated by okurz 3 months ago

  • Due date deleted (2021-04-09)

#17 Updated by Xiaojing_liu 2 months ago

  • Estimated time changed from 16.00 h to 20.00 h

Also available in: Atom PDF