action #80372

coordination #58184: [saga][epic][use case] full version control awareness within openQA, e.g. user forks and branches, fully versioned test schedules and configuration settings

[epic] Cleanup vars.json as initial information container between openQA worker and isotovideo

Added by okurz about 2 months ago. Updated about 1 month ago.

Feature requests
Target version:
Start date:
Due date:
% Done:


Estimated time:
(Total: 0.00 h)



openQA leaks internal data about cache directory and other data into vars.json and to isotovideo. This causes problems like #67723 that we try to fix, makes it impossible for any user to run tests just by downloading the vars.json file without changes and also makes the "investigation" tab show up irrelevant changes when a scenario has been running jobs on different worker hosts or instances which is likely to happen. We should ensure a proper design from openQA side that we only write to vars.json what is portable and make os-autoinst cope with that.

Also see #58184

Acceptance criteria

  • AC1: #67723 can be fixed without hacky workarounds (okurz: I know, this is quite subjective)
  • AC2: A vars.json file downloaded from any production worker can be run locally with isotovideo without needing to replace all "cache" related paths to point to local paths
  • AC3: A diff of vars.json files between two jobs from different worker instances and/or machines within one scenario show less irrelevant, automatic changes, e.g. no "cache" related ones


  • As os-autoinst already supports relative paths, how about not writing something like PRODUCTDIR=/var/lib/openqa/cache/openqa1-opensuse/tests/opensuse/products/opensuse but symlink the test distribution into the current pool folder of the worker and use just PRODUCTDIR=opensuse/products/opensuse? As an alternative, maybe we do not need to care about PRODUCTDIR in openQA at all but move that logic to os-autoinst.

  • As discussed in daily 2020-11-27 we were thinking about another approach, probably even easier: We can add a default fallback value for CASEDIR to read the current directory. Then in the openQA worker we create the symlink as mentioned above and do not need to specify CASEDIR at all


action #67723: Remote openQA worker fails to run tests from openqa-clone-custom-git-refspecIn ProgressXiaojing_liu

action #81703: The values of 'ISO' and 'HDD' includes absolute path in vars.jsonNew


#1 Updated by Xiaojing_liu about 2 months ago

  • Blocks action #67723: Remote openQA worker fails to run tests from openqa-clone-custom-git-refspec added

#2 Updated by okurz about 2 months ago

  • Description updated (diff)

#3 Updated by cdywan about 2 months ago

  • Status changed from New to Feedback

Another approach from Jane is being discussed on GitHub, hence Feedback:

#4 Updated by okurz about 2 months ago

  • Status changed from Feedback to Workable

it's good that we explore the ideas but actually I would like to set the epic to "Workable" as anyone can be the epic owner and create according subtasks

#5 Updated by okurz about 1 month ago

Based on our discussion today and me looking into the current use of DISTRI and other variables within os-autoinst I am once again sure that we should not need any additional variables within os-autoinst.
Unless I am mistaken os-autoinst already has all variables that are needed to describe all possible situations that we need to support. And all variables should support relative directories, i.e. CASEDIR, PRODUCTDIR, NEEDLES_DIR. We already mentioned this in .

My suggestion in is still true, mainly about relative directories and providing symlinks within the scope of openQA.

My idea is (still) that with just relative directories specified by openQA and all necessary data available from these relatives directories it should be possible to fully specify what os-autoinst needs but still provide clone-able jobs.
I am trying to come up with some different cases we need to support with according example variable settings:

  1. absolute casedir path with needles in same repo, no caching, no git: casedir=/var/lib/openqa/share/tests/foo (+ needles in needles/, not necessary to specify, read by default in
  2. absolute casedir path with productdir, no caching, no git: casedir=/var/lib/openqa/share/tests/foo + productdir=products/opensuse (+ needles in needles/, now relative to productdir, still not necessary to specify, also covered by
  3. casedir path with productdir, caching, no git: casedir=foo (symlink to /var/lib/…/cache/…) + productdir=products/opensuse (+ needles in needles/, still relative to productdir, same as in 2.)
  4. casedir path with productdir, caching, git, needles not in same git: casedir= + productdir=products/opensuse + needles_dir=needles (now specified explicitly, symlinked from cache into pool)

So with this I expect that we can get the change done without needing to change anything in os-autoinst, just by providing symlinks in the openQA worker pool directory and specifying relative directory variables instead of absolute ones.

#6 Updated by Xiaojing_liu 16 days ago

  • Blocks action #81703: The values of 'ISO' and 'HDD' includes absolute path in vars.json added

#7 Updated by Xiaojing_liu 16 days ago

  • Blocks deleted (action #81703: The values of 'ISO' and 'HDD' includes absolute path in vars.json)

Also available in: Atom PDF