QA » openQA Project

It looks like we have very little logic in openQA itself. The code of the openQA rsync plugin is mainly providing a UI for displaying generated scripts and acts as a generic runner for a shell script, see https://github.com/os-autoinst/openQA/pull/5693.

The shell script in question is https://github.com/os-autoinst/openqa-trigger-from-obs/blob/master/script/rsync.sh and that script defines all the logic. It actually defines the sequence in which to run some other (generated) scripts and lists of commands. So the place to add a hook script would probably be that script and thus the https://github.com/os-autoinst/openqa-trigger-from-obs repository (and not openQA).

I think that this makes things actually even simpler:

• I just added a new file under my local openqa-trigger-from-obs/openSUSE:Factory:Staging:A/.run_240612_111353_551.1/ directory and it just showed up on http://localhost:9526/admin/obs_rsync/openSUSE:Factory:Staging:A/runs/.run_240612_111353_551.1 without code modifications. The web UI is really just displaying a directory listing here. So you can generate whatever scripts/artifacts you like and they will display on the web UI without extra effort.
• To actually run an additional script and save its output you can just add a line like the existing line [ ! -e "$subfolder/print_rsync_iso.sh" ] || bash -e "$subfolder/print_rsync_iso.sh" > $logdir/rsync_iso.cmd 2> >(tee $logdir/generate_rsync_iso.err) somewhere in script/rsync.sh where needed. In this script you can call e.g. some OBS APIs to gather additional information about the project in question and store it in a way so it can be consumed in subsequent scripts as needed.
• You probably need to modify some of the existing scripts as well as you probably want to modify the set of openQA jobs being created - and that is done by the script print_openqa.sh. You needed to modify it so it generates a different list of openqa-cli commands based on the information gathered before (as mentioned in the previous point). However, that script is actually just generated. It is generated by the function def gen_print_openqa(self, f): in script/scriptgen.py. So you probably needed to modify that function instead of the script itself.

Well, the last point of modifying the set of jobs to create is actually not that simple. In fact, modifying a Python script in order to modify a shell script is rather indirect. However, that is were it is decided what set of jobs will be scheduled - so that's what we need to modify and there's not much we can do about it from the openQA side. (In case this indirection gets more in the way than it is helpful it would be possible to not use it for new projects, though. You could simply not use script/scriptgen.py for your new projects and instead just handwrite one script to rule them all. The script could just get its parameters as needed (or consider its current working directory and files in it)¹. This would require only minor changes in script/rsync.sh about how the script generating jobs is invoked.)

That means: There is already a way to add a "hook" script. Just add another script invocation within script/rsync.sh where needed. The hard part is modifying the code that determines the set of jobs to create.

¹ And when I compare e.g. colordiff -Naur openSUSE:Factory:Staging:{A,B}/print_openqa.sh I see that those scripts really mainly differ in the name of the staging ("A" vs. "B"). So just considering that, using script/scriptgen.py is really an overkill in that case. One could just write one script once and it could determine the difference based on its project directory. Even the diff between e.g. colordiff -Naur openSUSE:{Factory:Staging:A,Slowroll:Staging}/print_openqa.sh is not that big.

You could simply not use script/scriptgen.py for your new projects and instead just handwrite one script to rule them all.

I was there Marius, 3000 days ago, when we had rsync.pl... and I would like to avoid the mess that rsync.pl was back then.

Well, the last point of modifying the set of jobs to create is actually not that simple. In fact, modifying a Python script in order to modify a shell script is rather indirect. However, that is were it is decided what set of jobs will be scheduled - so that's what we need to modify and there's not much we can do about it from the openQA side.

openQA only needs to sit there, be pretty and process the isos post. The sync script is just an rsync + glue code to generate an isos post.

As discussed please break out a "[timeboxed:20h][spike solution] adaptable product scheduling in openqa-trigger-from-obs" ticket including pair-programming with a member of QE Core.