QA » openQA Project

Discussed with tinita in person: We should collect our experiences from this proof of concept and then close this ticket. We can follow up with this work elsewhere to not make this too big.

I made a Proof of Concept in the branch https://github.com/os-autoinst/os-autoinst/compare/perlpunk/test-actions

Workflow definition: https://github.com/os-autoinst/os-autoinst/blob/perlpunk/test-actions/.github/workflows/linux.yml

Test run: https://github.com/os-autoinst/os-autoinst/runs/666857696

I had to use the patched qemu packages. Thanks Andrii for showing me.

I split up the workflows already in test and style.

Pros¶

• Supports workflows, optionally depending on each other, so we can get early results
• Caching can be used to cache results of bigger tasks for later runs (for example exported docker images)
• Artifacts are supported and can also be used to share files between different workflows
• Shows timestamps for every log line optionally, so it makes it easier to see what took longer than expected
• Every workflow is shown in the Github PR seperately, like in CircleCI
• Supports colored output. All our debug log messages from autoinst are shown in purple. Easier for the eyes to see unexpected warnings

Cons¶

• Artifacts can not be viewed directly in the Web UI, but must be downloaded as zip files
• Seems a bit slower than Travis (total time for the two seperated workflows 12m, vs. 9-10m in Travis)
• Doesn't support YAML *aliases (and therefor also no Merge Keys <<)
• No shared directory between different workflows, so you have to use Artifacts or Caching to share something

TODO¶

codecov is not yet working, complaining with:

{"meta": {"status": 400}, "error": {"reason": "Please provide the repository token to upload reports via `-t :repository-token`", "context": null}}

Conclusion¶

We could already use this, if we fix two issues:

• We should put the patched qemu into the openqa_dev container first. (But installing it takes only half a minute)
• codecov needs to be working

Notes¶

An example of using artifacts can be seen here:

• Upload: https://github.com/ingydotnet/yaml-libyaml-pm/blob/master/.github/workflows/linux.yml#L61
• Download: https://github.com/ingydotnet/yaml-libyaml-pm/blob/master/.github/workflows/linux.yml#L92

Created https://progress.opensuse.org/issues/66721 as a followup