QA » openQA Project

As a general draft including multiple changes: gh#os-autoinst/os-autoinst#1563

But for easier reviewing I created specific pull requests for smaller, not inter-dependant changes, e.g.:

• Use Test::Most instead of Test::More for convenience gh#os-autoinst/os-autoinst#1564 (merged)
• Cover all test output in t/13-osutils and remove unneeded debug messages gh#os-autoinst/os-autoinst#1565 (merged)
• README: Prefer user-centric documentation over developer-centric instructions gh#os-autoinst/os-autoinst#1566 (merged)
• docker: Add Dockerfile for isotovideo+qemu gh#os-autoinst/os-autoinst#1567

Now I wonder what is the best approach for testing. I would like to have a testing solution that is callable by every developer locally and also in CI. I tried with a simple perl test module where I call something like qx{podman build …} which fails with errors like:

#   Failed test 'container could be built successfully'
#   at t/30-container.t line 36.
#          got: '125'
#     expected: '0'
# Output: STEP 1: FROM registry.opensuse.org/opensuse/tumbleweed
# Getting image source signatures
# Copying blob sha256:7fce653214d4500372a2df7b523918146b0f8e61e62613feaab18a46393e96e1
# Copying config sha256:2eac6045a15c8023b0a89aa585850c17ba02bae23185557f6edc0ac57939c316
# Writing manifest to image destination
# Storing signatures
# time="2020-11-15T14:45:15+01:00" level=error msg="Error while applying layer: ApplyLayer exit status 1 stdout:  stderr: there might not be enough IDs available in the namespace (requested 0:15 for /etc/shadow): lchown /etc/shadow: invalid argument"
# Error: error creating build container: Error committing the finished image: error adding layer with blob "sha256:7fce653214d4500372a2df7b523918146b0f8e61e62613feaab18a46393e96e1": ApplyLayer exit status 1 stdout:  stderr: there might not be enough IDs available in the namespace (requested 0:15 for /etc/shadow): lchown /etc/shadow: invalid argument

which I need to understand first or not follow up with at all as it can be just enough to build-test the container within OBS.

EDIT: I understood the error which I could reproduce with perl -e 'qx{podman build -f docker/isotovideo/Dockerfile.qemu . }'. Problem is my alias alias podman='sudo podman' because rootless podman does not work out of the box, at least not in my env. Trying with docker.

Further ideas what I found on the go:

• Use https://github.com/wagoodman/dive like CI=true dive <your-image> to check image
• Use dgoss https://medium.com/@aelsabbahy/tutorial-how-to-test-your-docker-image-in-half-a-second-bbd13e06a4a9 for having a YAML based expectation document to test against our containers

I created https://github.com/os-autoinst/os-autoinst-distri-example/pull/4 with a github actions definition file using a temporary OBS project https://build.opensuse.org/package/show/home:okurz:container/isotovideo-qemu

https://github.com/okurz/os-autoinst-distri-example/runs/1402935643?check_suite_focus=true shows the results when running with qemu_no_kvm=1 as github actions do not provide /dev/kvm, so no nested virt. I could define an example in travis CI in parallel as well.
Changed the Dockerfile to do RUN zypper -n in os-autoinst qemu-x86 qemu-tools instead. Probably I should be explicit and define a "isotovideo:qemu-x86" container and one "isotovideo:qemu-kvm". Have one https://build.opensuse.org/package/show/home:okurz:container/isotovideo-qemu-kvm now and
https://build.opensuse.org/package/show/home:okurz:container/isotovideo-qemu-x86

https://github.com/os-autoinst/os-autoinst-distri-example/pull/4 is ready now with a correct expected successful isotovideo run in
https://github.com/okurz/os-autoinst-distri-example/runs/1403357323?check_suite_focus=true
where "successful" means a completed test run with result "failed" as the example repo has no needles which is left as the training step to complete for beginners using the example repo.

So I could update https://github.com/os-autoinst/os-autoinst/pull/1567

waiting for https://build.opensuse.org/package/live_build_log/devel:openQA/isotovideo-qemu-x86/containers/x86_64 and it being published on registry.opensuse.org