Project

General

Profile

Actions
Copy link

action #77905

closed

coordination #58184: [saga][epic][use case] full version control awareness within openQA

coordination #48641: [epic] Trigger openQA tests in pull requests of any product github pull request

coordination #77698: [epic] synchronous qemu based system level test in pull request CI runs, e.g. standalone isotovideo or openQA tests

CI pipeline proof-of-concept running isotovideo

Added by okurz over 3 years ago. Updated over 3 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
okurz
Category:
Feature requests
Target version:
Ready
Start date:
2020-11-15
Due date:
% Done:

0%

Estimated time:

Description

Acceptance criteria

  • AC1: A proof-of-concept CI configuration and run showing how isotovideo executes tests exists

Suggestions

Actions
Copy link
 #1

Updated by okurz over 3 years ago

  • Status changed from Workable to In Progress
  • Assignee set to okurz

I have https://build.opensuse.org/package/view_file/home:okurz:container/isotovideo/Dockerfile?expand=1 since about a year. Let's see if I can streamline that for upstream inclusion and use it in github actions.

Actions
Copy link
 #2

Updated by okurz over 3 years ago

As a general draft including multiple changes: gh#os-autoinst/os-autoinst#1563

But for easier reviewing I created specific pull requests for smaller, not inter-dependant changes, e.g.:

Now I wonder what is the best approach for testing. I would like to have a testing solution that is callable by every developer locally and also in CI. I tried with a simple perl test module where I call something like qx{podman build …} which fails with errors like:

#   Failed test 'container could be built successfully'
#   at t/30-container.t line 36.
#          got: '125'
#     expected: '0'
# Output: STEP 1: FROM registry.opensuse.org/opensuse/tumbleweed
# Getting image source signatures
# Copying blob sha256:7fce653214d4500372a2df7b523918146b0f8e61e62613feaab18a46393e96e1
# Copying config sha256:2eac6045a15c8023b0a89aa585850c17ba02bae23185557f6edc0ac57939c316
# Writing manifest to image destination
# Storing signatures
# time="2020-11-15T14:45:15+01:00" level=error msg="Error while applying layer: ApplyLayer exit status 1 stdout:  stderr: there might not be enough IDs available in the namespace (requested 0:15 for /etc/shadow): lchown /etc/shadow: invalid argument"
# Error: error creating build container: Error committing the finished image: error adding layer with blob "sha256:7fce653214d4500372a2df7b523918146b0f8e61e62613feaab18a46393e96e1": ApplyLayer exit status 1 stdout:  stderr: there might not be enough IDs available in the namespace (requested 0:15 for /etc/shadow): lchown /etc/shadow: invalid argument

which I need to understand first or not follow up with at all as it can be just enough to build-test the container within OBS.

EDIT: I understood the error which I could reproduce with perl -e 'qx{podman build -f docker/isotovideo/Dockerfile.qemu . }'. Problem is my alias alias podman='sudo podman' because rootless podman does not work out of the box, at least not in my env. Trying with docker.

Further ideas what I found on the go:

I created https://github.com/os-autoinst/os-autoinst-distri-example/pull/4 with a github actions definition file using a temporary OBS project https://build.opensuse.org/package/show/home:okurz:container/isotovideo-qemu

https://github.com/okurz/os-autoinst-distri-example/runs/1402935643?check_suite_focus=true shows the results when running with qemu_no_kvm=1 as github actions do not provide /dev/kvm, so no nested virt. I could define an example in travis CI in parallel as well.
Changed the Dockerfile to do RUN zypper -n in os-autoinst qemu-x86 qemu-tools instead. Probably I should be explicit and define a "isotovideo:qemu-x86" container and one "isotovideo:qemu-kvm". Have one https://build.opensuse.org/package/show/home:okurz:container/isotovideo-qemu-kvm now and
https://build.opensuse.org/package/show/home:okurz:container/isotovideo-qemu-x86

https://github.com/os-autoinst/os-autoinst-distri-example/pull/4 is ready now with a correct expected successful isotovideo run in
https://github.com/okurz/os-autoinst-distri-example/runs/1403357323?check_suite_focus=true
where "successful" means a completed test run with result "failed" as the example repo has no needles which is left as the training step to complete for beginners using the example repo.

So I could update https://github.com/os-autoinst/os-autoinst/pull/1567

Actions
Copy link
 #3

Updated by okurz over 3 years ago

  • Description updated (diff)
Actions
Copy link
 #4

Updated by okurz over 3 years ago

  • Status changed from In Progress to Feedback

waiting for review

Actions
Copy link
 #5

Updated by okurz over 3 years ago

  • Status changed from Feedback to In Progress

After a reminder I could motivate team colleagues to followup with the pending review :)

https://github.com/os-autoinst/os-autoinst/pull/1567 is merged and also another idea was mentioned and followed up with https://github.com/os-autoinst/os-autoinst/pull/1573 to add specific subpackages.

https://github.com/os-autoinst/os-autoinst-distri-example/pull/4 was merged and https://github.com/os-autoinst/os-autoinst-distri-example/actions/runs/385231860 shows the successful run.

  • Generate the container within OBS, e.g. checkout Dockerfile from VCS

EDIT: Did osc copypac devel:openQA os-autoinst_dev devel:openQA isotovideo-qemu-x86 and created the file _service with content:

<services>
<service name="download_url">
  <param name="protocol">https</param>
  <param name="host">raw.githubusercontent.com</param>
  <param name="path">/os-autoinst/os-autoinst/master/docker/isotovideo/Dockerfile.qemu-x86</param>
  <param name="filename">Dockerfile</param>
</service>
</services>

I realized that this builds against "containers15.2" but I wanted to use Tumbleweed as base. https://build.opensuse.org/projects/devel:openQA/meta had "containers" but referencing openSUSE Leap 15.1. I think we don't need that anymore. I switched openSUSE:Containers:Leap:15.1 to "openSUSE:Containers:Tumbleweed". I also removed:

    <disable repository="containers"/>

and more old cruft, e.g. for SLE12 which mkittler already removed elsewhere. Then I still needed to enable the container build for Tumbleweed in https://build.opensuse.org/package/meta/devel:openQA/isotovideo-qemu-x86

  • Update the Dockerfile to use the new convenience packages instead

Created https://github.com/os-autoinst/os-autoinst/pull/1575 to add the qemu-kvm as well

Includes the updated OBS path

  • Build qemu-kvm as well and document

https://github.com/os-autoinst/os-autoinst/pull/1575

  • Review and consider update of os-autoinst-distri-example

Done with https://github.com/os-autoinst/os-autoinst-distri-example/pull/5

Actions
Copy link
 #6

Updated by okurz over 3 years ago

Actions
Copy link
 #7

Updated by okurz over 3 years ago

  • Status changed from In Progress to Feedback
Actions
Copy link

Also available in: Atom PDF