QA » openQA Project

Simplest thing we can try is just to trigger isotovideo to evaluate some modules and see if it fails with some compilation error. We need to check if we can include many modules to have this covered.

I've started openQA tests with the minimal vars.json file and isotovideo.

{
   "ARCH" : "x86_64",
   "ASSETDIR" : "/var/lib/openqa/share/factory",
   "BACKEND" : "qemu",
   "CASEDIR" : "/path/to/os-autoinst-distri-opensuse",
   "DESKTOP" : "gnome",
   "DISTRI" : "sle",
   "FLAVOR" : "Server-DVD",
   "ISO" : "/var/lib/openqa/factory/iso/SLE-12-SP5-Server-DVD-x86_64-Build0201-Media1.iso",
   "JOBTOKEN" : "hUpWvBMnjC",
   "NEEDLES_DIR" : "/path/to/os-autoinst-distri-opensuse/products/sle/needles",
   "PRJDIR" : "/path/to/os-autoinst-distri-opensuse",
   "PRODUCTDIR" : "/path/to/os-autoinst-distri-opensuse/products/sle",
   "QEMUPORT" : 15222,
   "TEST" : "test",
   "VERSION" : "12-SP5",
   "VNC" : "91",
   "_EXIT_AFTER_SCHEDULE" : 1
}

It doesn't give any valuable advantages. For example, if remove imports (e.g. remove use registration; from bootloader.pm) and run isotovideo, the will be no compilation or runtime errors.

The other problem is that running isotovideo with "_EXIT_AFTER_SCHEDULE" : 1 exits with "1" in console, which means negative result of execution, though it was executed correctly and without errors. So, it does not allow to rely on the result of execution.

After discussing with @riafarov it was decided to close the ticket as it cannot be resolved with the initial plan.

We need to re-think what we need to test and how it should be done.

due to changes in a related task

In light of the recent epic #58184 we brought up within the tools team I will give this another try.

https://travis-ci.org/okurz/os-autoinst-distri-opensuse/builds/613699612 shows the latest travis CI tests. What I tried in https://github.com/os-autoinst/os-autoinst-distri-opensuse/compare/master...okurz:feature/isotovideo_ci is to start isotovideo evaluating the schedule and comparing that against a reference but so far isotovideo failed to startup due to missing dependencies as visible in https://travis-ci.org/okurz/os-autoinst-distri-opensuse/jobs/613699616#L18288 for IPC::System::Simple . https://travis-ci.org/okurz/os-autoinst-distri-opensuse/jobs/613699613 should show all cpan dependencies. Tried to patch this but so far with no success.

https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/13462

PR fully tested and ready. After merge we just need to update the URL for the Dockerfile from which the specified container is built

the container image path was updated. After 2 weeks with no feedback I now merged the PR myself. With this being part of a test that runs for every pull request I would like to monitor the impact over the next days.

Everything looks good so far. Some example pull request tests:

• https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/13648 showing all good with a passed job https://github.com/os-autoinst/os-autoinst-distri-opensuse/runs/4152111008?check_suite_focus=true
• https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/13645 with currently broken changes that rightfully cause a test failure https://github.com/os-autoinst/os-autoinst-distri-opensuse/runs/4152798500?check_suite_focus=true