QA » openQA Project » openQA Tests » qe-yam

Motivation¶

As starting point we have 429 yaml files in schedule/yast after applying solution based on default without further adjustment. There is plenty of space for improvement and now we are in position to tackle individual test test suites without affecting much or break other areas.

One of the main motivation is to have a test case that can be run across products, sle, sle micro, openSUSE Tumblewee, openSUSE Leap, etc, so first we need to create `schedule/yam/test_cases/ . Inside this folder we will place our yaml that will be valid (in theory) for each product (we have to see what techniques we will use in each case to make this possible and we will document those).

This are all the installation, (we didn't go too far how to apply this for other than installation that contains steps and it fits quite well, so for now the scope are only the interactive installation, not AutoYaST ones):
https://openqa.suse.de/tests/overview?arch=&flavor=&machine=&test=&modules=accept_license&module_re=&distri=sle&version=15-SP5&build=102.1&groupid=129#

Let's go in the order that are displayed for example, first one is USBInstall, for some strange human mistake the test suites are in the following paths:
schedule/yast/sle/usb_install_x86_64.yaml
schedule/yast/sle/usb_install.yaml

When we will find vars, we will check if they are really needed and if we cannot drop them, we will move them to job group yaml configuration.

vars:
  DESKTOP: textmode
  USBBOOT: 1
  YUI_REST_API: 1

The difference between these two test suites is in the following.

  registration:
    - installation/registration/skip_registration
  extension_module_selection:
    - installation/module_selection/skip_module_selection
  add_on_product:
    - installation/add_on_product_installation/accept_add_on_installation
  default_systemd_target:
    - installation/installation_settings/validate_default_target

This differences are not related with what is tested, the test case shouldn't include it. Let's check one by one:

• Test modules skip_module_selection, skip_module_selection, and accept_add_on_installation are default behaviors of the flavor Full, the most simple solution is to create a default file for Full medium.
• For default_systemd_target we shouldn't do this check everywhere as we do now, only makes sense to do it in a few test suite, one textmode, one with gnome, and somewhere else, but not everywhere, we could remove it and file a ticket to apply this in the whole YaST group.

We were lucky with this one, but in another test suite we might prefer code solution, because in this one was a default behaviour, the way to use the Full medium should always be to skip registration and register later after installation, but other cases might have steps unrelated with defaults. Nevertheless for this case there was not gain on file reduction, we have 2 and we will have 2, but there was gain on having a clear test suite to use.

Acceptance criteria¶

AC1: Kick off the reduction of yaml files using the test suite picked above.
AC2: File additional tickets needed to get rid of the differences (very important not forget!)
AC3: Create documentation in Confluence about the strategies used to reduce number of files, like a check list.