QA » openQA Project » openQA Tests » qe-yam

One comment, Please don't change leap to sle migration jobs.
We still need use 'test suites' to update 'HDD_1_URL' setting to avoid open more MRs.

Step 1: Look what testsuites from openQA are still used in the SLE* jobgroups

openqa-job-groups/JobGroups> grep -r testsuite SLE* | grep -v null | sort | uniq
SLE_15_DEV/aarch64.yaml:        testsuite: yast2_cmd
SLE_15_DEV/ppc64le.yaml:        testsuite: offline_install+skip_registration
SLE_15_DEV/ppc64le.yaml:        testsuite: transactional_server_helper_apps
SLE_15_DEV/ppc64le.yaml:        testsuite: yast2_nfs_v3_client
SLE_15_DEV/ppc64le.yaml:        testsuite: yast2_nfs_v3_server
SLE_15_DEV/ppc64le.yaml:        testsuite: yast2_nfs_v4_client
SLE_15_DEV/ppc64le.yaml:        testsuite: yast2_nfs_v4_server
SLE_15_DEV/x86_64.yaml:        testsuite: lvm+RAID1
SLE_15_DEV/x86_64.yaml:        testsuite: msdos
SLE_15_DEV/x86_64.yaml:        testsuite: yast2_cmd
SLE_15/x86_64.yaml:        testsuite: gnome_install_from_source
SLE_15/x86_64.yaml:        testsuite: installer_extended

So many references in SLE_15_DEV and a few in SLE_15 job group.

rainerkoenig wrote in #note-5:

Step 1: Look what testsuites from openQA are still used in the SLE* jobgroups

openqa-job-groups/JobGroups> grep -r testsuite SLE* | grep -v null | sort | uniq
SLE_15_DEV/aarch64.yaml:        testsuite: yast2_cmd
SLE_15_DEV/ppc64le.yaml:        testsuite: offline_install+skip_registration
SLE_15_DEV/ppc64le.yaml:        testsuite: transactional_server_helper_apps
SLE_15_DEV/ppc64le.yaml:        testsuite: yast2_nfs_v3_client
SLE_15_DEV/ppc64le.yaml:        testsuite: yast2_nfs_v3_server
SLE_15_DEV/ppc64le.yaml:        testsuite: yast2_nfs_v4_client
SLE_15_DEV/ppc64le.yaml:        testsuite: yast2_nfs_v4_server
SLE_15_DEV/x86_64.yaml:        testsuite: lvm+RAID1
SLE_15_DEV/x86_64.yaml:        testsuite: msdos
SLE_15_DEV/x86_64.yaml:        testsuite: yast2_cmd
SLE_15/x86_64.yaml:        testsuite: gnome_install_from_source
SLE_15/x86_64.yaml:        testsuite: installer_extended

So many references in SLE_15_DEV and a few in SLE_15 job group.

test suites from db do not need to have 'testsuite', ie this one so that filter might not give you all that you need I think.

Exactly: I just wrote this comment at the same time when you were posting your comment, so we had a "comment collison".

Unfortunately comment#5 was not the full picture. It just selects test suites where we have a line with testsuite: that is not null.

BUT there is much more in this:

    - textmode:
        settings:
          YAML_SCHEDULE: schedule/yast/textmode/textmode_aarch64.yaml
          YAML_TEST_DATA: test_data/yast/installation/network_config.yaml
          YAML_SCHEDULE_DEFAULT: schedule/yast/sle/flows/default_aarch64.yaml
    - transactional_server_helper_apps:

The textmode definition in our job group has no testsuite: line at all, but this means, that it will use the settings from what is stored in openQA in the test suite textmode (which by chance you find on page 105 if you search for "textmode" in the test suites search field).
And our job group setings have a lot of those implicit usage of stored test suites. This will make the effort for this ticket explode.

I don't think so, it is easy to spot them visually I think...but not 100 % sure, I like to start visually and sometimes I don't need any tool or script, but other people seems to prefer the scripting first, which could lead you a trap :) Let me know if I can help somehow.

The easy way to spot the tests that need adaption is to compile the jobgroup YAML file. Unfortunately we have that testsuite: null in many occasions in test_suites.yaml, so even if you look at the x86_64.yaml and think you found one the chances are high that testsuite: null gets imported from test_suites.yaml. But looking at the end result you clearly see, which testsuites have no testsuite reference at all.

MR is ready: https://gitlab.suse.de/qe-yam/openqa-job-groups/-/merge_requests/137

Checked that in the compiled job group definitions for SLE_15 and SLE_15_DEV all testsuites have a line testsuite: null in them.
Descriptions are mostly copied, and sometimes adapted to the new repo where the test suite is maintained now.