QA » openQA Project » openQA Tests

Motivation¶

At some point the setting INSTALLONLY=1 started to be used to avoid the schedule of tests module that tests the installed system.
Since the tests in os-autoinst-distri-opensuse are used by a lot of projects, the schedule in main.pm started to limit the schedule of different.
Some test suites intended to perform an installation, but scheduling a different set of modules for the installed system. The complexity of main.pm made them to look for an easy way of setting the schedule. Using the setting INSTALLONLY=1 to avoid the schedule of common post installation tests, and explicitly adding other modules.
This inconsistency has been taking for a long time. We started to have some problems to maintain and adapt those test.

Acceptance criteria¶

• AC: All test suites that use the setting INSTALLONLY=1 should perform an install only scenario.

Suggestions¶

• A Could we use the feature SCHEDULE of os-autoinst in each test suite to explicitly set the list of modules?
  • Pros: main.pm doesn't need to be used. Schedule is clear to test developer before starting a job.
  • Cons: Schedule cannot be set programmatically.
• B Instead of using INSTALLONLY to avoid the schedule of post installation tests, using a setting POST_INSTALLATION which defines the set of modules to be schedule ["transactional_server", "extra_tests_textmode", ...]
  • Pros: We can adapt the schedule programmatically. We don't need to continually check for special cases in main.pm
  • Cons: To know the schedule, the test developer needs to follow the logic of main.pm.
• C Adapt all existing tests that are abusing the setting INSTALLONLY to don't use that setting.
  • Pros: We can adapt the schedule programmatically.
  • Cons: We need to always check for special cases in main.pm. To know the schedule, the test developer needs to follow the logic of main.pm.
• D Using YML schedule