QA » openQA Project » openQA Tests » qe-yam

# Running Playwright tests in openQA

**Note:** At the time of writing (2023-06-29) this we're still experementing with an Agama Live-CD, so things might change over time.

# Building blocks

## e2e-agama-playwright repo on GitHub

The [e2e-agama-playwright repo](https://github.com/jknphy/e2e-agama-playwright) was created by Joaquin and is supposed to be the repository that provides the Playwright tests. 

## yupdate

This is a script provided with the Agama Live-CD. The main purpose (at least for the Yam squad) is to provide a method of updating the playwright code in the Live-CD which is located under `/usr/share/agama-playwright`. For this use case we use `yupdate patch <github_repo> <branch>` which uses the branch on GitHub to update the Playwright and agama configuration.

## Integration of yupdate into openQA

[PR 17228](https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/17228/) implements test modules to call `yupdate` from inside openQA. 

The test module is stored  under `yam/agama/patch_agama.pm` and uses a setting `YUPDATE_GIT` which is describred as follow:

~~~

YUPDATE_GIT | string | | Github link used by yast help script yupdate, format is repo#branch such as yast/agama#main.

~~~

## Test module agama_installation

There is a test module `yam/agama/agama-installation` that has the purpose to run the installation. For that it uses the following code:

~~~

 assert_script_run('RUN_INSTALLATION=1 playwright test --trace on --project chromium --config /usr/share/agama-playwright take_screenshots', timeout => 600);

~~~

The `take_screenshots` test for Playwright has some specialties that we need to know:

~~~

    // check for multiple texts in parallel, avoid waiting for timeouts

    let action = await Promise.any([

      page.getByText("Product selection").waitFor().then(() => actions.setProduct),

      page.getByText("None authentication method").waitFor().then(() => actions.setPassword),

      page.getByText("Root authentication set").waitFor().then(() => actions.done),

    ]);

    // optional product selection

    if (action === actions.setProduct) {

      await test.step("Select the product", async () => {

        // select openSUSE Tumbleweed

        await page.getByText("openSUSE Tumbleweed").click();

        await page.screenshot({ path: "screenshots/product-selection.png" });

        await page.getByRole("button", { name: "Select" }).click();

      });

~~~

This means the test is able to determine if we're still on the product selection page and then select Tumbleweed.

~~~

    if (process.env.RUN_INSTALLATION === "1") {

      await test.step("Run installation",  async () => {

        // start the installation

        await page.getByRole("button", { name: "Install", exact: true }).click();

        await expect(page.getByText("Confirm Installation")).toBeVisible();

        await page.getByRole("button", { name: "Continue" }).click();

~~~

Here we check if we run the test with `RUN_INSTALLATION=1` (as our openQA test module `agama_installation` does, so then we launch the installation by making Playwright click on the install button.

**Keep that in mind if your adapted installation test module will call another Playwright test!**

# Experiments made so far

## Clean tumbleweed installation

[Test 3267958](https://openqa.opensuse.org/tests/3267958) on O3 executes a clean Tumbleweed installation using what is in the `e2e-agama-playwright` repo without updating it. The test run is still `take screenshots`.

## Install tumbleweed after yupdate

[Test 3374624](https://openqa.opensuse.org/tests/3374624) attempts a Tumbleweed installation, but Playwright reports the error, that there is no install button and the `click()` method runs into a timeout. The test makes use of `yupdate` by using the `patch_agama` test module, but in fact there shouldn't be updates since `YUPDATE_GIT` is still set to `jknphy/e2e-agama-playwright#main`. ** This needs to be investigated further**, differences to the above clean installation that worked:

* we now use the Agama-Live CD from staging, version "agama-2.1-staging". 

## Install tumbleweed with LVM settings

[Test 3370827](https://openqa.opensuse.org/tests/3370827) by Lemon does a Tumbleweed installation with a specific `lvm` Playwright test module.

For this the `agama_installation` test module got tweaked and uses a direct run of `yupdate` instead of using the `patch_agama` test module in the YAML schedule:

~~~

assert_script_run("yupdate patch lemon-suse/e2e-agama-playwright add-lvm-installation", timeout => 300);

~~~

# Open questions so far

## Page Object Model

Both Lemon and Rainer have made Playwright tests with POM. Lemon places the `pages` directory inside the root of his repo, Rainer has stored it under `tests/`. Which place shall we use to have a common understanding?

## What is the long term approach?

On the experiments above we practically launch exactly **one** Playwright test to do an installation. On the other hand wie have tickets that say "Implement XYZ using Page Object Model". So in the long run we should have pages and tests that e.g configure LVM or setup disk encrpytion, but we are lacking of a sort of "Playwright scheduler" that could make use of this in a similar way as we now use the diffeent pages to configure settings from the YaST installation overview when we run install tests in openQA. Shall we provide individual 'agama_installation_xzy' test modules where 'xyz' stands for a combination of different options or shall we think of a method to e.g. submit a YAML file that tells the "Playwright scheduler" to execute the tests in a defined order and then do the installation?