I wonder if we should file bugs for the failures found in https://openqa.opensuse.org/tests/overview?distri=opensuse&version=Tumbleweed&build=20230802&groupid=38

e.g. https://openqa.opensuse.org/tests/3477310#step/_root_BCI-tests_all_podman/1 looks like a product bug (packaging issue) to me?

The tests are still far away from passing completely. which tox -e environments are you currently running? we can focus on "all" for example to finish first.

I have a really hard time navigating openqa, can you please point me at the failures that you're looking at?

I have found this:

    @pytest.mark.skipif(
        OS_VERSION != "tumbleweed",
        reason="product flavors only available for openSUSE",
    )
    def test_opensuse_product_flavor(auto_container):
        """Checks that this is an appliance-docker flavored product."""
>       auto_container.connection.run_expect(
            [0], "rpm -q --whatprovides 'flavor(appliance-docker)'"
        )
E       AssertionError: Unexpected exit code 127 for CommandResult(command=b"rpm -q --whatprovides 'flavor(appliance-docker)'", exit_status=127, stdout=None, stderr=b'/bin/sh: line 1: rpm: command not found\n')
E       assert 127 in [0]
E        +  where 127 = CommandResult(command=b"rpm -q --whatprovides 'flavor(appliance-docker)'", exit_status=127, stdout=None, stderr=b'/bin/sh: line 1: rpm: command not found\n').rc

which I'm aware of (this test cannot be run for containers that do not include rpm). I'll see what I can do about it. the other thing that is failign are the sizes, they need updates.

I have a draft PR that I need to get back to to solve the remaining test failures.

New failure: https://openqa.opensuse.org/tests/3690008#step/bci_prepare/42

Test died: command 'pip --quiet install --upgrade pip' failed at /usr/lib/os-autoinst/testapi.pm line 926.¶

Thats a bug in the testsuite. it shouldn't install pip from the internet. it should use the pip that the distribution provides.

damn, I have noticed that tests have been broken in TW because of missing dependency.

This has been fixed, but the schedule is broken for some reason :/

https://openqa.opensuse.org/tests/3870280#

s/CONTAINER_RUNTIME/CONTAINER_RUNTIMES/

dirkmueller wrote in #note-8:

Test died: command 'pip --quiet install --upgrade pip' failed at /usr/lib/os-autoinst/testapi.pm line 926.¶

Thats a bug in the testsuite. it shouldn't install pip from the internet. it should use the pip that the distribution provides.

Does this apply to all suse/opensuse hosts or only to TW?

o3:

• http://kepler.suse.cz/tests/22446#step/bci_test_podman/53

use distro python packages -> https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/18457

Hey @dirkmueller, can we close this ticket?

Hey @dirkmueller, can we close this ticket?

What's the current state? as far as I can see the tests are failing (e.g. https://openqa.opensuse.org/tests/3886274#step/bci_test_podman/39 ) because the invocation of the testsuite is incorrect. should we be fixing that? is it actually gating, e.g. a test failure preventing the container from being published?

dirkmueller wrote in #note-15:

Hey @dirkmueller, can we close this ticket?

What's the current state? as far as I can see the tests are failing (e.g. https://openqa.opensuse.org/tests/3886274#step/bci_test_podman/39 ) because the invocation of the testsuite is incorrect. should we be fixing that? is it actually gating, e.g. a test failure preventing the container from being published?

The tests are running just in the TW dev group, thus it is more of a PoC and we should make them stable prior setting them as gating and moving to the main TW group.

Meanwhile, I have fixed the BCI markers for rust and postgres (thanks for the hint):

• https://openqa.opensuse.org/tests/3892476#step/bci_test_podman/38
• https://openqa.opensuse.org/tests/3892477#step/bci_test_podman/38

However, I am not sure what is wrong with these, could you have a look please?

• https://openqa.opensuse.org/tests/3890962#step/bci_test_podman/53
• https://openqa.opensuse.org/tests/3890963#step/bci_test_podman/39
• https://openqa.opensuse.org/tests/3890969#step/bci_test_podman/39

https://openqa.opensuse.org/tests/3900969#details golang is passing after fixing BCI_TEST_ENVS to go

VR for ruby https://openqa.opensuse.org/tests/3941288

So the tests are passing as of now but they are still in the development TW group