action #132338
closedEnable BCI-tests for openSUSE Tumbleweed
Added by dirkmueller about 1 year ago. Updated 7 months ago.
0%
Description
With https://github.com/SUSE/BCI-tests/pull/267 being merged the BCI-testing can now be run on Tumbleweed by using OS_VERSION=latest
I have started to submit some containers to openSUSE:Factory and it would be great to have some form of testing for them. I could setup periodic testing on github but it might be better to have openqa added to the container publishing pipeline for tumbleweed?
Files
screenshot-2023-07-19-12_36_56.jpg (44.7 KB) screenshot-2023-07-19-12_36_56.jpg | current job status | mloviska, 2023-07-19 10:38 |
Updated by jlausuch about 1 year ago
- Tags set to bci
- Status changed from New to Workable
Updated by mloviska about 1 year ago
- Status changed from Workable to In Progress
- Assignee set to mloviska
Updated by mloviska about 1 year ago
- Status changed from In Progress to Feedback
Updated by mloviska about 1 year ago
A subset of BCI tests can be found in TW's development group e.g
Updated by ph03nix about 1 year ago
I wonder if we should file bugs for the failures found in https://openqa.opensuse.org/tests/overview?distri=opensuse&version=Tumbleweed&build=20230802&groupid=38
e.g. https://openqa.opensuse.org/tests/3477310#step/_root_BCI-tests_all_podman/1 looks like a product bug (packaging issue) to me?
Updated by dirkmueller about 1 year ago
The tests are still far away from passing completely. which tox -e environments are you currently running? we can focus on "all" for example to finish first.
I have a really hard time navigating openqa, can you please point me at the failures that you're looking at?
I have found this:
@pytest.mark.skipif(
OS_VERSION != "tumbleweed",
reason="product flavors only available for openSUSE",
)
def test_opensuse_product_flavor(auto_container):
"""Checks that this is an appliance-docker flavored product."""
> auto_container.connection.run_expect(
[0], "rpm -q --whatprovides 'flavor(appliance-docker)'"
)
E AssertionError: Unexpected exit code 127 for CommandResult(command=b"rpm -q --whatprovides 'flavor(appliance-docker)'", exit_status=127, stdout=None, stderr=b'/bin/sh: line 1: rpm: command not found\n')
E assert 127 in [0]
E + where 127 = CommandResult(command=b"rpm -q --whatprovides 'flavor(appliance-docker)'", exit_status=127, stdout=None, stderr=b'/bin/sh: line 1: rpm: command not found\n').rc
which I'm aware of (this test cannot be run for containers that do not include rpm). I'll see what I can do about it. the other thing that is failign are the sizes, they need updates.
I have a draft PR that I need to get back to to solve the remaining test failures.
Updated by dirkmueller 10 months ago
Test died: command 'pip --quiet install --upgrade pip' failed at /usr/lib/os-autoinst/testapi.pm line 926.¶
Thats a bug in the testsuite. it shouldn't install pip from the internet. it should use the pip that the distribution provides.
Updated by mloviska 9 months ago
dirkmueller wrote in #note-8:
Test died: command 'pip --quiet install --upgrade pip' failed at /usr/lib/os-autoinst/testapi.pm line 926.¶
Thats a bug in the testsuite. it shouldn't install pip from the internet. it should use the pip that the distribution provides.
Does this apply to all suse/opensuse hosts or only to TW?
Updated by mloviska 9 months ago
use distro python packages -> https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/18457
Updated by dirkmueller 8 months ago
Hey @dirkmueller, can we close this ticket?
What's the current state? as far as I can see the tests are failing (e.g. https://openqa.opensuse.org/tests/3886274#step/bci_test_podman/39 ) because the invocation of the testsuite is incorrect. should we be fixing that? is it actually gating, e.g. a test failure preventing the container from being published?
Updated by mloviska 8 months ago
dirkmueller wrote in #note-15:
Hey @dirkmueller, can we close this ticket?
What's the current state? as far as I can see the tests are failing (e.g. https://openqa.opensuse.org/tests/3886274#step/bci_test_podman/39 ) because the invocation of the testsuite is incorrect. should we be fixing that? is it actually gating, e.g. a test failure preventing the container from being published?
The tests are running just in the TW dev group, thus it is more of a PoC and we should make them stable prior setting them as gating and moving to the main TW group.
Meanwhile, I have fixed the BCI markers for rust and postgres (thanks for the hint):
- https://openqa.opensuse.org/tests/3892476#step/bci_test_podman/38
- https://openqa.opensuse.org/tests/3892477#step/bci_test_podman/38
However, I am not sure what is wrong with these, could you have a look please?
Updated by mloviska 8 months ago
https://openqa.opensuse.org/tests/3900969#details golang is passing after fixing BCI_TEST_ENVS
to go
Updated by mloviska 7 months ago
VR for ruby https://openqa.opensuse.org/tests/3941288
So the tests are passing as of now but they are still in the development TW group
Updated by mloviska 4 months ago
- Related to action #161741: Enable BCI tests for Factory (Tumbleweed containers) in openqa.opensuse.org added