QA » openQA Project

firewall complain comes from https://github.com/os-autoinst/openQA/blob/master/contrib/ay-openqa-worker.xml.erb#L101

<FW_CONFIGURATIONS_EXT>sshd</FW_CONFIGURATIONS_EXT>

failure to run the scripts is due to run-scripts=true in the check-profile and makes sense I think locally. But I guess it shouldnt happen in the worker, right?

Awesome info for firewall migration https://yast.opensuse.org/blog/2018/02/09/highlights-of-yast-development-sprint-50/#firewalld-enhancements

https://github.com/os-autoinst/openQA/pull/5839 addressing the firewall warning

However the automation and validation of erb is kinda tricky and with some challenges.
I talk to ancorgs who told me that it's more like evaluating a C or C++ code than a static XML
and

validation of ERB will be always tricky, e.g. if there is an "if" condition then you will always validate only one branch, you would need to run it once more with settings or environment where it evaluates the second branch

So far I am struggling with find the proper container. Different container different issue

• registry.opensuse.org/devel/openqa/ci/containers/base:latest is not multi-user system
• registry.opensuse.org/devel/openqa/containers/tumbleweed:client
• registry.opensuse.org/opensuse/tumbleweed:latest is hanging
• registry.opensuse.org/opensuse/leap:latest Target multi-user not found
• registry.opensuse.org/yast/sle-15/sp6/images/opensuse/leap:15.6 Target multi-user not found

For testing based on a virtual machine installed by the autoyast profile I can also recommend to try https://github.com/okurz/scripts/blob/master/vm

Just to be sure we are on the same page. AC1 says Automatic CI tests. The suggestions include an openQA test or a workflow on GitHub. A one-off is not enough for this ticket.

Existing tests work by setting AUTOYAST, for instance https://openqa.opensuse.org/tests/4396756 - see lib/autoyast.pm for the implementation details. Maybe we can simply add a new test based on that?

livdywan wrote in #note-15:

Just to be sure we are on the same page. AC1 says Automatic CI tests. The suggestions include an openQA test or a workflow on GitHub. A one-off is not enough for this ticket.

Existing tests work by setting AUTOYAST, for instance https://openqa.opensuse.org/tests/4396756 - see lib/autoyast.pm for the implementation details. Maybe we can simply add a new test based on that?

As I see I have to reinvent the whole autoyast.pm right? What do you mean by implementation details?
If we follow the openqa implementation this is not anymore an easy ticket. I think also it is my first time touching this repo.

ybonatakis wrote in #note-16:

livdywan wrote in #note-15:

Just to be sure we are on the same page. AC1 says Automatic CI tests. The suggestions include an openQA test or a workflow on GitHub. A one-off is not enough for this ticket.

A one-off is not enough but could help to understand what's necessary. And while it's suggested to use an "openQA test" that does not mean that a full autoyast based installation should be done.

Existing tests work by setting AUTOYAST, for instance https://openqa.opensuse.org/tests/4396756 - see lib/autoyast.pm for the implementation details. Maybe we can simply add a new test based on that?

As I see I have to reinvent the whole autoyast.pm right? What do you mean by implementation details?

I think having an automatic installation test based on the autoyast profile would be helpful but likely it would also take quite some time to conduct so is likely not feasible to include in CI tests running on every PR. That would be something for "post-master" merge. But please don't consider reinventing osado tests.

I suggest to do the automatic validation within an openQA tests, so e.g. install yast and call yast2 autoyast check-profile …. Do not conduct a full installation within an openQA test. Only do a manual verification by installing in a virtual machine once.

If we follow the openqa implementation this is not anymore an easy ticket.

Nobody ever said it would easy, right? :)

I created https://github.com/os-autoinst/os-autoinst-distri-openQA/pull/198 but it is not ready.
Locally the test fails to match the screen, even I have created from the UI. But I think because of the git url ref to the needles, the needle cannot be found.

https://github.com/os-autoinst/os-autoinst-needles-openQA/pull/25 needles added
https://github.com/os-autoinst/os-autoinst-distri-openQA/pull/198 is also updated
and I run manual installation

ybonatakis wrote in #note-23:

installation at least finished successfully

I dont know creds to login tho

Did you set a password then? See
https://progress.opensuse.org/projects/openqav3/wiki/#Network-legacy-boot-via-PXE-and-OSworker-setup
which mentions the console parameter for installation rootpassword=<password_to_login_as_root>

ybonatakis wrote in #note-22:

https://github.com/os-autoinst/os-autoinst-needles-openQA/pull/25 needles added
https://github.com/os-autoinst/os-autoinst-distri-openQA/pull/198 is also updated

As discussed please try and provide a verification run on o3 that uses the new needles

okurz wrote in #note-24:

ybonatakis wrote in #note-23:

installation at least finished successfully

...

I dont know creds to login tho

Did you set a password then? See
https://progress.opensuse.org/projects/openqav3/wiki/#Network-legacy-boot-via-PXE-and-OSworker-setup
which mentions the console parameter for installation rootpassword=<password_to_login_as_root>

I reinstalled the system with the system boots. Some things which are in inconsistency with the autoyast profile

• firewalld is not installed and as such not configured
• ip_forwarding doesnt seem enabled

So I can add the firewalld in the template but I would like a confirmation first

https://github.com/os-autoinst/os-autoinst-distri-openQA/pull/198 merged. Open question:

Should we also schedule this test regularly? Or do you want to do this in a followup PR?

We definitely need this to run regularly to fulfill AC1.

livdywan wrote in #note-27:

https://github.com/os-autoinst/os-autoinst-distri-openQA/pull/198 merged. Open question:

Should we also schedule this test regularly? Or do you want to do this in a followup PR?

We definitely need this to run regularly to fulfill AC1.

I did this already, right? https://progress.opensuse.org/issues/156169#note-26

Validation run with the fixed profile https://openqa.opensuse.org/tests/4415391 from https://github.com/os-autoinst/openQA/pull/5839#pullrequestreview-2260181851

• One more review needed
• Schedule to be added

livdywan wrote in #note-29:

Validation run with the fixed profile https://openqa.opensuse.org/tests/4415391 from https://github.com/os-autoinst/openQA/pull/5839#pullrequestreview-2260181851

• One more review needed
• Schedule to be added

https://github.com/os-autoinst/os-autoinst-distri-openQA/pull/202

ybonatakis wrote in #note-31:

job run successful in O3

https://openqa.opensuse.org/tests/4436591