QA » openQA Project » openQA Tests

The only concern I have about sharing current PC accounts for openSUSE testing is the exposure of the credentials, which are currently in a private repo in gitlab along with all the OSD workers. If the aim is to share the current accounts for SLE, we would need to involve our managers, as it might impact budget.

jlausuch wrote:

The only concern I have about sharing current PC accounts for openSUSE testing is the exposure of the credentials, which are currently in a private repo in gitlab along with all the OSD workers. If the aim is to share the current accounts for SLE, we would need to involve our managers, as it might impact budget.

Wondering if we could create instances in each cloud provider account and register them as openQA workers. We could leverage the cloud providers RBAC systems to assign our worker the ability to upload images/create instances. For example, in the case of AWS we could create an IAM role that has the requisite permissions and assign it to the worker instance. This would eliminate the need to expose the credentials, however this would increase the cloud spend in those accounts.

jlausuch wrote:

I would say, someone (the community) should come up with a proper planning for this, including budget plan.
Our budgets were planned to test SLE only (which was the original plan).

@jlausuch I think being part of the community can be very beneficial and provide long-term sustainability. As a final product SLE might be of higher importance but please consider SUSE's Open Source policy https://www.suse.com/c/the-suse-open-source-policy/ , in particular "Factory First" which certainly should also apply for testing related work. In the best case you only need to provide the initial "catalyst" for the community to be able to further drive and support relevant work.

okurz wrote:

jlausuch wrote:

I would say, someone (the community) should come up with a proper planning for this, including budget plan.
Our budgets were planned to test SLE only (which was the original plan).

@jlausuch I think being part of the community can be very beneficial and provide long-term sustainability. As a final product SLE might be of higher importance but please consider SUSE's Open Source policy https://www.suse.com/c/the-suse-open-source-policy/ , in particular "Factory First" which certainly should also apply for testing related work. In the best case you only need to provide the initial "catalyst" for the community to be able to further drive and support relevant work.

I'm not sure why you are trying to teach me Factory First policy at this stage, probably you miss a lot of things we are doing for openSUSE, so please don't go this way.
And yes, I am willing to do provide any kind of help and support to the community as I'm doing every day of my work, but without budget plan, there is nothing to do, as Public Cloud testing is not free.

Who is a good contact to speak with about getting the appropriate resources configured/created in each respective cloud provider? If we already have Azure,GCE,AWS accounts, we could configure RBAC properly in each cloud provider for the instances to use and not expose credentials. If someone could provide access to the existing cloud accounts I could come up with a proposal and cost estimate for having either long lived openQA workers living in each provider, or "on demand" instances that are created when test are run.

zfeldstein wrote:

Who is a good contact to speak with about getting the appropriate resources configured/created in each respective cloud provider? If we already have Azure,GCE,AWS accounts, we could configure RBAC properly in each cloud provider for the instances to use and not expose credentials. If someone could provide access to the existing cloud accounts I could come up with a proposal and cost estimate for having either long lived openQA workers living in each provider, or "on demand" instances that are created when test are run.

short answer current tests which we using to test Public Cloud instances in internal openQA instance is not possible to use in openqa.opensuse.org. You can schedule meeting with me and we will discuss details

Jose could help with avoiding leaking the credentials.

lkocman wrote:

Jose could help with avoiding leaking the credentials.

@lkocman: ok. We are in the process of setting up a VM with a service to provide the credentials via URL. This URL is protected with user/password which are secret variables in openQA, which means they are hidden and not visible in the logs, only reduced group of people with access to the worker machines (including myself) will be able to see them as they will be stored in /etc/openqa/workers.ini.

However, the VM that we have setup today is under suse.de and only available for OSD workers. We can do the same for O3, but we would need to have some Host or VM somewhere where O3 workers can reach.

However, the VM that we have setup today is under suse.de and only available for OSD workers. We can do the same for O3, but we would need to have some Host or VM somewhere where O3 workers can reach.
Do we need a VM just to host a bunch of static files?

pdostal wrote:

However, the VM that we have setup today is under suse.de and only available for OSD workers. We can do the same for O3, but we would need to have some Host or VM somewhere where O3 workers can reach.
Do we need a VM just to host a bunch of static files?

They could reside in the same worker that we could use for PC tests.. however, we need to provide them as web service. Maybe a container on the worker should suffice? (thinking loud)

jlausuch wrote:

They could reside in the same worker that we could use for PC tests.. however, we need to provide them as web service. Maybe a container on the worker should suffice? (thinking loud)

We have https://static.opensuse.org/ (https://github.com/openSUSE/static.opensuse.org) - if we don't plan on DoS'ing that host with gigabytes of transfers, that might be a good fit? (Not sure I got the requirements straight .. sounded like some static stuff being served)

dimstar wrote:

jlausuch wrote:

They could reside in the same worker that we could use for PC tests.. however, we need to provide them as web service. Maybe a container on the worker should suffice? (thinking loud)

We have https://static.opensuse.org/ (https://github.com/openSUSE/static.opensuse.org) - if we don't plan on DoS'ing that host with gigabytes of transfers, that might be a good fit? (Not sure I got the requirements straight .. sounded like some static stuff being served)

as we speaking here about very confidential data here I would not mix it with something which has totally different purpose on my understanding . Also again due to same reason I really prefer to keep it hidden in same network as openQA workers are

jlausuch wrote:

lkocman wrote:

Jose could help with avoiding leaking the credentials.

@lkocman: ok. We are in the process of setting up a VM with a service to provide the credentials via URL. This URL is protected with user/password which are secret variables in openQA, which means they are hidden and not visible in the logs, only reduced group of people with access to the worker machines (including myself) will be able to see them as they will be stored in /etc/openqa/workers.ini.

However, the VM that we have setup today is under suse.de and only available for OSD workers. We can do the same for O3, but we would need to have some Host or VM somewhere where O3 workers can reach.

AFAIK O3 workers are running in isolated network , it would be perfect if we could install web server on any machine running in this subnet it could be apache/nginx or anything else which capable to expose directory structure via HTTPS and protect it with password.

dimstar wrote:

jlausuch wrote:

They could reside in the same worker that we could use for PC tests.. however, we need to provide them as web service. Maybe a container on the worker should suffice? (thinking loud)

We have https://static.opensuse.org/ (https://github.com/openSUSE/static.opensuse.org) - if we don't plan on DoS'ing that host with gigabytes of transfers, that might be a good fit? (Not sure I got the requirements straight .. sounded like some static stuff being served)

as we speaking here about very confidential data here I would not mix it with something which has totally different purpose on my understanding . Also again due to same reason I really prefer to keep it hidden in same network as openQA workers are
+1

Let's just use openqaworker1 for this purpose. Podman is installed there and I we just need to add the _SECRET_ vars to /etc/openqa/workers.ini and run a container with the credentials web service...
The jobs will just need WORKER_CLASS=openqaworker1.

jlausuch wrote:

Let's just use openqaworker1 for this purpose. Podman is installed there and I we just need to add the _SECRET_ vars to /etc/openqa/workers.ini and run a container with the credentials web service...
The jobs will just need WORKER_CLASS=openqaworker1.

+1

asmorodskyi wrote:

AFAIK O3 workers are running in isolated network , it would be perfect if we could install web server on any machine running in this subnet it could be apache/nginx or anything else which capable to expose directory structure via HTTPS and protect it with password.

There is already a generic webserver within the o3 network, apache on o3 itself. And it serves the assets from /var/lib/openqa so just put something there which can even be downloaded from elsewhere with the help of openQA features and then where necessary adapt the openQA config.

A container on openqaworker1 is also fine but no permanent solution as we can't guarantee for any worker to stay up indefinitely.

okurz wrote:

There is already a generic webserver within the o3 network, apache on o3 itself. And it serves the assets from /var/lib/openqa so just put something there which can even be downloaded from elsewhere with the help of openQA features and then where necessary adapt the openQA config.

Is this webserver accessible from any public place, or only O3 workers can access to it?

May I suggest to define some sync call to find a common ground regarding all this ? I think it would be more effective way to find solution which suits everyone

jlausuch wrote:

okurz wrote:

There is already a generic webserver within the o3 network, apache on o3 itself. And it serves the assets from /var/lib/openqa so just put something there which can even be downloaded from elsewhere with the help of openQA features and then where necessary adapt the openQA config.

Is this webserver accessible from any public place, or only O3 workers can access to it?

The webserver is generally available from outside but parts are restricted to the internal network. Look into o3:/etc/apache2/vhosts.d/openqa.conf for some examples already doing that.

okurz wrote:

Is this webserver accessible from any public place, or only O3 workers can access to it?

The webserver is generally available from outside but parts are restricted to the internal network. Look into o3:/etc/apache2/vhosts.d/openqa.conf for some examples already doing that.

Ok, sounds interesting for our use case. Not sure if we still should isolate things in containers, but let's discuss it further.

asmorodskyi wrote:

May I suggest to define some sync call to find a common ground regarding all this ? I think it would be more effective way to find solution which suits everyone

+1
@lkocman can you arrange some meeting to discuss this? Attendees: Anton, Oliver, myself and whoever you consider (maybe Pavel and Dimstar also wanna join?).

Talk is scheduled for tomorrow at 14:30 CEST (teams).

Meeting minutes: https://etherpad.opensuse.org/p/ReleaseEngineering-20220616

Infra ticket created: https://sd.suse.com/servicedesk/customer/portal/1/SD-90058

This ticket was set to Normal priority but was not updated within the SLO period. Please consider picking up this ticket or just set the ticket to the next lower priority.

slo-gin wrote:

This ticket was set to Normal priority but was not updated within the SLO period. Please consider picking up this ticket or just set the ticket to the next lower priority.

To correct: Actually the problem is that the due date is exceeded

This ticket was set to Normal priority but was not updated within the SLO period. Please consider picking up this ticket or just set the ticket to the next lower priority.

Hello team,

what's the current status. Are we waiting for infra, qe or Zack?

Thank you

Related to https://progress.opensuse.org/issues/154501 can we link these please?