action #157468
closedopenQA Project - coordination #105624: [saga][epic] Reconsider how openQA handles secrets
openQA Project - coordination #157537: [epic] Secure setup of openQA test machines with secure network+secure authentication
Handle internal test machines with compromised root password size:M
0%
Description
Motivation¶
In https://sd.suse.com/servicedesk/customer/portal/1/SD-150437 we are asked to handle "compromised root passwords in QA segments" including s390zl11…16
Acceptance criteria¶
- AC1: All steps asked in https://sd.suse.com/servicedesk/customer/portal/1/SD-150437 have been sufficiently handled
Suggestions¶
- DONE Change root password for s390zl11…16 and in sync update in https://gitlab.suse.de/openqa/salt-pillars-openqa/-/blob/master/openqa/workerconf.sls
- DONE Ensure according tests work
- DONE Ask wegao about their personal machine also referenced
- DONE Find a working solution covering s390kvm080…099 -> see related tickets
- Depending on response in https://sd.suse.com/servicedesk/customer/portal/1/SD-150437 either resolve this ticket or block on sibling tasks, e.g. new password for s390x openQA tests, rotating password, ssh key based authentication and/or more secured network
Updated by okurz about 1 month ago
Replacing compromised insecure testing password for unreal6, win-jump, vmware-jump, jumphost, openqaw5-xen, s390zl12…17. I created and merged https://gitlab.suse.de/openqa/salt-pillars-openqa/-/merge_requests/745 and in parallel (as quickly as possible) I changed the root password to the new one.
Updated by okurz about 1 month ago
- Due date set to 2024-04-01
- Status changed from In Progress to Feedback
https://suse.slack.com/archives/C02CANHLANP/p1710766640218969
@here based on request by the Cyber-Security group I changed the root password of the following machines: unreal6, win-jump, vmware-jump, jumphost, openqaw5-xen, s390zl12, s390zl13, s390zl15, s390zl16, s390zl17. I changed the password on the host and accordingly in https://gitlab.suse.de/openqa/salt-pillars-openqa/-/merge_requests/745 . The new password is the same we use on other hypervisor hosts. The password "nots3cr3t" should only be used on transient openQA based installations, not any other part of our testing infrastructure. See https://progress.opensuse.org/issues/157468 for more details
Updated by okurz about 1 month ago
- Tags changed from infra, password, security to infra, password, security, reactive work
Updated by okurz about 1 month ago
Multiple ideas we should follow:
- be able to set a different password valid for tests, in particular s390kvm…, e.g. be able to set password by test variable and follow through in the complete test platform
- key based authentication
- rotating, automatic passwords saved as test variables connected to images, e.g. to be able to use a pre-installed image
- better secure the networks to have s390kvm… (and others) less accessible -> We have stated the requirement in https://confluence.suse.com/pages/viewpage.action?pageId=1006108843 that ssh 22/tcp needs to be reachable. We could try to replicate the setup we know from o3 to give OSD a second network interface which allows ssh 22/tcp and block ssh 22/tcp on .oqa.prg2.suse.org as usually we don't need ssh to workers, just from within the oqa network as well as for administrative purposes for which we could go over OSD which we also already normally do for salt.
Updated by okurz about 1 month ago
More subtasks planned in #157537. I now need to reprioritize tasks accordingly to put at least some of those tasks on our backlog.
Updated by okurz about 1 month ago
- Status changed from In Progress to Feedback
- Target version changed from Ready to Tools - Next
I wrote in https://sd.suse.com/servicedesk/customer/portal/1/SD-150437
Moreno Daltin, we have secured the machines
10.145.0.3 Contact: wegao@suse.com QE-QA (source:racktables)
10.145.10.42 Contact: ? QE-OpenQA (source:racktables)
10.145.10.43 Contact: ? QE-OpenQA (source:racktables)
10.145.10.44 Contact: ? QE-OpenQA (source:racktables)
10.145.10.46 Contact: ? QE-OpenQA (source:racktables)
for the remaining s390kvm0{80..099} I have now planned the according improvement tasks with the team to use 1. different passwords, 2. ssh key authentication, 3. a more secure network setup. You stated “So, it's not P1 but please let's not this linger forver”. My current estimate would be that we can cover at least one of 1.-3. within the next months. Is that sufficient? If not I will need to rediscuss priorities of other tasks with our stakeholders to prioritize security over their feature requests.
Updated by okurz about 1 month ago
- Subject changed from Handle internal test machines with compromised root password to Handle internal test machines with compromised root password size:M
Updated by okurz 24 days ago
- Due date deleted (
2024-04-01) - Status changed from Feedback to Resolved
- Target version changed from Tools - Next to Ready
There was no reaction in https://sd.suse.com/servicedesk/customer/portal/1/SD-150437 so I assume we are ok to continue in the related tickets according to plan but not need to finish within days.