action #158383
closedopenQA Project - coordination #155485: [saga][epic] Efficient openQA worker pool resource handling in datacenters
coordination #158374: [epic] Prevention of inefficient hardware resource use
Crosscheck which machines marked as "unused" in racktables are still pingable (as they should not be powered on at all) size:M
0%
Description
Acceptance criteria¶
- AC1: A one-time check of all "unused" machines for pingability was conducted
Suggestions¶
- Find examples like doener8.qe.nue2.suse.org https://racktables.nue.suse.com/index.php?page=object&object_id=9228 that has the tag "Unused" as defined on https://wiki.suse.net/index.php/SUSE-Quality_Assurance/Labs
- Machine-scrape https://racktables.nue.suse.com/index.php?andor=and&cft%5B%5D=197&cfe=%7B%24typeid_4%7D+and+not+%7BDecommissioned%7D&page=depot&tab=default&submit.x=9&submit.y=24 to get FQDNs and try to ping them. If pingable then raise an alert, e.g. fail an gitlab CI pipeline
- Optional: Make that a periodic check, e.g. scheduled in gitlab CI pipeline schedule every day, e.g. in https://gitlab.suse.de/openqa/scripts-ci/
- And netbox does not have an entry for "unused" machines
Updated by okurz 29 days ago
- Subject changed from Crosscheck which machines marked as "unused" in racktables are still pingable (as they should not be powered on at all) to Crosscheck which machines marked as "unused" in racktables are still pingable (as they should not be powered on at all) size:S
- Description updated (diff)
- Status changed from New to Workable
Updated by okurz 29 days ago
- Subject changed from Crosscheck which machines marked as "unused" in racktables are still pingable (as they should not be powered on at all) size:S to Crosscheck which machines marked as "unused" in racktables are still pingable (as they should not be powered on at all) size:M
- Description updated (diff)
Updated by openqa_review 26 days ago
- Due date set to 2024-04-20
Setting due date based on mean cycle time of SUSE QE Tools
Updated by okurz 24 days ago
That's how I would do it with python:
import mechanize
br.add_password("https://racktables.nue.suse.com", "okurz", "…")
br.open("https://racktables.nue.suse.com/index.php?andor=and&cft%5B%5D=197&cfe=%7B%24typeid_4%7D+and+not+%7BDecommissioned%7D&page=depot&tab=default&submit.x=9&submit.y=24")
for i in br.links(url_regex="page=object"):
print(i.text)
There are some machines which don't have an FQDN as name, e.g. unreal and quake. In the first step I would ignore those.
Updated by nicksinger 24 days ago
I created https://gitlab.suse.de/openqa/racktables-check and ran it with:
RT_USERNAME=nicksinger RT_PASSWORD=$(pass show Arbeit/idp-portal.suse.com | head -n 1) ./run.sh
styx.qe.nue2.suse.org up
so this is currently the only machine we should shut down. I currently fail to setup a gitlab CI as I have no service account which I could use for racktables but it should be trivial to implement.
Updated by nicksinger 21 days ago
- Status changed from In Progress to Feedback
- Assignee changed from nicksinger to okurz
As discussed in the daily the mentioned machine was shutdown by using the PDU toggle as nobody had access to the running system or BMC.
@okurz if I understood you correctly you wanted to create a follow-up ticket regarding automated pipelines so assigning it to you as reminder.
Updated by okurz 20 days ago
- Copied to action #158907: Automated check for machines marked as "unused" in racktables but still pingable (as they should not be powered on at all) size:M added