Project

General

Profile

action #120279

Proper maintainership for openqa_service VM size:M

Added by okurz 3 months ago. Updated about 1 month ago.

Status:
Workable
Priority:
Normal
Assignee:
-
Target version:
Start date:
2022-11-10
Due date:
% Done:

0%

Estimated time:

Description

Motivation

The VM openqa_service runs the openqa_bugfetcher.

We need to talk about maintainership of the openqa_service VM, e.g. updates, upgrades, backup, config management, monitoring, etc.

Possibly we can just add the VM to salt as we do for other QE machines already, see parent

Acceptance criteria

  • AC1: We know who maintains the machine
  • AC2: The machine is updated+upgraded
  • AC3: There exists backup and/or config management
  • AC4: The machine is properly monitored

Suggestions

  • Clarify with Eng-Infra about maintainership of the machine including updates, upgrades, backup, config management, monitoring, etc.
  • Ask someone with access to the system to provide access, e.g. dheidler should add ssh keys from openqa-salt-pillars for SUSE QE Tools members
  • Check the state of the system, e.g. OS level, etc.
  • If we need to maintain it maybe it's easier to just transform into a CI pipeline, e.g. on gitlab.suse.de, and then decommission the VM

Related issues

Related to openQA Infrastructure - action #120874: openqa_service VM is down size:MResolved2022-11-10

Copied from openQA Infrastructure - action #120276: Prepare openqa_bugfetcher for upcoming bugzilla updateBlocked2022-11-10

History

#1 Updated by okurz 3 months ago

  • Copied from action #120276: Prepare openqa_bugfetcher for upcoming bugzilla update added

#2 Updated by okurz 3 months ago

  • Description updated (diff)
  • Parent task set to #118636

#3 Updated by okurz 3 months ago

Apparently the machine is not reachable over IPv6 but a AAAA record exists. ssh -4 root@openqa-service.suse.de establishes a connection but I can't login with usual passwords.

dheidler if you have access you could add the ssh keys from our team from https://gitlab.suse.de/openqa/salt-pillars-openqa/-/blob/master/sshd/users.sls

The machine seems to be salt controlled by Eng-Infra so the question is about who conducts upgrades, how is monitoring done, etc.

#4 Updated by cdywan 3 months ago

  • Subject changed from Proper maintainership for openqa_service VM to Proper maintainership for openqa_service VM size:M
  • Description updated (diff)
  • Status changed from New to Workable

#5 Updated by okurz 3 months ago

  • Target version changed from Ready to future
  • Parent task deleted (#118636)

Let me reconsider based on the "hope" that SUSE-IT sufficiently maintains this machine. We still have #120276 to make sure the main service for the openQA bug fetching still works compatible with a new bugzilla version.

#7 Updated by dheidler 3 months ago

Documented ssh access via https://gitlab.suse.de/openqa/password/

#8 Updated by dheidler 3 months ago

I asked Bernhard Wiedemann for clarification:

Die VM laeuft auf unserem morla cluster mit libvirt/KVM.
Solange die an unserem salt master haengt, taucht die auch bei unseren woechentlichen Updates im Infra Team mit auf, dann kuemmern wir uns um die Updates.
Weiss nicht, ob man 2 verschiedene salt master haben kann.
Zumindest mit salt-ssh solltet ihr die zusaetzlich managen koennen.

#9 Updated by dheidler 3 months ago

What is actually done on that machine:

openqa_bugfetcher is installed from devel:openQA repo.

openQA api keys are configured via /etc/openqa/client.conf.

Bugzilla, jira, github and progress credentials are set via /etc/openqa/bugfetcher.conf and /etc/openqa/bugfetcher_o3.conf.

Cronjobs are set up via /etc/crontab:

*/10     *       *       *       *       root  (date;fetch_openqa_bugs) > /tmp/fetch_openqa_bugs_osd.log
1        */10    *       *       *       root  (date;fetch_openqa_bugs /etc/openqa/bugfetcher_o3.conf) > /tmp/fetch_openqa_bugs_o3.log

#10 Updated by tinita 2 months ago

#11 Updated by cdywan 2 months ago

There's a related infra ticket concerning sending of alert emails.

#12 Updated by okurz about 1 month ago

  • Target version changed from future to Ready

#13 Updated by okurz about 1 month ago

  • Target version changed from Ready to future

Also available in: Atom PDF