action #134453
closedQA - coordination #121720: [saga][epic] Migration to QE setup in PRG2+NUE3 while ensuring availability
QA - coordination #131525: [epic] Up-to-date and usable LSG QE NUE1 machines
backup.qam.suse.de is Failed according to netbox and not creating backups size:M
Added by livdywan 9 months ago. Updated 9 months ago.
0%
Description
Motivation¶
Netbox includes backup.qam.suse.de as Failed. We didn't get any emails, though?
Acceptance criteria¶
- AC1: It is known what backup server(s) we should have in netbox
- AC2: The failure has been resolved.
Suggestions¶
- Update the FQDN of the failed entry or create a new entry in netbox
- The actual domain we want here is backup-qam.qe.nue2.suse.org
- https://netbox.suse.de/search/?q=backup-qam.qe.nue2.suse.org should show the server
- Check possibly already existing tickets about the backup servers
- Clarify what's documented in the wiki i.e. under Operate/backups https://progress.opensuse.org/projects/qa/wiki/Tools#Common-tasks-for-team-members
- Checkout and possibly update https://confluence.suse.com/pages/viewpage.action?spaceKey=maintenanceqa&title=Backup+Server
Updated by livdywan 9 months ago
- Copied from action #131528: Bring backup.qam.suse.de up-to-date size:M added
Updated by livdywan 9 months ago
- Tags changed from infra, backup.qam.suse.de, machine, nue1, dct migration, next-maxtorhof-visit to infra, backup.qam.suse.de
- Subject changed from backup.qam.suse.de is Failed according to netbox and not runnin backups size:M to backup.qam.suse.de is Failed according to netbox and not creating backups
- Assignee deleted (
okurz) - Priority changed from Normal to High
- Start date deleted (
2023-06-28)
Updated by tinita 9 months ago
# rsnapshot configtest
----------------------------------------------------------------------------
rsnapshot encountered an error! The program was invoked with these options:
/usr/bin/rsnapshot configtest
----------------------------------------------------------------------------
ERROR: /etc/rsnapshot.conf on line 42:
ERROR: backup>.root@s.qa:/srv/www/schort/data/links.sqlite s.qa.suse.de/ - \
missing tabs to separate words - change spaces to tabs.
ERROR: ---------------------------------------------------------------------
ERROR: Errors were found in /etc/rsnapshot.conf,
ERROR: rsnapshot can not continue. If you think an entry looks right, make
ERROR: sure you don't have spaces where only tabs should be.
Updated by okurz 9 months ago
The Redmine comment issue is discussed in https://progress.opensuse.org/issues/133532
backup.qam.suse.de is now backup-qam.qe.nbg2.suse.org
Updated by livdywan 9 months ago
- Related to action #134051: Eng-Infra maintained DNS server for .qa.suse.de taking over from qanet size:M added
Updated by tinita 9 months ago
Why is the wiki still talking about backup.qa.suse.de then?
https://progress.opensuse.org/projects/openqav3/wiki/#Backup
Updated by tinita 9 months ago
In this comment
https://progress.opensuse.org/issues/132143#note-52
and the following we see related activity around the time the rsnapshot.conf was broken.
This MR https://gitlab.suse.de/qa-sle/backup-server-salt/-/merge_requests/11 is also related, and looking at /root/.ssh/config it has the same content as on backup.qa.suse.de.
So for now I assume I did the right thing, and we have a backup again.
It would be nice if someone could clarify if backup.qa.suse.de is the correct backup machine or not. Oliver, your comment was raiding more questions than answering. Basically only Liv and me are working today, and we are confused.
Then, as Liv suggested, we should investigate why noone was notified that backups weren't running.
Updated by tinita 9 months ago
Wow, looking at https://gitlab.suse.de/qa-sle/backup-server-salt/-/blob/master/rsnapshot/rsnapshot.conf#L42 this actually shows the broken config, but the last change of that file was July 2022??
Maybe this wasn't a problem in the past and rsnapshot got updated and is now more strict?
Updated by tinita 9 months ago
https://gitlab.suse.de/qa-sle/backup-server-salt/-/merge_requests/12 Fix rsnapshot.conf syntax
Updated by openqa_review 9 months ago
- Due date set to 2023-09-05
Setting due date based on mean cycle time of SUSE QE Tools
Updated by tinita 9 months ago
- Status changed from In Progress to Feedback
https://gitlab.suse.de/qa-sle/backup-server-salt/-/merge_requests/12 merged
We still don't know why we weren't notified.
Updated by tinita 9 months ago
- Copied to action #134489: backup.qa.suse.de does not create backups added
Updated by mkittler 9 months ago
The actual domain is backup-qam.qe.nue2.suse.org (and not backup-qam.qe.nbg2.suse.org and also not backup.qam.suse.de). I've updated the the corresponding confluence page: https://confluence.suse.com/display/maintenanceqa/Backup+Server
(This is a salt controlled host so a simple salt-key -L on OSD helps to find the FQDN.)
Updated by mkittler 9 months ago
- Status changed from Workable to Feedback
I've just updated the netbox entry. I have also updated the management status to "Active" resolving AC2.
I've also updated the FQDN on https://confluence.suse.com/pages/viewpage.action?spaceKey=maintenanceqa&title=Backup+Server.
Now we only need to clarify whether this server is actually still used at all.