action #62309: logrotate fails on QA-Power8-4-kvm (and powerqaworker-qam-1) with "error: destination /var/log/messages-20200118.xz already exists, skipping rotation" - openQA Infrastructure - openSUSE Project Management Tool

Actions

Copy link

action #62309

closed

logrotate fails on QA-Power8-4-kvm (and powerqaworker-qam-1) with "error: destination /var/log/messages-20200118.xz already exists, skipping rotation"

Added by okurz over 4 years ago. Updated about 4 years ago.

Status:

Resolved

Priority:

Normal

Assignee:

okurz

Category:

Target version:

Start date:

2020-01-19

Due date:

2020-04-14

% Done:

Estimated time:

Description

> sudo systemctl status logrotate
● logrotate.service - Rotate log files
   Loaded: loaded (/usr/lib/systemd/system/logrotate.service; static; vendor preset: disabled)
   Active: failed (Result: exit-code) since Sun 2020-01-19 00:00:00 CET; 22h ago
     Docs: man:logrotate(8)
           man:logrotate.conf(5)
  Process: 48560 ExecStart=/usr/sbin/logrotate /etc/logrotate.conf (code=exited, status=1/FAILURE)
 Main PID: 48560 (code=exited, status=1/FAILURE)

Jan 19 00:00:00 QA-Power8-4-kvm systemd[1]: Starting Rotate log files...
Jan 19 00:00:00 QA-Power8-4-kvm logrotate[48560]: error: destination /var/log/messages-20200118.xz already exists, skipping rotation
Jan 19 00:00:00 QA-Power8-4-kvm systemd[1]: logrotate.service: Main process exited, code=exited, status=1/FAILURE
Jan 19 00:00:00 QA-Power8-4-kvm systemd[1]: Failed to start Rotate log files.
Jan 19 00:00:00 QA-Power8-4-kvm systemd[1]: logrotate.service: Unit entered failed state.
Jan 19 00:00:00 QA-Power8-4-kvm systemd[1]: logrotate.service: Failed with result 'exit-code'.

Related issues 1 (0 open — 1 closed)

Actions

Copy link

Updated by okurz over 4 years ago

Copied from action #62306: osd logrotate fails sporadically on "error opening /var/log/salt/master: Permission denied", only at 00:00, i.e. midnight every day. added

Actions

Copy link

Updated by okurz about 4 years ago

Status changed from New to Rejected
Assignee set to okurz

logrotate is very stable for a month on that host:

QA-Power8-4-kvm:/home/okurz # journalctl -u logrotate
Jan 29 00:00:00 QA-Power8-4-kvm systemd[1]: Starting Rotate log files...
…
Mar 01 00:00:34 QA-Power8-4-kvm systemd[1]: Started Rotate log files.

I assume the problem happened because by accident, maybe manually, logrotate was triggered a second time during the same day which fails in the way as described above. I don't think we need to do anything further about it.

Actions

Copy link

Updated by okurz about 4 years ago

Status changed from Rejected to Workable

I think the same thing happened again now on powerqaworker-qam-1:

okurz@powerqaworker-qam-1:~> sudo systemctl --failed
  UNIT              LOAD   ACTIVE SUB    DESCRIPTION                                                                                                                                                                                                                                       
● logrotate.service loaded failed failed Rotate log files                                                                                                                                                                                                                                  

LOAD   = Reflects whether the unit definition was properly loaded.
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
SUB    = The low-level unit activation state, values depend on unit type.

1 loaded units listed. Pass --all to see loaded but inactive units, too.
To show all installed unit files use 'systemctl list-unit-files'.
okurz@powerqaworker-qam-1:~> sudo systemctl status logrotate
● logrotate.service - Rotate log files
   Loaded: loaded (/usr/lib/systemd/system/logrotate.service; static; vendor preset: disabled)
   Active: failed (Result: exit-code) since Sat 2020-03-07 00:00:00 CET; 11h ago
     Docs: man:logrotate(8)
           man:logrotate.conf(5)
  Process: 43196 ExecStart=/usr/sbin/logrotate /etc/logrotate.conf (code=exited, status=1/FAILURE)
 Main PID: 43196 (code=exited, status=1/FAILURE)

Mar 07 00:00:00 powerqaworker-qam-1 systemd[1]: Starting Rotate log files...
Mar 07 00:00:00 powerqaworker-qam-1 logrotate[43196]: error: destination /var/log/messages-20200306.xz already exists, skipping rotation
Mar 07 00:00:00 powerqaworker-qam-1 logrotate[43196]: error: destination /var/log/zypper.log-20200306.xz already exists, skipping rotation
Mar 07 00:00:00 powerqaworker-qam-1 systemd[1]: logrotate.service: Main process exited, code=exited, status=1/FAILURE
Mar 07 00:00:00 powerqaworker-qam-1 systemd[1]: Failed to start Rotate log files.
Mar 07 00:00:00 powerqaworker-qam-1 systemd[1]: logrotate.service: Unit entered failed state.
Mar 07 00:00:00 powerqaworker-qam-1 systemd[1]: logrotate.service: Failed with result 'exit-code'.
okurz@powerqaworker-qam-1:~> sudo journalctl --since=yesterday -u logrotate
-- Logs begin at Fri 2020-01-31 19:22:28 CET, end at Sat 2020-03-07 11:37:10 CET. --
Mar 06 00:00:00 powerqaworker-qam-1 systemd[1]: Starting Rotate log files...
Mar 06 00:00:22 powerqaworker-qam-1 systemd[1]: Started Rotate log files.
Mar 07 00:00:00 powerqaworker-qam-1 systemd[1]: Starting Rotate log files...
Mar 07 00:00:00 powerqaworker-qam-1 logrotate[43196]: error: destination /var/log/messages-20200306.xz already exists, skipping rotation
Mar 07 00:00:00 powerqaworker-qam-1 logrotate[43196]: error: destination /var/log/zypper.log-20200306.xz already exists, skipping rotation
Mar 07 00:00:00 powerqaworker-qam-1 systemd[1]: logrotate.service: Main process exited, code=exited, status=1/FAILURE
Mar 07 00:00:00 powerqaworker-qam-1 systemd[1]: Failed to start Rotate log files.
Mar 07 00:00:00 powerqaworker-qam-1 systemd[1]: logrotate.service: Unit entered failed state.
Mar 07 00:00:00 powerqaworker-qam-1 systemd[1]: logrotate.service: Failed with result 'exit-code'.

Normally it seems logrotate should have used 20200307 as timestamp, not the previous day. This is also what happened on all other hosts:

$ ssh osd 'sudo salt -C G@roles:worker cmd.run "ls -ltra /var/log/zypper.log-202003*"'
QA-Power8-5-kvm.qa.suse.de:
    -rw-r----- 1 root root 491672 Mar  1 23:32 /var/log/zypper.log-20200302.xz
    -rw-r----- 1 root root 297324 Mar  2 23:07 /var/log/zypper.log-20200303.xz
    -rw-r----- 1 root root 275048 Mar  3 23:07 /var/log/zypper.log-20200304.xz
    -rw-r----- 1 root root 834036 Mar  4 23:07 /var/log/zypper.log-20200305.xz
    -rw-r----- 1 root root 553060 Mar  6 23:36 /var/log/zypper.log-20200307.xz
powerqaworker-qam-1:
    -rw-r----- 1 root root 502428 Mar  1 23:32 /var/log/zypper.log-20200302.xz
    -rw-r----- 1 root root 305112 Mar  2 23:07 /var/log/zypper.log-20200303.xz
    -rw-r----- 1 root root 267896 Mar  3 23:07 /var/log/zypper.log-20200304.xz
    -rw-r----- 1 root root 839212 Mar  4 23:07 /var/log/zypper.log-20200305.xz
    -rw-r----- 1 root root 252548 Mar  5 23:07 /var/log/zypper.log-20200306.xz
QA-Power8-4-kvm.qa.suse.de:
    -rw-r----- 1 root root 505444 Mar  1 23:32 /var/log/zypper.log-20200302.xz
    -rw-r----- 1 root root 304500 Mar  2 23:07 /var/log/zypper.log-20200303.xz
    -rw-r----- 1 root root 270648 Mar  3 23:07 /var/log/zypper.log-20200304.xz
    -rw-r----- 1 root root 830240 Mar  4 23:07 /var/log/zypper.log-20200305.xz
    -rw-r----- 1 root root 546260 Mar  6 23:36 /var/log/zypper.log-20200307.xz
malbec.arch.suse.de:
    -rw-r----- 1 root root  498072 Mar  1 23:32 /var/log/zypper.log-20200302.xz
    -rw-r----- 1 root root  306368 Mar  2 23:07 /var/log/zypper.log-20200303.xz
    -rw-r----- 1 root root 1083864 Mar  4 23:07 /var/log/zypper.log-20200305.xz
    -rw-r----- 1 root root  550884 Mar  6 23:36 /var/log/zypper.log-20200307.xz
grenache-1.qa.suse.de:
    -rw-r----- 1 root root 517372 Mar  1 23:32 /var/log/zypper.log-20200302.xz
    -rw-r----- 1 root root 305372 Mar  2 23:08 /var/log/zypper.log-20200303.xz
    -rw-r----- 1 root root 266464 Mar  3 23:07 /var/log/zypper.log-20200304.xz
    -rw-r----- 1 root root 831740 Mar  4 23:07 /var/log/zypper.log-20200305.xz
    -rw-r----- 1 root root 544836 Mar  6 23:36 /var/log/zypper.log-20200307.xz
openqaworker6.suse.de:
    -rw-r----- 1 root root 519520 Mar  1 23:32 /var/log/zypper.log-20200302.xz
    -rw-r----- 1 root root 317444 Mar  2 23:07 /var/log/zypper.log-20200303.xz
    -rw-r----- 1 root root 282172 Mar  3 23:07 /var/log/zypper.log-20200304.xz
    -rw-r----- 1 root root 843860 Mar  4 23:07 /var/log/zypper.log-20200305.xz
    -rw-r----- 1 root root 555188 Mar  6 23:36 /var/log/zypper.log-20200307.xz
openqaworker9.suse.de:
    -rw-r----- 1 root root 513000 Mar  1 23:32 /var/log/zypper.log-20200302.xz
    -rw-r----- 1 root root 314220 Mar  2 23:07 /var/log/zypper.log-20200303.xz
    -rw-r----- 1 root root 273420 Mar  3 23:07 /var/log/zypper.log-20200304.xz
    -rw-r----- 1 root root 839824 Mar  4 23:07 /var/log/zypper.log-20200305.xz
    -rw-r----- 1 root root 563696 Mar  6 23:36 /var/log/zypper.log-20200307.xz
openqaworker5.suse.de:
    -rw-r----- 1 root root 523112 Mar  1 23:32 /var/log/zypper.log-20200302.xz
    -rw-r----- 1 root root 320096 Mar  2 23:07 /var/log/zypper.log-20200303.xz
    -rw-r----- 1 root root 281220 Mar  3 23:07 /var/log/zypper.log-20200304.xz
    -rw-r----- 1 root root 846016 Mar  4 23:07 /var/log/zypper.log-20200305.xz
    -rw-r----- 1 root root 566624 Mar  6 23:36 /var/log/zypper.log-20200307.xz
openqaworker3.suse.de:
    -rw-r----- 1 root root 528432 Mar  1 23:32 /var/log/zypper.log-20200302.xz
    -rw-r----- 1 root root 325784 Mar  2 23:07 /var/log/zypper.log-20200303.xz
    -rw-r----- 1 root root 281320 Mar  3 23:07 /var/log/zypper.log-20200304.xz
    -rw-r----- 1 root root 856288 Mar  4 23:07 /var/log/zypper.log-20200305.xz
    -rw-r----- 1 root root 564260 Mar  6 23:36 /var/log/zypper.log-20200307.xz
openqaworker8.suse.de:
    -rw-r----- 1 root root 261048 Feb 29 23:32 /var/log/zypper.log-20200301.xz
    -rw-r----- 1 root root 271804 Mar  1 23:32 /var/log/zypper.log-20200302.xz
    -rw-r----- 1 root root 320844 Mar  2 23:07 /var/log/zypper.log-20200303.xz
    -rw-r----- 1 root root 275896 Mar  3 23:07 /var/log/zypper.log-20200304.xz
    -rw-r----- 1 root root 859244 Mar  4 23:07 /var/log/zypper.log-20200305.xz
    -rw-r----- 1 root root 269160 Mar  5 23:07 /var/log/zypper.log-20200306.xz
    -rw-r----- 1 root root 307260 Mar  6 23:36 /var/log/zypper.log-20200307.xz
openqaworker2.suse.de:
    -rw-r----- 1 root root 483668 Mar  1 23:32 /var/log/zypper.log-20200302.xz
    -rw-r----- 1 root root 544864 Mar  3 23:07 /var/log/zypper.log-20200304.xz
    -rw-r----- 1 root root 818552 Mar  4 23:07 /var/log/zypper.log-20200305.xz
    -rw-r----- 1 root root 526228 Mar  6 23:36 /var/log/zypper.log-20200307.xz
openqaworker10.suse.de:
    -rw-r----- 1 root root 510000 Mar  1 23:32 /var/log/zypper.log-20200302.xz
    -rw-r----- 1 root root 318432 Mar  2 23:07 /var/log/zypper.log-20200303.xz
    -rw-r----- 1 root root 273408 Mar  3 23:07 /var/log/zypper.log-20200304.xz
    -rw-r----- 1 root root 841204 Mar  4 23:07 /var/log/zypper.log-20200305.xz
    -rw-r----- 1 root root 556536 Mar  6 23:36 /var/log/zypper.log-20200307.xz
openqaworker13.suse.de:
    -rw-r----- 1 root root  486856 Mar  1 23:32 /var/log/zypper.log-20200302.xz
    -rw-r----- 1 root root  306572 Mar  2 23:07 /var/log/zypper.log-20200303.xz
    -rw-r----- 1 root root 1101104 Mar  4 23:07 /var/log/zypper.log-20200305.xz
    -rw-r----- 1 root root  531576 Mar  6 23:36 /var/log/zypper.log-20200307.xz
openqaworker-arm-1.suse.de:
    -rw-r----- 1 root root  515504 Mar  1 23:34 /var/log/zypper.log-20200302.xz
    -rw-r----- 1 root root  307764 Mar  2 23:09 /var/log/zypper.log-20200303.xz
    -rw-r----- 1 root root 2227856 Mar  4 23:28 /var/log/zypper.log-20200305.xz
    -rw-r----- 1 root root  535372 Mar  6 23:36 /var/log/zypper.log-20200307.xz
openqaworker-arm-2.suse.de:
    -rw-r----- 1 root root 471068 Mar  1 23:49 /var/log/zypper.log-20200302.xz
    -rw-r----- 1 root root 302652 Mar  2 23:10 /var/log/zypper.log-20200303.xz
    -rw-r----- 1 root root 891148 Mar  4 23:48 /var/log/zypper.log-20200305.xz
    -rw-r----- 1 root root 475816 Mar  6 23:16 /var/log/zypper.log-20200307.xz
openqaworker-arm-3.suse.de:
    -rw-r----- 1 root root  481840 Feb 29 23:16 /var/log/zypper.log-20200301.xz
    -rw-r----- 1 root root  251688 Mar  1 23:16 /var/log/zypper.log-20200302.xz
    -rw-r----- 1 root root  293092 Mar  2 23:08 /var/log/zypper.log-20200303.xz
    -rw-r----- 1 root root  270428 Mar  3 23:08 /var/log/zypper.log-20200304.xz
    -rw-r----- 1 root root 1966164 Mar  4 23:08 /var/log/zypper.log-20200305.xz
    -rw-r----- 1 root root  547748 Mar  6 23:37 /var/log/zypper.log-20200307.xz

But I see a different inconsistency:

$ ssh osd 'sudo salt \* cmd.run "date +\"%Z %z\""'
QA-Power8-5-kvm.qa.suse.de:
    CET +0100
QA-Power8-4-kvm.qa.suse.de:
    CET +0100
powerqaworker-qam-1:
    CET +0100
malbec.arch.suse.de:
    CET +0100
grenache-1.qa.suse.de:
    CET +0100
openqaworker8.suse.de:
    CET +0100
openqaworker2.suse.de:
    CET +0100
openqaworker3.suse.de:
    CET +0100
openqaworker9.suse.de:
    CET +0100
openqaworker5.suse.de:
    CET +0100
openqa-monitor.qa.suse.de:
    CET +0100
openqaworker6.suse.de:
    CET +0100
openqa.suse.de:
    CET +0100
openqaworker13.suse.de:
    CET +0100
openqaworker10.suse.de:
    UTC +0000
openqaworker-arm-1.suse.de:
    UTC +0000
openqaworker-arm-3.suse.de:
    CET +0100
openqaworker-arm-2.suse.de:
    UTC +0000

IMHO we should switch all hosts to the same timezone. I personally would prefer UTC but CET seems more common, should be easy to adjust openqaworker10, arm1 and arm3. o3 is similar, CET more common, power8 and imagetester UTC. However powerqaworker-qam-1 is not any different than the common. It seems like logrotate triggered at exactly 00:00:00 took by accident still the date from the previous day, as if it was started at 23:59:59.

Another point I found:

$ ssh osd 'sudo salt \* cmd.run "grep DAILY_TIME /etc/sysconfig/cron"'
QA-Power8-4-kvm.qa.suse.de:
    DAILY_TIME=""
    # Maximum days not running when using a fixed time set in DAILY_TIME.
QA-Power8-5-kvm.qa.suse.de:
    DAILY_TIME=""
    # Maximum days not running when using a fixed time set in DAILY_TIME.
powerqaworker-qam-1:
    DAILY_TIME=""
    # Maximum days not running when using a fixed time set in DAILY_TIME.
malbec.arch.suse.de:
    DAILY_TIME=""
    # Maximum days not running when using a fixed time set in DAILY_TIME.
grenache-1.qa.suse.de:
    DAILY_TIME=""
    # Maximum days not running when using a fixed time set in DAILY_TIME.
openqaworker8.suse.de:
    DAILY_TIME=""
    # Maximum days not running when using a fixed time set in DAILY_TIME.
openqaworker3.suse.de:
    DAILY_TIME=""
    # Maximum days not running when using a fixed time set in DAILY_TIME.
openqaworker5.suse.de:
    DAILY_TIME=""
    # Maximum days not running when using a fixed time set in DAILY_TIME.
openqaworker6.suse.de:
    DAILY_TIME=""
    # Maximum days not running when using a fixed time set in DAILY_TIME.
openqaworker2.suse.de:
    DAILY_TIME=""
    # Maximum days not running when using a fixed time set in DAILY_TIME.
openqaworker9.suse.de:
    DAILY_TIME=""
    # Maximum days not running when using a fixed time set in DAILY_TIME.
openqa-monitor.qa.suse.de:
    DAILY_TIME=""
    # Maximum days not running when using a fixed time set in DAILY_TIME.
openqaworker10.suse.de:
    DAILY_TIME=""
    # Maximum days not running when using a fixed time set in DAILY_TIME.
openqaworker13.suse.de:
    DAILY_TIME=""
    # Maximum days not running when using a fixed time set in DAILY_TIME.
openqaworker-arm-1.suse.de:
    DAILY_TIME=""
    # Maximum days not running when using a fixed time set in DAILY_TIME.
openqa.suse.de:
    DAILY_TIME="19:30"
    # Maximum days not running when using a fixed time set in DAILY_TIME.
openqaworker-arm-2.suse.de:
    DAILY_TIME=""
    # Maximum days not running when using a fixed time set in DAILY_TIME.
openqaworker-arm-3.suse.de:
    DAILY_TIME=""
    # Maximum days not running when using a fixed time set in DAILY_TIME.

so all machines trigger daily jobs at 00:00 local time which is 23:00 UTC.

Actions

Copy link

Updated by okurz about 4 years ago

Subject changed from logrotate fails on QA-Power8-4-kvm with "error: destination /var/log/messages-20200118.xz already exists, skipping rotation" to logrotate fails on QA-Power8-4-kvm (and powerqaworker-qam-1) with "error: destination /var/log/messages-20200118.xz already exists, skipping rotation"

Actions

Copy link

Updated by okurz about 4 years ago

Status changed from Workable to Feedback

As the problem happened for rotating /var/log/messages which we shouldn't need when we rather rely on persistent systemd journal I suggest we simply remove rsyslog completely as we do not have it on many other machines anyway:

# salt -l error --no-color -C  '*' cmd.run 'rpm -qa | grep syslog'
openqaworker2.suse.de:
openqaworker5.suse.de:
openqaworker3.suse.de:
openqaworker6.suse.de:
openqaworker8.suse.de:
openqa-monitor.qa.suse.de:
malbec.arch.suse.de:
    syslog-service-2.0-lp151.3.4.noarch
    rsyslog-8.33.1-lp151.6.10.1.ppc64le
QA-Power8-5-kvm.qa.suse.de:
    rsyslog-8.33.1-lp151.6.10.1.ppc64le
    syslog-service-2.0-lp151.3.4.noarch
openqaworker10.suse.de:
openqaworker13.suse.de:
QA-Power8-4-kvm.qa.suse.de:
    syslog-service-2.0-lp151.3.4.noarch
    rsyslog-8.33.1-lp151.6.10.1.ppc64le
powerqaworker-qam-1:
    syslog-service-2.0-lp151.3.4.noarch
    rsyslog-8.33.1-lp151.6.10.1.ppc64le
grenache-1.qa.suse.de:
    rsyslog-8.33.1-lp151.6.10.1.ppc64le
    syslog-service-2.0-lp151.3.4.noarch
openqaworker-arm-1.suse.de:
    syslog-service-2.0-lp151.3.3.noarch
    rsyslog-8.33.1-lp151.6.10.1.aarch64
openqaworker-arm-2.suse.de:
    rsyslog-8.33.1-lp151.6.10.1.aarch64
    syslog-service-2.0-lp151.3.3.noarch
openqa.suse.de:

https://gitlab.suse.de/openqa/salt-states-openqa/-/blob/master/openqa/journal.sls is where we ensure a persistent journal so we can just make sure the package is absent.

Created https://gitlab.suse.de/openqa/salt-states-openqa/-/merge_requests/295 to fix by removing syslog.

Actions

Copy link

Updated by okurz about 4 years ago

Due date set to 2020-04-14

Actions

Copy link

Updated by okurz about 4 years ago

Status changed from Feedback to Resolved

syslog has been removed from all workers, logrotate shows no problems on no worker so far:

# salt -l error --no-color -C  '*' cmd.run 'journalctl --since=yesterday -e -u logrotate'
QA-Power8-4-kvm.qa.suse.de:
    -- Logs begin at Sun 2020-03-01 16:44:16 CET, end at Tue 2020-03-31 10:07:11 CEST. --
    Mar 30 00:00:00 QA-Power8-4-kvm systemd[1]: Starting Rotate log files...
    Mar 30 00:00:13 QA-Power8-4-kvm systemd[1]: Started Rotate log files.
    Mar 31 00:00:00 QA-Power8-4-kvm systemd[1]: Starting Rotate log files...
    Mar 31 00:00:03 QA-Power8-4-kvm systemd[1]: Started Rotate log files.
powerqaworker-qam-1:
    -- Logs begin at Thu 2020-03-05 07:58:23 CET, end at Tue 2020-03-31 10:07:11 CEST. --
    Mar 30 00:00:00 powerqaworker-qam-1 systemd[1]: Starting Rotate log files...
    Mar 30 00:00:19 powerqaworker-qam-1 systemd[1]: Started Rotate log files.
    Mar 31 00:00:00 powerqaworker-qam-1 systemd[1]: Starting Rotate log files...
    Mar 31 00:00:02 powerqaworker-qam-1 systemd[1]: Started Rotate log files.
openqaworker2.suse.de:
    -- Logs begin at Mon 2020-03-16 21:45:09 CET, end at Tue 2020-03-31 10:07:10 CEST. --
    Mar 30 00:00:00 openqaworker2 systemd[1]: Starting Rotate log files...
    Mar 30 00:00:03 openqaworker2 systemd[1]: Started Rotate log files.
    Mar 31 00:00:00 openqaworker2 systemd[1]: Starting Rotate log files...
    Mar 31 00:00:02 openqaworker2 systemd[1]: Started Rotate log files.
openqaworker5.suse.de:
    -- Logs begin at Tue 2020-03-24 00:38:19 CET, end at Tue 2020-03-31 10:07:10 CEST. --
    Mar 30 00:00:00 openqaworker5 systemd[1]: Starting Rotate log files...
    Mar 30 00:00:04 openqaworker5 systemd[1]: Started Rotate log files.
    Mar 31 00:00:00 openqaworker5 systemd[1]: Starting Rotate log files...
    Mar 31 00:00:02 openqaworker5 systemd[1]: Started Rotate log files.
openqaworker-arm-1.suse.de:
    -- Logs begin at Wed 2020-01-08 12:17:31 UTC, end at Tue 2020-03-31 08:07:10 UTC. --
    Mar 30 00:00:04 openqaworker-arm-1 systemd[1]: Starting Rotate log files...
    Mar 30 00:00:16 openqaworker-arm-1 systemd[1]: Started Rotate log files.
    Mar 31 00:00:04 openqaworker-arm-1 systemd[1]: Starting Rotate log files...
    Mar 31 00:00:11 openqaworker-arm-1 systemd[1]: Started Rotate log files.
openqaworker6.suse.de:
    -- Logs begin at Mon 2019-02-18 15:10:35 CET, end at Tue 2020-03-31 10:07:10 CEST. --
    Mar 30 00:00:00 openqaworker6 systemd[1]: Starting Rotate log files...
    Mar 30 00:00:02 openqaworker6 systemd[1]: Started Rotate log files.
    Mar 31 00:00:00 openqaworker6 systemd[1]: Starting Rotate log files...
    Mar 31 00:00:02 openqaworker6 systemd[1]: Started Rotate log files.
QA-Power8-5-kvm.qa.suse.de:
    -- Logs begin at Thu 2020-02-27 00:28:46 CET, end at Tue 2020-03-31 10:07:11 CEST. --
    Mar 30 00:00:00 QA-Power8-5-kvm systemd[1]: Starting Rotate log files...
    Mar 30 00:00:11 QA-Power8-5-kvm systemd[1]: Started Rotate log files.
    Mar 31 00:00:00 QA-Power8-5-kvm systemd[1]: Starting Rotate log files...
    Mar 31 00:00:04 QA-Power8-5-kvm systemd[1]: Started Rotate log files.
openqa-monitor.qa.suse.de:
    -- Logs begin at Mon 2020-03-30 21:15:36 CEST, end at Tue 2020-03-31 10:07:10 CEST. --
    Mar 31 00:00:00 openqa-monitor systemd[1]: Starting Rotate log files...
    Mar 31 00:00:09 openqa-monitor systemd[1]: Started Rotate log files.
openqaworker10.suse.de:
    -- Logs begin at Mon 2018-06-18 12:58:07 UTC, end at Tue 2020-03-31 08:07:11 UTC. --
    Mar 30 00:00:00 openqaworker10 systemd[1]: Starting Rotate log files...
    Mar 30 00:00:04 openqaworker10 systemd[1]: Started Rotate log files.
    Mar 31 00:00:00 openqaworker10 systemd[1]: Starting Rotate log files...
    Mar 31 00:00:02 openqaworker10 systemd[1]: Started Rotate log files.
openqaworker8.suse.de:
    -- Logs begin at Thu 2018-01-11 09:56:40 CET, end at Tue 2020-03-31 10:07:11 CEST. --
    Mar 30 00:00:00 openqaworker8 systemd[1]: Starting Rotate log files...
    Mar 30 00:00:00 openqaworker8 systemd[1]: Started Rotate log files.
    Mar 31 00:00:00 openqaworker8 systemd[1]: Starting Rotate log files...
    Mar 31 00:00:03 openqaworker8 systemd[1]: Started Rotate log files.
openqaworker3.suse.de:
    -- Logs begin at Mon 2020-03-23 08:26:37 CET, end at Tue 2020-03-31 10:07:11 CEST. --
    Mar 30 00:00:00 openqaworker3 systemd[1]: Starting Rotate log files...
    Mar 30 00:00:04 openqaworker3 systemd[1]: Started Rotate log files.
    Mar 31 00:00:00 openqaworker3 systemd[1]: Starting Rotate log files...
    Mar 31 00:00:02 openqaworker3 systemd[1]: Started Rotate log files.
openqa.suse.de:
    -- Logs begin at Mon 2020-03-30 11:06:59 CEST, end at Tue 2020-03-31 10:07:08 CEST. --
    Mar 31 00:00:00 openqa systemd[1]: Starting Rotate log files...
    Mar 31 00:42:24 openqa systemd[1]: Started Rotate log files.
malbec.arch.suse.de:
    -- Logs begin at Wed 2020-01-15 09:58:59 CET, end at Tue 2020-03-31 10:07:10 CEST. --
    Mar 30 00:00:00 malbec systemd[1]: Starting Rotate log files...
    Mar 30 00:00:04 malbec systemd[1]: Started Rotate log files.
    Mar 31 00:00:00 malbec systemd[1]: Starting Rotate log files...
    Mar 31 00:00:05 malbec systemd[1]: Started Rotate log files.
grenache-1.qa.suse.de:
    -- Logs begin at Wed 2020-03-11 04:26:26 CET, end at Tue 2020-03-31 10:07:11 CEST. --
    Mar 30 00:00:00 grenache-1 systemd[1]: Starting Rotate log files...
    Mar 30 00:00:23 grenache-1 systemd[1]: Started Rotate log files.
    Mar 31 00:00:00 grenache-1 systemd[1]: Starting Rotate log files...
    Mar 31 00:00:03 grenache-1 systemd[1]: Started Rotate log files.
openqaworker13.suse.de:
    -- Logs begin at Thu 2018-10-04 14:09:46 CEST, end at Tue 2020-03-31 10:07:15 CEST. --
    Mar 30 00:00:00 openqaworker13 systemd[1]: Starting Rotate log files...
    Mar 30 00:00:01 openqaworker13 systemd[1]: Started Rotate log files.
    Mar 31 00:00:00 openqaworker13 systemd[1]: Starting Rotate log files...
    Mar 31 00:00:06 openqaworker13 systemd[1]: Started Rotate log files.

Actions

Copy link

Also available in: Atom PDF

Project

General

Profile

QA » openQA Project » openQA Infrastructure

Tags

Custom queries

action #62309

logrotate fails on QA-Power8-4-kvm (and powerqaworker-qam-1) with "error: destination /var/log/messages-20200118.xz already exists, skipping rotation"

Updated by okurz over 4 years ago

Updated by okurz about 4 years ago

Updated by okurz about 4 years ago

Updated by okurz about 4 years ago

Updated by okurz about 4 years ago

Updated by okurz about 4 years ago

Updated by okurz about 4 years ago