Project

General

Profile

Actions

action #128045

closed

/var on qanet is 100%

Added by okurz over 1 year ago. Updated over 1 year ago.

Status:
Resolved
Priority:
High
Assignee:
Start date:
2023-04-20
Due date:
% Done:

0%

Estimated time:
Tags:

Description

Observation

From qanet.qa.suse.de df -h:

# df -h
Filesystem                                    Size  Used Avail Use% Mounted on
devtmpfs                                      2.9G     0  2.9G   0% /dev
tmpfs                                         3.0G     0  3.0G   0% /dev/shm
tmpfs                                         3.0G  1.9M  3.0G   1% /run
tmpfs                                         3.0G     0  3.0G   0% /sys/fs/cgroup
/dev/sda2                                      20G  7.3G   12G  40% /
/dev/sdb2                                      72G   31G   38G  45% /data
/dev/sda3                                      20G   16G  2.7G  86% /srv
/dev/sda6                                      40G   38G     0 100% /var
/dev/sda1                                     469M   95M  349M  22% /boot
/dev/sda5                                      20G  1.5M   19G   1% /tmp

Related issues 1 (0 open1 closed)

Related to openQA Infrastructure (public) - action #126821: [openQA][infra][worker] ppc64 fails to load grub2 completely over tftp/pxe on qanet and PXE load timeout issues size:MResolvednicksinger2023-03-282023-05-05

Actions
Actions #1

Updated by okurz over 1 year ago

I deleted many logfiles in /var/log getting to 90%. /var/log/atftpd/atftp.log is 26G, starting 2023-03-27. I guess we have debug log level enabled to debug some PXE related problems but logrotate might help as well. /etc/logrotate.d/atftp exists but the logrotate service was not enabled. I enabled now with systemctl enable --now logrotate but the service does not look healthy. I triggered manually logrotate /etc/logrotate.conf for now

Actions #2

Updated by okurz over 1 year ago

  • Related to action #126821: [openQA][infra][worker] ppc64 fails to load grub2 completely over tftp/pxe on qanet and PXE load timeout issues size:M added
Actions #3

Updated by openqa_review over 1 year ago

  • Due date set to 2023-05-05

Setting due date based on mean cycle time of SUSE QE Tools

Actions #4

Updated by okurz over 1 year ago

After logrotate rotated and compressed mostly the atftpd files now /var is on 14%, using 5.0G. However I am wondering as I don't see any update in /var/log/atftpd/atftp.log since 2023-04-20 14:22. Apparently the atftpd process was not running. I assume nsinger has started the service manually now. systemctl status shows

$ sudo systemctl status atftpd.service 
● atftpd.service - LSB: launch atftpd server
   Loaded: loaded (/etc/init.d/atftpd; bad; vendor preset: disabled)
   Active: active (running) since Fri 2023-04-21 12:17:14 CEST; 2h 0min ago
     Docs: man:systemd-sysv-generator(8)
    Tasks: 1 (limit: 512)
   CGroup: /system.slice/atftpd.service
           └─20112 /usr/sbin/atftpd --pidfile /var/run/atftpd/pid --daemon --user tftp --group tftp --logfile /var/log/atftpd/atftp.log --no-multicast --trace --retry-timeout 30 --daemo...

Apr 21 12:17:13 qanet systemd[1]: Starting LSB: launch atftpd server...
Apr 21 12:17:14 qanet atftpd[20101]: Starting Advanced Trivial FTP server: ..done
Apr 21 12:17:14 qanet systemd[1]: Started LSB: launch atftpd server.

so it's running but it says "bad". nsinger said that he enabled the service and started it. I should try with a reboot.

After the service was started (again?) /var/log/atftpd/atftp.log was very noisy. I removed the --trace in /etc/sysconfig/atftpd

Actions #5

Updated by okurz over 1 year ago

  • Due date deleted (2023-05-05)
  • Status changed from In Progress to Resolved

situation seems to be resolved now

Actions

Also available in: Atom PDF