action #59933
closed
Prevent depletion of space on / with extra partition or quota on /home on o3 size:M
Added by okurz over 4 years ago.
Updated over 2 years ago.
Description
Observation¶
Happened again that we receive a monitoring alert for / being full (but recovered soon after) after a user logged in, did some actions involing bigger space needs and realized the problem.
Problem¶
This can cause disruption of services. The root volume is only 10GB for both osd and o3 which is ok for VMs but users do not realize that /home and /tmp is therefore not able to hold big data.
Acceptance criteria¶
- AC1: /home on o3 is on separate storage
- AC2: DONE same on osd
Suggestions¶
- Enable user quotas (using aquota)
- Mount another partition for /home, e.g. a qcow file, same as on OSD where we already have that
I now created a (raw) disk on OSD located at /srv/homes.img (5G in size) and added an according fstab entry for it. I did go with a raw file since it is easier compared to a qcow2 to mount it via fstab. This should for now at least ease our problems with users writing too big files into their homes.
- Priority changed from Normal to Low
- Related to action #70939: [alert] ** PROBLEM Service Alert: ariel-opensuse.suse.de/root partition is WARNING ** added
On o3 we still have 3 TiB disk space where we could place the home directories on: /dev/vdb1 5,0T 2,1T 3,0T 42% /space
So I guess we could easily move /home to that partition and bind-mount that directory as /home.
I suggest we follow #59933#note-1 with the same approach. This way we can limit the space that is used by /home
- Target version set to Ready
- Target version changed from Ready to future
- Due date set to 2021-12-07
- Assignee set to nicksinger
- Target version changed from future to Ready
- Subject changed from Prevent depletion of space on / with extra partition or quota on /home and/or /tmp (both o3 and osd) to Prevent depletion of space on / with extra partition or quota on /home on o3 size:M
- Description updated (diff)
- Status changed from New to Workable
- Status changed from Workable to Resolved
I created a file-based hdd in /space/homes.img with 5G allocated. We currently only use ~400MB for all in /home so I assume 5G should be plenty and hopefully reminds people not to pollute their home directories too much but also can be enlarged anytime in the future.
Entry added to /etc/fstab for automatic configuration/mounting:
/space/homes.img /home ext4 defaults 1 1
and tested with mount -a. Old /home/* content moved to /space/OLD_HOMES_DELETEME_AFTER_2021_12_25_POO59933 because rm felt wrong and in case anything went wrong we can copy it back.
Quick relogin with my own account doesn't seem to show any problems (permissions all preserved, noting missing). Please reopen if you find any issues.
Also available in: Atom
PDF