Project

General

Profile

Actions
Copy link

action #162062

open

No space left on device causing GitLab pipelines to fail

Added by livdywan 6 months ago. Updated 3 months ago.

Status:
Blocked
Priority:
Low
Assignee:
livdywan
Category:
Regressions/Crashes
Target version:
openQA Project (public) - Ready
Start date:
2024-06-10
Due date:
% Done:

0%

Estimated time:
Tags:
alert, infra, reactive work

Description

Observation

Various pipelines fail either running the container or during the script because there is no space left.

https://gitlab.suse.de/openqa/openqa-review/-/jobs/2706267

WARNING: Failed to pull image with policy "always": write /var/lib/docker/tmp/GetImageBlob2814730492: no space left on device (manager.go:250:0s)
ERROR: Job failed: failed to pull image "registry.opensuse.org/home/okurz/container/ca/containers/tumbleweed:openqa-review" with specified policies [always]: write /var/lib/docker/tmp/GetImageBlob2814730492: no space left on device (manager.go:250:0s)

https://gitlab.suse.de/openqa/os-autoinst-needles-opensuse-mirror/-/jobs/2706665

 - Download (curl) error for 'http://download.opensuse.org/distribution/leap/15.5/repo/oss/repodata/d0aae74c050dca8d30fbccd949a136d8ed209eccf8fdf435ac8c1d739271d8e7-appdata.xml.gz':
   Error code: Write error
   Error message: Failure writing output to destination
[...]
Can't create metadata cache directory.

https://gitlab.suse.de/qa-maintenance/bot-ng/-/jobs/2706694

$ rpm --query qam-metadata-openqabot
rpmxdbOpen: No space left on device
error: cannot open Name index using unknown db - Operation not permitted (1)
rpmxdbOpen: No space left on device
error: cannot open Name index using unknown db - Operation not permitted (1)
package qam-metadata-openqabot is not installed

https://gitlab.suse.de/openqa/scripts-ci/-/jobs/2706024

ERROR: Preparation failed: adding cache volume: set volume permissions: create permission container for volume "runner-25ldi6vv-project-11950-concurrent-0-cache-3c3f060a0374fc8bc39395164f415a70": Error response from daemon: mkdir /var/lib/docker/overlay2/3efbcea02b9b1d07ff7de63478a83f5fcd6dd3e36949db2b1766d2ac77c5f8ec-init: no space left on device (linux_set.go:95:0s)

Acceptance criteria

  • AC1:

Suggestions

  • DONE File an infra SD ticket to get the GitLab runners checked
Actions
Copy link

Also available in: Atom PDF