⚲

Project

General

Profile

Home
Projects
Help

Search:

openQA Infrastructure (public)

All Projects

openQA Infrastructure (public)

Overview
Activity
Roadmap
Issues

Custom queries

openQA Infrastructure Project
openqa-review - Closed tickets last updated by openqa-review, last 30 days
QA roadmap long-term
QA SLE functional
QA SLE Functional - closed in last 14 days
QA SLE Functional - High, need to be refined
QA SLE Functional - over cycle time median
QA SLE u
QA SLE y
QA tools (tag not necessary in openQA and subprojects)
QA tools tag (tag not necessary in openQA and subprojects; excluding tickets in "Ready" version as they are already on the backlog)
QAC - Backlog
QE tools team - backlog (dev)
QE tools team - backlog (ready issues)
QE tools team - backlog SLA high
QE tools team - backlog SLA immediate
QE tools team - backlog SLA no immediate/urgent in feedback/blocked
QE tools team - backlog SLA normal
QE tools team - backlog SLA urgent
QE tools team - backlog SLO high
QE tools team - backlog SLO normal
QE tools team - backlog SLO urgent
QE tools team - backlog, high-level view (epics and higher)
QE tools team - backlog, non-reactive work, needs parent
QE tools team - backlog, top-level view (all sagas)
QE Tools Team - Beginner
QE tools team - closed within last 14 days
QE tools team - closed within last 60 days
QE tools team - closed yesterday
QE Tools Team - Collaborative Session
QE tools team - due date forecast
QE tools team - exceeding due-date
QE Tools Team - Expert
QE tools team - infrastructure backlog
QE tools team - next - sorted by update time
QE tools team - next issues
QE tools team - non-estimated (unblocked) issues (dev)
QE tools team - non-estimated (unblocked) issues (infra)
QE tools team - ready issues - Workable
QE tools team - ready, not assigned/blocked/low
QE tools team - SLO high forecast
QE tools team - update forecast
QE tools team - updated by priority
QE tools team - what members of the team are working on - Feedback (not-low)
QE Tools Team Backlog By Assignee
Tools Team Retrospective
Tools Team Retrospective (not estimated or assigned)

Actions

Copy link

action #133250

closed

gitlab.suse.de unusable "Failed to write to log, write /srv/www/vhosts/gitlab-ce/log/gitlab-shell.log: no space left on device" on remote operations

Added by okurz over 1 year ago. Updated over 1 year ago.

Status:

Resolved

Priority:

Urgent

Assignee:

okurz

Category:

Target version:

openQA Project (public) - Ready

Start date:

2023-07-25

Due date:

2023-08-09

% Done:

Estimated time:

Tags:

gitlab, infra, IT

Description

Observation¶

Changes from gitlab.suse.de can not even be fetched getting error

Failed to write to log, write /srv/www/vhosts/gitlab-ce/log/gitlab-shell.log: no space left on device

reproducibly.

Steps to reproduce¶

In any git checkout connected to gitlab.suse.de run git remote update and observe the error

Rollback steps¶

Enable bot-ng schedule "Approve incidents" again https://gitlab.suse.de/qa-maintenance/bot-ng/-/pipeline_schedules/111/edit coordinating with fniederwanger in https://suse.slack.com/archives/C02CANHLANP/p1690270323953269

Related issues 1 (0 open — 1 closed)

Related to QA (public) - action #133307: mtui: Connection to svn+ssh is not possible or the "inconsistent submission"

Resolved

jbaier_cz

2023-07-25

Actions

Issue # Delay: days Cancel

History
Notes
Property changes

Actions

Copy link

Updated by okurz over 1 year ago

Status changed from New to Blocked
Priority changed from Normal to Urgent

Created https://sd.suse.com/servicedesk/customer/portal/1/SD-128313

Actions

Copy link

Updated by okurz over 1 year ago

Description updated (diff)

disabled the bot-ng schedule "approve incidents"

Actions

Copy link

Updated by okurz over 1 year ago

Description updated (diff)

Actions

Copy link

Updated by crameleon over 1 year ago

Status changed from Blocked to Resolved

Hi,

disk space has been increased. We will work with the owner of the problematic repository to resolve the issue going forward, they currently store over 1TB of data which accumulates in their Git history by the nature of how Git works.
I understand that increasing the disk space does not solve the long term issue with our lack of alerting which could prevent such issues from turning into an outage. I filed https://jira.suse.com/browse/ENGINFRA-2474 as it seems we did not have alerting tracked albeit previous incidents having been caused by the lack of it.

Best,
Georg

Actions

Copy link