Project

General

Profile

Actions

action #101373

closed

worker openqa-aarch64 fails on cache

Added by ggardet_arm over 2 years ago. Updated over 2 years ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
-
Target version:
Start date:
2021-10-22
Due date:
% Done:

0%

Estimated time:

Description

worker openqa-aarch64 fails on cache service.
systemctl status openqa-worker-cacheservice-minion.service:

● openqa-worker-cacheservice-minion.service - OpenQA Worker Cache Service Minion
   Loaded: loaded (/usr/lib/systemd/system/openqa-worker-cacheservice-minion.service; enabled; vendor preset: disabled)
   Active: active (running) since Fri 2021-10-22 14:29:48 CEST; 39min ago
 Main PID: 9903 (openqa-workerca)
    Tasks: 1
   CGroup: /system.slice/openqa-worker-cacheservice-minion.service
           └─9903 /usr/bin/perl /usr/share/openqa/script/openqa-workercache run -m production --reset-locks

oct. 22 15:09:30 openqa-aarch64 openqa-worker-cacheservice-minion[9903]: [9903] [i] Purging "/var/lib/openqa/cache/openqa1-opensuse/Tumbleweed.aarch64-1.0-libvirt_aarch64-Snapshot20211021.vagrant.libvirt.box" because it appears pending after service startup
oct. 22 15:09:30 openqa-aarch64 openqa-worker-cacheservice-minion[9903]: [9903] [i] Cache size of "/var/lib/openqa/cache" is 10GiB, with limit 160GiB
oct. 22 15:09:30 openqa-aarch64 openqa-worker-cacheservice-minion[9903]: [9903] [i] Resetting all leftover locks after restart
oct. 22 15:09:30 openqa-aarch64 openqa-worker-cacheservice-minion[9903]: [9903] [i] Worker 9903 started
oct. 22 14:29:48 openqa-aarch64 systemd[1]: openqa-worker-cacheservice-minion.service: Failed with result 'exit-code'.
oct. 22 15:09:30 openqa-aarch64 openqa-worker-cacheservice-minion[9903]: DBD::SQLite::db do failed: database is locked at /usr/share/openqa/script/../lib/OpenQA/CacheService.pm line 83.
oct. 22 14:29:48 openqa-aarch64 systemd[1]: openqa-worker-cacheservice-minion.service: Service RestartSec=100ms expired, scheduling restart.
oct. 22 15:09:30 openqa-aarch64 openqa-worker-cacheservice-minion[9903]: DBD::SQLite::db do failed: database is locked at /usr/share/openqa/script/../lib/OpenQA/CacheService.pm line 83.
oct. 22 14:29:48 openqa-aarch64 systemd[1]: Stopped OpenQA Worker Cache Service Minion.
oct. 22 14:29:48 openqa-aarch64 systemd[1]: Started OpenQA Worker Cache Service Minion.

systemctl status openqa-worker-cacheservice returns:

● openqa-worker-cacheservice.service - OpenQA Worker Cache Service
   Loaded: loaded (/usr/lib/systemd/system/openqa-worker-cacheservice.service; disabled; vendor preset: disabled)
   Active: active (running) since Fri 2021-10-22 03:34:26 CEST; 11h ago
 Main PID: 2871 (openqa-workerca)
    Tasks: 5
   CGroup: /system.slice/openqa-worker-cacheservice.service
           ├─ 2871 /usr/bin/perl /usr/share/openqa/script/openqa-workercache prefork -m production -i 100 -H 400 -w 4 -G 80
           ├─ 4149 /usr/bin/perl /usr/share/openqa/script/openqa-workercache prefork -m production -i 100 -H 400 -w 4 -G 80
           ├─ 9376 /usr/bin/perl /usr/share/openqa/script/openqa-workercache prefork -m production -i 100 -H 400 -w 4 -G 80
           ├─ 9819 /usr/bin/perl /usr/share/openqa/script/openqa-workercache prefork -m production -i 100 -H 400 -w 4 -G 80
           └─12829 /usr/bin/perl /usr/share/openqa/script/openqa-workercache prefork -m production -i 100 -H 400 -w 4 -G 80

oct. 22 15:09:30 openqa-aarch64 openqa-workercache-daemon[2871]: [7909] [e] [dWll7_IGGkib] DBD::SQLite::db do failed: database is locked at /usr/share/openqa/script/../lib/OpenQA/CacheService.pm line 83.
oct. 22 15:09:30 openqa-aarch64 openqa-workercache-daemon[2871]: [7909] [e] [NfGgABn2_ds8] DBD::SQLite::db do failed: database is locked at /usr/share/openqa/script/../lib/OpenQA/CacheService.pm line 83.
oct. 22 15:09:30 openqa-aarch64 openqa-workercache-daemon[2871]: [7909] [e] [Q2WvmyLMqkG2] DBD::SQLite::db do failed: database is locked at /usr/share/openqa/script/../lib/OpenQA/CacheService.pm line 83.
oct. 22 15:09:30 openqa-aarch64 openqa-workercache-daemon[2871]: [7909] [e] [PbIxplqBzJFp] DBD::SQLite::db do failed: database is locked at /usr/share/openqa/script/../lib/OpenQA/CacheService.pm line 83.
oct. 22 15:09:30 openqa-aarch64 openqa-workercache-daemon[2871]: [7909] [e] [pnjrAJGY31Sm] DBD::SQLite::db do failed: database is locked at /usr/share/openqa/script/../lib/OpenQA/CacheService.pm line 83.
oct. 22 15:09:30 openqa-aarch64 openqa-workercache-daemon[2871]: [7909] [e] [OLuNhy-wmRTP] DBD::SQLite::st execute failed: database is locked at /usr/lib/perl5/vendor_perl/5.26.1/Minion/Backend/SQLite.pm line 138.
oct. 22 15:09:30 openqa-aarch64 openqa-workercache-daemon[2871]: [7909] [e] [4mdwMlirG86I] DBD::SQLite::st execute failed: database is locked at /usr/lib/perl5/vendor_perl/5.26.1/Minion/Backend/SQLite.pm line 138.
oct. 22 15:09:30 openqa-aarch64 openqa-workercache-daemon[2871]: [2871] [i] Worker 7909 stopped
oct. 22 15:09:30 openqa-aarch64 openqa-workercache-daemon[2871]: [2871] [e] Worker 4845 has no heartbeat (400 seconds), restarting
oct. 22 15:09:30 openqa-aarch64 openqa-workercache-daemon[2871]: [2871] [i] Stopping worker 4845 gracefully (80 seconds)

Files

openqa-aarch64-bootlog.txt (233 KB) openqa-aarch64-bootlog.txt mkittler, 2021-10-25 15:31

Related issues 1 (0 open1 closed)

Related to openQA Infrastructure - action #101033: openqaworker13: Too many Minion job failures alert - sqlite failed: database is locked size:MResolvedkraih2021-10-152021-11-12

Actions
Actions

Also available in: Atom PDF