While it is restarting my findings so far:
The only process writing to /var/log/openqa
anymore was the web socket server and it didn't log anything interesting (only worker status updates). The machine was generally responsive. I couldn't find any suspicious processes via htop
.
In the end the web UI did not came up anymore at all. Strace shows that it stopped on locking /var/log/openqa
:
sudo sudo -u geekotest strace -y /usr/share/openqa/script/openqa-webui-daemon
[...]
getpid() = 12220
getpid() = 12220
sendto(5<socket:[681345334]>, "Q\0\0\0\nbegin\0", 11, MSG_NOSIGNAL, NULL, 0) = 11
poll([{fd=5<socket:[681345334]>, events=POLLIN|POLLERR}], 1, -1) = 1 ([{fd=5, revents=POLLIN}])
recvfrom(5<socket:[681345334]>, "C\0\0\0\nBEGIN\0Z\0\0\0\5T", 16384, 0, NULL, NULL) = 17
sendto(5<socket:[681345334]>, "P\0\0\0\252\0INSERT INTO audit_events ("..., 297, MSG_NOSIGNAL, NULL, 0) = 297
poll([{fd=5<socket:[681345334]>, events=POLLIN|POLLERR}], 1, -1) = 1 ([{fd=5, revents=POLLIN}])
recvfrom(5<socket:[681345334]>, "1\0\0\0\0042\0\0\0\4T\0\0\0\33\0\1id\0\0\0@\30\0\1\0\0\0\27\0\4"..., 16384, 0, NULL, NULL) = 78
getpid() = 12220
sendto(5<socket:[681345334]>, "Q\0\0\0\vcommit\0", 12, MSG_NOSIGNAL, NULL, 0) = 12
poll([{fd=5<socket:[681345334]>, events=POLLIN|POLLERR}], 1, -1) = 1 ([{fd=5, revents=POLLIN}])
recvfrom(5<socket:[681345334]>, "C\0\0\0\vCOMMIT\0Z\0\0\0\5I", 16384, 0, NULL, NULL) = 18
stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=114, ...}) = 0
stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=114, ...}) = 0
stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=114, ...}) = 0
flock(3</var/log/openqa>, LOCK_EX^C) = ? ERESTARTSYS (To be restarted if SA_RESTART is set)
strace: Process 12220 detached