action #48674
closedopenqaworker3 MM is broken
0%
Description
systemctl status os-autoinst-openvswitch.service
● os-autoinst-openvswitch.service - os-autoinst openvswitch helper
Loaded: loaded (/etc/systemd/system/os-autoinst-openvswitch.service; enabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Mon 2019-02-25 17:03:34 CET; 1 weeks 0 days ago
Main PID: 1589 (code=exited, status=255)
Feb 25 17:03:32 linux-5h37 systemd[1]: Starting os-autoinst openvswitch helper...
Feb 25 17:03:34 linux-5h37 systemd[1]: Started os-autoinst openvswitch helper.
Feb 25 17:03:34 linux-5h37 os-autoinst-openvswitch[1589]: can't parse bridge local port IP at /usr/lib/os-autoinst/os-autoinst-openvswitch line 37.
Feb 25 17:03:34 linux-5h37 systemd[1]: os-autoinst-openvswitch.service: Main process exited, code=exited, status=255/n/a
Feb 25 17:03:34 linux-5h37 systemd[1]: os-autoinst-openvswitch.service: Unit entered failed state.
Feb 25 17:03:34 linux-5h37 systemd[1]: os-autoinst-openvswitch.service: Failed with result 'exit-code'.
Updated by coolo about 5 years ago
- Project changed from openQA Project to openQA Infrastructure
Updated by asmorodskyi about 5 years ago
example of failed jobs :
https://openqa.suse.de/tests/2517409
https://openqa.suse.de/tests/2517271
https://openqa.suse.de/tests/2517288
Updated by nicksinger about 5 years ago
- Status changed from New to In Progress
- Assignee set to nicksinger
Updated by nicksinger about 5 years ago
- Status changed from In Progress to Feedback
There was some old service-file floating around on this worker:
QA-Power8-5-kvm.qa.suse.de:
# /usr/lib/systemd/system/os-autoinst-openvswitch.service
QA-Power8-4-kvm.qa.suse.de:
# /usr/lib/systemd/system/os-autoinst-openvswitch.service
powerqaworker-qam-1:
# /usr/lib/systemd/system/os-autoinst-openvswitch.service
malbec.arch.suse.de:
# /usr/lib/systemd/system/os-autoinst-openvswitch.service
grenache-1.qa.suse.de:
# /usr/lib/systemd/system/os-autoinst-openvswitch.service
openqaworker8.suse.de:
# /usr/lib/systemd/system/os-autoinst-openvswitch.service
openqaworker7.suse.de:
# /usr/lib/systemd/system/os-autoinst-openvswitch.service
openqaworker5.suse.de:
# /usr/lib/systemd/system/os-autoinst-openvswitch.service
openqaworker2.suse.de:
# /usr/lib/systemd/system/os-autoinst-openvswitch.service
openqaworker6.suse.de:
# /usr/lib/systemd/system/os-autoinst-openvswitch.service
openqaworker9.suse.de:
# /usr/lib/systemd/system/os-autoinst-openvswitch.service
openqa.suse.de:
Unit os-autoinst-openvswitch.service is not loaded: No such file or directory
openqaworker3.suse.de:
# /etc/systemd/system/os-autoinst-openvswitch.service
openqaworker-arm-1.suse.de:
# /usr/lib/systemd/system/os-autoinst-openvswitch.service
openqaw1.qa.suse.de:
# /usr/lib/systemd/system/os-autoinst-openvswitch.service
openqaw2.qa.suse.de:
# /usr/lib/systemd/system/os-autoinst-openvswitch.service
openqaworker-arm-2.suse.de:
# /usr/lib/systemd/system/os-autoinst-openvswitch.service
openqaworker13.suse.de:
# /usr/lib/systemd/system/os-autoinst-openvswitch.service
This file had wrong dependencies on network:
[Unit]
Description=os-autoinst openvswitch helper
BindsTo=openvswitch.service
After=openvswitch.service
Before=openqa-worker.target salt-minion.service SuSEfirewall2.service
I had the impression of wrong deps after restarting the service by hand. Before it showed the auto-generated hostname in the logs:
Feb 25 17:03:32 linux-5h37 systemd[1]: Starting os-autoinst openvswitch helper...
While it shows the actual hostname after a manual restart:
Mar 06 10:00:37 openqaworker3 systemd[1]: Starting os-autoinst openvswitch helper...
and indeed, a systemctl cat os-autoinst-openvswitch
showed that it actually uses /etc/systemd/system/os-autoinst-openvswitch.service
instead of /usr/lib/systemd/system/os-autoinst-openvswitch.service
.
I've moved the old service-file away to /root/poo48674_old_service.os-autoinst-openvswitch.service
and did a systemctl daemon-reload
and restarted the service once again to have the new service startup this time.
Now every worker is using exactly the same service file:
openqa:~ # salt '*' cmd.run 'systemctl cat os-autoinst-openvswitch.service | head -n 1'
QA-Power8-5-kvm.qa.suse.de:
# /usr/lib/systemd/system/os-autoinst-openvswitch.service
malbec.arch.suse.de:
# /usr/lib/systemd/system/os-autoinst-openvswitch.service
powerqaworker-qam-1:
# /usr/lib/systemd/system/os-autoinst-openvswitch.service
QA-Power8-4-kvm.qa.suse.de:
# /usr/lib/systemd/system/os-autoinst-openvswitch.service
grenache-1.qa.suse.de:
# /usr/lib/systemd/system/os-autoinst-openvswitch.service
openqaworker8.suse.de:
# /usr/lib/systemd/system/os-autoinst-openvswitch.service
openqaworker6.suse.de:
# /usr/lib/systemd/system/os-autoinst-openvswitch.service
openqaworker2.suse.de:
# /usr/lib/systemd/system/os-autoinst-openvswitch.service
openqaworker7.suse.de:
# /usr/lib/systemd/system/os-autoinst-openvswitch.service
openqaworker9.suse.de:
# /usr/lib/systemd/system/os-autoinst-openvswitch.service
openqaworker5.suse.de:
# /usr/lib/systemd/system/os-autoinst-openvswitch.service
openqa.suse.de:
Unit os-autoinst-openvswitch.service is not loaded: No such file or directory
openqaworker3.suse.de:
# /usr/lib/systemd/system/os-autoinst-openvswitch.service
openqaworker-arm-1.suse.de:
# /usr/lib/systemd/system/os-autoinst-openvswitch.service
openqaw2.qa.suse.de:
# /usr/lib/systemd/system/os-autoinst-openvswitch.service
openqaw1.qa.suse.de:
# /usr/lib/systemd/system/os-autoinst-openvswitch.service
openqaworker-arm-2.suse.de:
# /usr/lib/systemd/system/os-autoinst-openvswitch.service
openqaworker13.suse.de:
# /usr/lib/systemd/system/os-autoinst-openvswitch.service
Updated by nicksinger about 5 years ago
- Status changed from Feedback to Resolved