openSUSE Project Management Tool: Issueshttps://progress.opensuse.org/https://progress.opensuse.org/themes/openSUSE/favicon/favicon.ico?15829177842021-09-26T04:34:44ZopenSUSE Project Management Tool
Redmine openQA Infrastructure - action #99288 (Resolved): [Alerting] openqaworker-arm-5 and openqaworker...https://progress.opensuse.org/issues/992882021-09-26T04:34:44ZXiaojing_liuxliu1@suse.com
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>use <code>ipmitool -I lanplus -C 3 -H ipmi.openqaworker-arm-4.qa.suse.de -U admin -P password sol activate</code> to connect arm-4, there is nothing shown.<br>
use ipmitool to connect openqaworker-arm-5, it shows</p>
<pre><code>Give root password for maintenance
(or press Control-D to continue): sulogin: cannot read /dev/ttyAMA0: Operation not permitted
[ 9426.877540] systemd-fstab-generator[3152]: x-systemd.device-timeout ignored for openqa.suse.de:/var/lib/openqa/share
</code></pre>
<a name="Acceptance-criteria"></a>
<h2 >Acceptance criteria<a href="#Acceptance-criteria" class="wiki-anchor">¶</a></h2>
<ul>
<li><strong>AC1:</strong> openqaworker-arm-4 is up and can be rebooted without problems multiple times</li>
<li><strong>AC2:</strong> Same for openqaworker-arm-5</li>
</ul>
<a name="Suggestions"></a>
<h2 >Suggestions<a href="#Suggestions" class="wiki-anchor">¶</a></h2>
<ul>
<li>DONE: Pause alerts</li>
<li>Investigate problem</li>
<li>Ensure reboot-stability with multiple reboots</li>
</ul>
<a name="Rollback-steps"></a>
<h2 >Rollback steps<a href="#Rollback-steps" class="wiki-anchor">¶</a></h2>
<ul>
<li>Unpause alerts for arm-4 and arm-5</li>
</ul>
<a name="Workaround"></a>
<h2 >Workaround<a href="#Workaround" class="wiki-anchor">¶</a></h2>
<p>used <code>ipmitool power cycle</code> to reboot arm-4 and arm-5</p>
openQA Infrastructure - action #94438 (Resolved): OSD deployment fails at 2021-06-21 because ' op...https://progress.opensuse.org/issues/944382021-06-22T04:31:31ZXiaojing_liuxliu1@suse.com
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>OSD deployment failed, reason showed:</p>
<pre><code> openqaworker-arm-2.suse.de:
Minion did not return. [Not connected]
openqaworker-arm-3.suse.de:
Minion did not return. [Not connected]
</code></pre>
<p>See details in: <a href="https://gitlab.suse.de/openqa/osd-deployment/-/jobs/466302" class="external">https://gitlab.suse.de/openqa/osd-deployment/-/jobs/466302</a></p>
<p>I have rebooted arm-2 by using <code>ipmitool power cycle</code>, but arm-3 cannot be connected with <code>ipmitool</code></p>
<pre><code> #ipmitool -I lanplus -C 3 -H openqaworker-arm-3-ipmi.suse.de chassis power status
Error: Unable to establish IPMI v2 / RMCP+ session
</code></pre>
<a name="Acceptance-criteria"></a>
<h2 >Acceptance criteria<a href="#Acceptance-criteria" class="wiki-anchor">¶</a></h2>
<ul>
<li><strong>AC1:</strong> OSD deployment continued</li>
<li><strong>AC2:</strong> both openqaworker-arm-2 and openqaworker-arm-3 are online again</li>
</ul>
openQA Infrastructure - action #94399 (Workable): No alert when arm workers are offline, alert if...https://progress.opensuse.org/issues/943992021-06-22T03:30:25ZXiaojing_liuxliu1@suse.com
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>On 2021-06-22, all arm workers (arm-1, arm-2, arm-3) couldn't be connected by using <code>ssh</code> or <code>ping</code>.<br>
But <a href="https://stats.openqa-monitor.qa.suse.de/d/4KkGdvvZk/osd-status-overview?orgId=1" class="external">https://stats.openqa-monitor.qa.suse.de/d/4KkGdvvZk/osd-status-overview?orgId=1</a> showed that all of them were <code>Online</code>.</p>
<a name="Acceptance-criteria"></a>
<h2 >Acceptance criteria<a href="#Acceptance-criteria" class="wiki-anchor">¶</a></h2>
<ul>
<li><del><strong>AC1:</strong> We can receive the alerting e-mail when arm workers down</del></li>
<li><strong>AC2:</strong> <a href="https://stats.openqa-monitor.qa.suse.de/d/4KkGdvvZk/osd-status-overview?orgId=1" class="external">https://stats.openqa-monitor.qa.suse.de/d/4KkGdvvZk/osd-status-overview?orgId=1</a> should show the correct state</li>
<li><strong>AC3:</strong> We receive alert notices for errors in telegraf on osd</li>
</ul>
<a name="Suggestions"></a>
<h2 >Suggestions<a href="#Suggestions" class="wiki-anchor">¶</a></h2>
<ol>
<li>We should look into feeding something into influxdb when the telegraf service especially on OSD shows errors or log error monitoring</li>
<li>Than one could add a dashboard/graph with an alert within Grafana using the data from <code>1.</code>.</li>
</ol>
openQA Infrastructure - action #93195 (Resolved): [Alerting] Failed systemd services alert (excep...https://progress.opensuse.org/issues/931952021-05-28T05:51:46ZXiaojing_liuxliu1@suse.com
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>Cheked the <code>systemctl --failed</code> on openqaworker-arm-1:</p>
<pre><code>xiaojing@openqaworker-arm-1:~> systemctl --failed
UNIT LOAD ACTIVE SUB DESCRIPTION
● logrotate.service loaded failed failed Rotate log files
</code></pre>
<p>The status of <code>logrotate</code> showed:</p>
<pre><code>● logrotate.service - Rotate log files
Loaded: loaded (/usr/lib/systemd/system/logrotate.service; static; vendor preset: disabled)
Active: failed (Result: exit-code) since Fri 2021-05-28 00:00:00 UTC; 5h 37min ago
Docs: man:logrotate(8)
man:logrotate.conf(5)
Main PID: 33959 (code=exited, status=1/FAILURE)
May 28 00:00:00 openqaworker-arm-1 systemd[1]: Starting Rotate log files...
May 28 00:00:00 openqaworker-arm-1 logrotate[33959]: error: failed to rename /var/log/openvswitch/ovs-vswitchd.log to /var/log/openvswitch/ovs-vswitchd.log-2 ... : Permission denied
May 28 00:00:00 openqaworker-arm-1 systemd[1]: logrotate.service: Main process exited, code=exited, status=1/FAILURE
May 28 00:00:00 openqaworker-arm-1 systemd[1]: Failed to start Rotate log files.
May 28 00:00:00 openqaworker-arm-1 systemd[1]: logrotate.service: Unit entered failed state.
May 28 00:00:00 openqaworker-arm-1 systemd[1]: logrotate.service: Failed with result 'exit-code'.
</code></pre>
<a name="Workaround"></a>
<h2 >Workaround<a href="#Workaround" class="wiki-anchor">¶</a></h2>
<p>After run <code>systemctl start logrotate.service</code>, got the [OK] email.</p>
openQA Infrastructure - action #92996 (Resolved): OSD deployment fails at 2021-05-24 because ' op...https://progress.opensuse.org/issues/929962021-05-24T08:22:36ZXiaojing_liuxliu1@suse.com
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>OSD deployment failed, reason was</p>
<pre><code> openqaworker-arm-3.suse.de:
Minion did not return. [Not connected]
</code></pre>
<p>Checked the service <code>salt-minion</code> on openqaworker-arm-3, it was not running. <br>
Run <code>systemctl start salt-minion.service</code>, but the worker rebooted automatically.</p>
openQA Infrastructure - action #92915 (Resolved): OSD deployment fails at 2021-05-21 because ' op...https://progress.opensuse.org/issues/929152021-05-21T06:41:58ZXiaojing_liuxliu1@suse.com
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>OSD deployment failed, the error message is</p>
<pre><code> openqaworker-arm-2.suse.de:
Minion did not return. [Not connected]
</code></pre>
<p>details in: <a href="https://gitlab.suse.de/openqa/osd-deployment/-/jobs/434022" class="external">https://gitlab.suse.de/openqa/osd-deployment/-/jobs/434022</a></p>
openQA Infrastructure - action #91803 (Resolved): openqaworker8 failed to reboot: [Alerting] open...https://progress.opensuse.org/issues/918032021-04-27T07:59:16ZXiaojing_liuxliu1@suse.com
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>We got an alerting email on 2021-04-25 <code>openqaworker8: host up alert</code><br>
Connected to that machine using <code>ipmitool -I lanplus -C 3 -H openqaworker8-ipmi.suse.de sol activate</code>, the content showed that:</p>
<pre><code>[SOL Session operational. Use ~? for help]
Login incorrect
Give root password for maintenance
(or press Control-D to continue):
</code></pre>
<p>Then pressed Control-D, the machine started to boot.</p>
<a name="Acceptance-criteria"></a>
<h2 >Acceptance criteria<a href="#Acceptance-criteria" class="wiki-anchor">¶</a></h2>
<ul>
<li><strong>AC1:</strong> The machine reboot successfully</li>
</ul>
openQA Infrastructure - action #90746 (Resolved): OSD deployment fails at 2021-04-07 because 'sto...https://progress.opensuse.org/issues/907462021-04-07T07:15:11ZXiaojing_liuxliu1@suse.com
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>OSD deployment failed, the error message is</p>
<pre><code>storage.qa.suse.de:
Minion did not return. [Not connected]
</code></pre>
<p>See details in: <a href="https://gitlab.suse.de/openqa/osd-deployment/-/jobs/382373" class="external">https://gitlab.suse.de/openqa/osd-deployment/-/jobs/382373</a></p>
openQA Infrastructure - action #88900 (Resolved): openqaworker13 was unreachablehttps://progress.opensuse.org/issues/889002021-02-22T06:43:18ZXiaojing_liuxliu1@suse.com
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>openqaworker13 was unreachable on 2021-02-22.</p>
<a name="Acceptance-criteria"></a>
<h2 >Acceptance criteria<a href="#Acceptance-criteria" class="wiki-anchor">¶</a></h2>
<ul>
<li><strong>AC1:</strong> machine is reboot-reliable, e.g. ensured by multiple reboots without failures after boot</li>
</ul>
<a name="Problem"></a>
<h2 >Problem<a href="#Problem" class="wiki-anchor">¶</a></h2>
<p>Cannot connect to this machine <code>ping</code>.</p>
<p>using <code>ipmitool sol activate</code> connect to this machine, shows:</p>
<pre><code>[FAILED] Failed to start save kernel crash dump.
See 'systemctl status kdump-save.service' for details.
Starting Reload Configuration from the Real Root...
[ OK ] Started Reload Configuration from the Real Root.
[ OK ] Reached target Initrd File Systems.
[ OK ] Reached target Initrd Default Target.
Starting dracut pre-pivot and cleanup hook...
[ OK ] Started dracut pre-pivot and cleanup hook.
Starting Cleaning Up and Shutting Down Daemons...
[ OK ] Stopped target Timers.
[ OK ] Stopped dracut pre-pivot and cleanup hook.
[ OK ] Stopped target Initrd Default Target.
[ OK ] Stopped target Basic System.
[ OK ] Stopped target Slices.
[ OK ] Stopped target Initrd Root Device.
[ OK ] Stopped target System Initialization.
[ OK ] Stopped Apply Kernel Variables.
[ OK ] Stopped Create Volatile Files and Directories.
[ OK ] Stopped target Local File Systems.
Unmounting /kdump/mnt1/var/crash...
Unmounting /kdump/mnt0...
Stopping udev Kernel Device Manager...
[ OK ] Stopped target Sockets.
[ OK ] Stopped Load Kernel Modules.
[ OK ] Stopped target Paths.
[ OK ] Stopped Dispatch Password Requests to Console Directory Watch.
[ OK ] Stopped target Swap.
[ OK ] Stopped target Remote File Systems.
[ OK ] Stopped target Remote File Systems (Pre).
[ OK ] Stopped dracut initqueue hook.
[ OK ] Stopped udev Coldplug all Devices.
[ OK ] Stopped udev Kernel Device Manager.
[ OK ] Unmounted /kdump/mnt1/var/crash.
[ OK ] Started Cleaning Up and Shutting Down Daemons.
[ OK ] Stopped Create Static Device Nodes in /dev.
[ OK ] Stopped Create list of required sta…vice nodes for the current kernel.
[ OK ] Stopped dracut pre-udev hook.
[ OK ] Stopped dracut cmdline hook.
[ OK ] Stopped dracut ask for additional cmdline parameters.
[ OK ] Closed udev Control Socket.
[ OK ] Closed udev Kernel Socket.
Starting Cleanup udevd DB...
[ OK ] Started Cleanup udevd DB.
[ OK ] Reached target Switch Root.
Starting Switch Root...
[FAILED] Failed to start Switch Root.
See 'systemctl status initrd-switch-root.service' for details.
Starting Setup Virtual Console...
[ OK ] Unmounted /kdump/mnt0.
[ OK ] Stopped File System Check on /dev/d…3d875-0e24-49a9-8dc8-ddf8cf83f04e.
[ OK ] Started Setup Virtual Console.
[ OK ] Started Emergency Shell.
[ OK ] Reached target Emergency Mode.
Generating "/run/initramfs/rdsosreport.txt"
Entering emergency mode. Exit the shell to continue.
Type "journalctl" to view system logs.
You might want to save "/run/initramfs/rdsosreport.txt" to a USB stick or /boot
after mounting them and attach it to a bug report.
Give root password for maintenance
(or press Control-D to continue):
[716615.675131] BTRFS info (device sda2): disk space caching is enabled
[716615.675134] BTRFS info (device sda2): has skinny extents
[716614.923484] dracut-cmdline[499]: mv: cannot stat '/etc/multipath.conf.kdump': No such file or directory
Unable to ioctl(KDSETLED) -- are you not on the console? (Inappropriate ioctl for device)
Extracting dmesg
-------------------------------------------------------------------------------
The dmesg log is saved to /kdump/mnt1/var/crash/2021-02-22-04:16/dmesg.txt.
makedumpfile Completed.
-------------------------------------------------------------------------------
Saving dump using makedumpfile
-------------------------------------------------------------------------------
Copying data : [100.0 %] / eta: 0s
The dumpfile is saved to /kdump/mnt1/var/crash/2021-02-22-04:16/vmcore.
makedumpfile Completed.
-------------------------------------------------------------------------------
</code></pre>
<p>Then did <code>ipmitool power cycle</code>. The workers on that host works well for now.</p>
<a name="Suggestions"></a>
<h2 >Suggestions<a href="#Suggestions" class="wiki-anchor">¶</a></h2>
<ul>
<li>Remove from salt</li>
<li>Identify problem</li>
<li>Ensure stability with multiple reboots</li>
<li>Add back to salt after stable</li>
</ul>
<a name="Workaround"></a>
<h2 >Workaround<a href="#Workaround" class="wiki-anchor">¶</a></h2>
<p>power cycle over IPMI</p>
openQA Infrastructure - action #88474 (Resolved): All workers on powerqaworker-qam-1 are offlinehttps://progress.opensuse.org/issues/884742021-02-08T03:16:54ZXiaojing_liuxliu1@suse.com
<p>All workers on powerqaworker-qam-1 are offline on OSD. We could see that from: <a href="https://openqa.suse.de/admin/workers" class="external">https://openqa.suse.de/admin/workers</a></p>
<p>Cannot ping it, <code>ipmitool</code> works well.</p>
<p>checked the network, the result shows that:</p>
<pre><code>/ # ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
2: enP1p3s0f0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq qlen 1000
link/ether 6c:ae:8b:69:21:74 brd ff:ff:ff:ff:ff:ff
3: enP1p3s0f1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop qlen 1000
link/ether 6c:ae:8b:69:21:75 brd ff:ff:ff:ff:ff:ff
4: enP1p3s0f2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop qlen 1000
link/ether 6c:ae:8b:69:21:76 brd ff:ff:ff:ff:ff:ff
5: enP1p3s0f3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop qlen 1000
link/ether 6c:ae:8b:69:21:77 brd ff:ff:ff:ff:ff:ff
6: enP3p9s0f0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop qlen 1000
link/ether 6c:ae:8b:69:20:20 brd ff:ff:ff:ff:ff:ff
7: enP3p9s0f1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop qlen 1000
link/ether 6c:ae:8b:69:20:21 brd ff:ff:ff:ff:ff:ff
8: enP3p9s0f2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop qlen 1000
link/ether 6c:ae:8b:69:20:22 brd ff:ff:ff:ff:ff:ff
9: enP3p9s0f3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop qlen 1000
link/ether 6c:ae:8b:69:20:23 brd ff:ff:ff:ff:ff:ff
10: tunl0@NONE: <NOARP> mtu 1480 qdisc noop qlen 1
link/ipip 0.0.0.0 brd 0.0.0.0
/ # ethtool enP1p3s0f0
Settings for enP1p3s0f0:
Supported ports: [ TP ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Half 1000baseT/Full
Supported pause frame use: No
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Half 1000baseT/Full
Advertised pause frame use: Symmetric
Advertised auto-negotiation: Yes
Speed: Unknown!
Duplex: Unknown! (255)
Port: Twisted Pair
PHYAD: 1
Transceiver: internal
Auto-negotiation: on
MDI-X: Unknown
Supports Wake-on: g
Wake-on: g
Current message level: 0x000000ff (255)
drv probe link timer ifdown ifup rx_err tx_err
Link detected: no
/ # SOL session closed by BMC
</code></pre>
<p>cable is unplugged?</p>
openQA Infrastructure - action #81020 (Resolved): QA-Power8-4-kvm start failed since reboot on 20...https://progress.opensuse.org/issues/810202020-12-14T04:41:25ZXiaojing_liuxliu1@suse.com
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>The host can be ping:</p>
<pre><code>liuxiaojing@linux-90dc:~/Downloads> ping qa-power8-4-kvm.qa.suse.de
PING QA-Power8-4-kvm.qa.suse.de (10.162.6.201) 56(84) bytes of data.
64 bytes from 10.162.6.201 (10.162.6.201): icmp_seq=1 ttl=62 time=153 ms
64 bytes from 10.162.6.201 (10.162.6.201): icmp_seq=2 ttl=62 time=153 ms
</code></pre>
<p>run <code>ipmitool sol activate</code>, it shows <br>
<code>exiting the petitboot. Type 'exit' to rerun</code> <br>
Then I typied <code>exit</code>, this menu showed:</p>
<pre><code>Petitboot (dev.20160310) 8348-21C 684F75A
──────────────────────────────────────────────────────────────────────────────
sled11-sp2-gm
sles11-sp1-vmware
sles10
sles10-sp1
sles10-sp2
sles10-sp3
sles11
sles11-sp1
sles11-sp2-vmware
sles10-sp4
sles-12-gm
harddisk
System information
*System configuration
Language
Rescan devices
Retrieve config from URL
Exit to shell
──────────────────────────────────────────────────────────────────────────────
Enter=accept, e=edit, n=new, x=exit, l=language, h=help
</code></pre>
<p>Seems it starts from the network? I don't know how to choose, and if it should start from network.</p>
<p>Then I run <code>sol deactivate</code> to close the ipmitool connection. (Failed to quit by using <code>~.</code>)</p>
openQA Infrastructure - action #80768 (Resolved): All workers in grenache-1 are broken at 2020-12-07https://progress.opensuse.org/issues/807682020-12-07T09:53:55ZXiaojing_liuxliu1@suse.com
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>All workers in <code>grenache-1</code> are broken, please see more details in <a href="https://openqa.suse.de/admin/workers" class="external">https://openqa.suse.de/admin/workers</a>, search <code>grenache-1</code>.<br>
The broken reason shows that:<br>
<code>Cache service info error: Can't connect: Address family for hostname not supported</code>.</p>
openQA Infrastructure - action #80656 (Resolved): OSD deployment failed at 2020-12-02 because 'ma...https://progress.opensuse.org/issues/806562020-12-02T07:08:35ZXiaojing_liuxliu1@suse.com
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>OSD deployment failed, the error message is </p>
<pre><code>malbec.arch.suse.de:
Minion did not return. [Not connected]
</code></pre>
<a name="Workaround"></a>
<h2 >Workaround<a href="#Workaround" class="wiki-anchor">¶</a></h2>
<p>Because malbec is down for a bit longer, so run<br>
<code>salt-key -d malbec.arch.suse.de</code> to remove malbec from salt-key on OSD. Then re-run the deployment.</p>
<p>When malbec is back, we should add it into salt-key.</p>
openQA Infrastructure - action #75448 (Resolved): OSD deployment fails because openqaworker-arm-3...https://progress.opensuse.org/issues/754482020-10-28T07:31:29ZXiaojing_liuxliu1@suse.com
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>openqaworker-arm-3.suse.de cannot be connected, so the OSD deployment failed.</p>
<pre><code>openqaworker-arm-3.suse.de:
Minion did not return. [Not connected]
</code></pre>
<a name="Workaround"></a>
<h2 >Workaround<a href="#Workaround" class="wiki-anchor">¶</a></h2>
<p>Remove openqaworker-arm-3.suse.de from salt keys by running command<br>
<code>salt-key -d openqaworker-arm-3.suse.de</code></p>
<a name="Suggestion"></a>
<h2 >Suggestion<a href="#Suggestion" class="wiki-anchor">¶</a></h2>
<p>when openqaworker-arm-3.suse.de is online, please add it into salt keys</p>
openQA Infrastructure - action #75067 (Resolved): save_needle minion task fails because "Your acc...https://progress.opensuse.org/issues/750672020-10-22T07:50:38ZXiaojing_liuxliu1@suse.com
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>save_needle minion task failed on OSD, the error message is:</p>
<pre><code> "result" => {
"error" => "<strong>Failed to save system-role-default-system-textmode-20201021.</strong><br><pre>Unable to push Git commit: remote: GitLab: Your account has been blocked.\nTo gitlab.suse.de:openqa/os-autoinst-needles-sles.git\n ! [remote rejected] master -> master (pre-receive hook declined)\nerror: failed to push some refs to 'gitlab\@gitlab.suse.de:openqa/os-autoinst-needles-sles.git'</pre>"
}
</code></pre>
<p>See more details in <a href="https://openqa.suse.de/minion/jobs?id=783852" class="external">https://openqa.suse.de/minion/jobs?id=783852</a></p>