https://progress.opensuse.org/https://progress.opensuse.org/themes/openSUSE/favicon/favicon.ico?15829177842020-10-27T08:08:03ZopenSUSE Project Management ToolopenQA Tests - action #75364: [qac] job incompletes with auto_review:"(?s)Error connecting to VNC server.*openqa.*-xen.*backend died: socket does not exist. Probably your backend instance could not start or died.*"https://progress.opensuse.org/issues/75364?journal_id=3437592020-10-27T08:08:03Zokurzokurz@suse.com
<ul><li><strong>Tags</strong> set to <i>qac, jeos, xen</i></li><li><strong>Project</strong> changed from <i>openQA Project</i> to <i>openQA Tests</i></li><li><strong>Subject</strong> changed from <i>job incompletes with auto_review:"backend died: socket does not exist. Probably your backend instance could not start or died.*"</i> to <i>[qac] job incompletes with auto_review:"(?s)Error connecting to VNC server.*openqa.*-xen.*backend died: socket does not exist. Probably your backend instance could not start or died.*"</i></li><li><strong>Category</strong> set to <i>Bugs in existing tests</i></li><li><strong>Assignee</strong> set to <i>jlausuch</i></li><li><strong>Priority</strong> changed from <i>Low</i> to <i>High</i></li></ul><p>Maintenance of special worker addendums including the Xen hypervisor host is ouf of scope for SUSE QA Tools (<a href="https://progress.opensuse.org/projects/qa/wiki#Out-of-scope" class="external">https://progress.opensuse.org/projects/qa/wiki#Out-of-scope</a>). As the test is about "JeOS" I will assign to QAC team.</p>
<p><a class="user active user-mention" href="https://progress.opensuse.org/users/32669">@Xiaojing_liu</a> I suggest to be a bit more specific with the auto_review regex to prevent matching on too many generic issues, e.g. if that symptom also appears for other backends or machines.</p>
openQA Tests - action #75364: [qac] job incompletes with auto_review:"(?s)Error connecting to VNC server.*openqa.*-xen.*backend died: socket does not exist. Probably your backend instance could not start or died.*"https://progress.opensuse.org/issues/75364?journal_id=3438552020-10-27T08:33:20Zokurzokurz@suse.com
<ul><li><strong>Has duplicate</strong> <i><a class="issue tracker-4 status-6 priority-3 priority-lowest closed" href="/issues/71236">action #71236</a>: job incompletes with auto_review:"backend died: Error connecting to VNC server <openqaw5-xen.qa.suse.de:5901>: IO::Socket::INET: connect: Connection refused"</i> added</li></ul> openQA Tests - action #75364: [qac] job incompletes with auto_review:"(?s)Error connecting to VNC server.*openqa.*-xen.*backend died: socket does not exist. Probably your backend instance could not start or died.*"https://progress.opensuse.org/issues/75364?journal_id=3444042020-10-28T15:09:44Zjlausuchjalausuch@suse.com
<ul></ul><p>What am I supposed to do with this? Just tag the failed test I suppose :) <br>
This looks like the same nature of <a href="https://progress.opensuse.org/issues/71236" class="external">https://progress.opensuse.org/issues/71236</a></p>
openQA Tests - action #75364: [qac] job incompletes with auto_review:"(?s)Error connecting to VNC server.*openqa.*-xen.*backend died: socket does not exist. Probably your backend instance could not start or died.*"https://progress.opensuse.org/issues/75364?journal_id=3449382020-10-29T21:52:19Zokurzokurz@suse.com
<ul></ul><p>jlausuch wrote:</p>
<blockquote>
<p>What am I supposed to do with this? Just tag the failed test I suppose :)</p>
</blockquote>
<p>Well, this is about incomplete jobs so "failed" tests would not really fit. And with the "auto_review" keyword in the subject line there should be no need to manually label builds ("tagging" is for builds). See more about auto-review on <a href="https://gitlab.suse.de/openqa/auto-review/" class="external">https://gitlab.suse.de/openqa/auto-review/</a> if you are interested</p>
<p>So what I can suggest to do is do a couple of things:</p>
<ul>
<li>Prevent the test from incompleting and turn them into failed by making sure that consoles are only tried to be activated when they are present. What specifically happened here I do not know. But in the complete test scenario <a href="https://openqa.suse.de/tests/latest?arch=x86_64&distri=sle&flavor=JeOS-for-kvm-and-xen&machine=svirt-xen-hvm&test=jeos-filesystem_xenhvm&version=15-SP3" class="external">https://openqa.suse.de/tests/latest?arch=x86_64&distri=sle&flavor=JeOS-for-kvm-and-xen&machine=svirt-xen-hvm&test=jeos-filesystem_xenhvm&version=15-SP3</a> I see only one incomplete and then previous and later tests were fine again, at least not incompleting. So the issue is likely not that severe.</li>
<li>Improve the backend that is used here so that the error feedback in case of problems is better than the "connection refused" and incomplete.</li>
<li>Help to improve how the hypervisor hosts are managed, maintained, monitored and alerting.</li>
</ul>
<p>The QE Tools team is happy to offer help but does not have the capacity to improve the "special worker addendums" that are used for tests here themselves.</p>
<blockquote>
<p>This looks like the same nature of <a class="issue tracker-4 status-6 priority-3 priority-lowest closed" title="action: job incompletes with auto_review:"backend died: Error connecting to VNC server <openqaw5-xen.qa.s... (Rejected)" href="https://progress.opensuse.org/issues/71236">#71236</a></p>
</blockquote>
<p>yes, this is why I rejected <a class="issue tracker-4 status-6 priority-3 priority-lowest closed" title="action: job incompletes with auto_review:"backend died: Error connecting to VNC server <openqaw5-xen.qa.s... (Rejected)" href="https://progress.opensuse.org/issues/71236">#71236</a> as a duplicate of this ticket. But you should not point back to the duplicate ticket otherwise you are caught in an infinite circle ;)</p>
openQA Tests - action #75364: [qac] job incompletes with auto_review:"(?s)Error connecting to VNC server.*openqa.*-xen.*backend died: socket does not exist. Probably your backend instance could not start or died.*"https://progress.opensuse.org/issues/75364?journal_id=3470652020-11-05T10:32:05Zcfconradcfamullaconrad@suse.com
<ul><li><strong>Priority</strong> changed from <i>High</i> to <i>Normal</i></li></ul><p>Set to prio Normal, as this was later run's didn't show this incomplete behaviors anymore.</p>
openQA Tests - action #75364: [qac] job incompletes with auto_review:"(?s)Error connecting to VNC server.*openqa.*-xen.*backend died: socket does not exist. Probably your backend instance could not start or died.*"https://progress.opensuse.org/issues/75364?journal_id=3500172020-11-12T10:48:21Zmloviskamloviska@suse.com
<ul><li><strong>Tags</strong> changed from <i>qac, jeos, xen</i> to <i>qac, xen</i></li><li><strong>Status</strong> changed from <i>New</i> to <i>Blocked</i></li><li><strong>Assignee</strong> deleted (<del><i>jlausuch</i></del>)</li><li><strong>Parent task</strong> set to <i>#64279</i></li></ul><p>A priori we need to resolve <a href="https://progress.opensuse.org/issues/64279" class="external">OS upgrade</a>. Let me set this one as blocked.</p>
openQA Tests - action #75364: [qac] job incompletes with auto_review:"(?s)Error connecting to VNC server.*openqa.*-xen.*backend died: socket does not exist. Probably your backend instance could not start or died.*"https://progress.opensuse.org/issues/75364?journal_id=3500292020-11-12T11:41:43Zokurzokurz@suse.com
<ul></ul><p>please be aware that <a class="issue tracker-4 status-3 priority-4 priority-default closed parent" title="action: [virtualization][OS upgrade] upgrade xen host openqaw5-xen.qa.suse.de (Resolved)" href="https://progress.opensuse.org/issues/64279">#64279</a> is out of scope of the SUSE QE Tools team, see <a href="https://progress.opensuse.org/projects/qa/wiki/Wiki#Out-of-scope" class="external">https://progress.opensuse.org/projects/qa/wiki/Wiki#Out-of-scope</a> .</p>
openQA Tests - action #75364: [qac] job incompletes with auto_review:"(?s)Error connecting to VNC server.*openqa.*-xen.*backend died: socket does not exist. Probably your backend instance could not start or died.*"https://progress.opensuse.org/issues/75364?journal_id=4263372021-07-13T12:38:51Zjlausuchjalausuch@suse.com
<ul><li><strong>Status</strong> changed from <i>Blocked</i> to <i>Resolved</i></li></ul><p>After XEN host update done by Martin, we haven't observed this issue.</p>