openSUSE Project Management Tool: Issueshttps://progress.opensuse.org/https://progress.opensuse.org/themes/openSUSE/favicon/favicon.ico?15829177842023-09-01T03:20:18ZopenSUSE Project Management Tool
Redmine openQA Tests - action #134969 (New): [qe-core] Require latest released quarterly update images fo...https://progress.opensuse.org/issues/1349692023-09-01T03:20:18Zxlaixlai@suse.com
<p>We recently get a suggestion and agreed in discussions with Heiko Rommel, Jan Stehlik, and Antoine Ginies, that when testing released products, it is better to use <code>latest QU image + scc updates</code>.</p>
<p>For detailed requirements,</p>
<ul>
<li>all supported products' latest QU image are needed,
<ul>
<li>sle12sp3~sle12sp5 (12sp3 has teradata tests after LTSS ends, 12sp4 seems ending but not final confirmed yet)</li>
<li>sle15sp2-sle15sp5 (and sle15sp6/7 when they are released)</li>
</ul></li>
<li>available in three locations' mirror server and pxe server so that local openqa servers' jobs can access them-- DE NUE2 lab(OSD), PRG1 lab (<a href="http://openqa.qam.suse.cz/" class="external">http://openqa.qam.suse.cz/</a> resides in) and BeiJing lab (<a href="http://openqa.qa2.suse.asia/" class="external">http://openqa.qa2.suse.asia/</a> resides in)
<ul>
<li>the QU images can be accessible via http(s) in repository way (unpacked from iso), and are put with FIXED names like SLExxSPx-QU-LATEST </li>
<li>the latest QU for all products are added in pxe server</li>
</ul></li>
</ul>
<p>I am not sure if this ticket's scope can be fully covered by openqa infrastructure (some should). It will help a lot, if anyone familiar with it can help point out which parts should go to which team. We will then split the ticket properly. Thanks in advance!</p>
openQA Project - action #90818 (Resolved): [openqa][tool] Not able to get group_overview json out...https://progress.opensuse.org/issues/908182021-04-08T06:15:56Zxlaixlai@suse.com
<p>I meet a new issue when querying osd group_overview json format output after osd deployed the latest update this Wednesday(April 7, 2021). It had been working for long time before it. Would you please help to have a look? It blocks our openqa job retrigger tool. Thanks a lot!</p>
<p><strong>Details:</strong></p>
<ul>
<li><p>Before update, works well, output is similar with below(did not paste original because output is very long):<br>
2021-04-05 01:00:01, general_utils.py, DEBUG:Command_args for subprocess to run is: ['/usr/share/openqa/script/client', '--json-output', '--host', '<a href="http://openqa.q">http://openqa.q</a><br>
a2.suse.asia', '--apikey', 'keyxxxx', '--apisecret', 'secretxxx', '--apibase', '/', 'group_overview/20', 'limit_builds=1']<br>
2021-04-05 01:00:02, general_utils.py, DEBUG:Command output is: b'{\n "comments" : [],\n "build_results" : [\n {\n "escaped_build" : "162_7",\n<br><br>
"failed" : 3,\n "build" : "162.7",\n "softfailed" : 0,\n "escaped_id" : "15_SP3-162_7",\n "passed" : 116,\n "version" : "15-SP<br>
3",\n "escaped_version" : "15_SP3",\n "oldest" : "2021-03-15T10:51:17",\n "unfinished" : 0,\n "distris" : {\n "sle" : 1\n<br><br>
},\n "key" : "15-SP3-162.7",\n "labeled" : 0,\n "skipped" : 1,\n "total" : 120,\n "reviewed" : "",\n "all_passed" : ""\n<br><br>
}\n ],\n "max_jobs" : 120,\n "pinned_comments" : [],\n "group" : {\n "id" : 20,\n "is_parent" : null,\n "name" : "SLE-15-SP3-Performance",\n<br><br>
"rendered_description" : null\n },\n "description" : null\n}\n'<br>
2021-04-05 01:00:02, general_utils.py, DEBUG:After json load, data is {'comments': [], 'build_results': [{'escaped_build': '162_7', 'failed': 3, 'build': '162.7',<br>
'softfailed': 0, 'escaped_id': '15_SP3-162_7', 'passed': 116, 'version': '15-SP3', 'escaped_version': '15_SP3', 'oldest': '2021-03-15T10:51:17', 'unfinished': 0, 'distris':<br>
{'sle': 1}, 'key': '15-SP3-162.7', 'labeled': 0, 'skipped': 1, 'total': 120, 'reviewed': '', 'all_passed': ''}], 'max_jobs': 120, 'pinned_comments': [], 'group': {'id': 20<br>
, 'is_parent': None, 'name': 'SLE-15-SP3-Performance', 'rendered_description': None}, 'description': None}<br>
2021-04-05 01:00:02, openqa_job_retrigger.py, INFO:Group 20's latest build is 162.7, will handle this build.</p></li>
<li><p>After update, return reports error:<br>
qa2-dhcp-53:~ # /usr/share/openqa/script/client --json-output --host <a href="http://openqa.suse.de">http://openqa.suse.de</a> --apikey --apisecret --apibase / group_overview/263 limit_builds=1<br><br>
hash- or arrayref expected (not a simple scalar, use allow_nonref to allow this) at /usr/share/openqa/script/client line 174.<br>
qa2-dhcp-53:~ #</p></li>
</ul>
QA - action #78444 (Closed): [virtualization] alice to deletehttps://progress.opensuse.org/issues/784442020-11-20T06:27:34Zxlaixlai@suse.comopenQA Infrastructure - action #55868 (Resolved): USB device is removed from gonzo.qa.suse.de.https://progress.opensuse.org/issues/558682019-08-23T08:20:01Zxlaixlai@suse.com
<p>We equipped on some servers some usb device to test virtualization pvusb feature. However today from a failure test <a href="https://openqa.suse.de/tests/3280807">https://openqa.suse.de/tests/3280807</a>, we noticed that the logitech mouse usb device on it was accidentally removed by someone and resulted in test failure.</p>
<p>SUT_IP: gonzo.qa.suse.de<br>
Missing USB device: <br>
Bus 003 Device 002: ID 046d:c001 Logitech, Inc. N48/M-BB48/M-UK96A [FirstMouse Plus]</p>
<p><a class="user active user-mention" href="https://progress.opensuse.org/users/24624">@nicksinger</a> Would you please help to find out who moved it and ask him/her to recover it? I know it is not easy since so many people have access to lab. If you can not find it, would you please help to remind people to not move our usb devices? Really appreciate if you can help paste some warning msg on these machines.</p>
<p>SUTs with usb device:<br>
21:<br>
WORKER_CLASS: virt-mm-64bit-ipmi,virt-pvusb-64bit-ipmi,64bit-ipmi<br>
IPMI_HOSTNAME: sp.kermit.qa.suse.de<br>
IPMI_PASSWORD: ADMIN<br>
IPMI_USER: ADMIN<br>
MAX_JOB_TIME: 32000<br>
SUT_IP: kermit.qa.suse.de<br>
PVUSB_DEVICE: Keyboard<br>
22: //missing now<br>
WORKER_CLASS: virt-mm-64bit-ipmi,virt-pvusb-64bit-ipmi,64bit-ipmi<br>
IPMI_HOSTNAME: sp.gonzo.qa.suse.de<br>
IPMI_PASSWORD: ADMIN<br>
IPMI_USER: ADMIN<br>
MAX_JOB_TIME: 32000<br>
SUT_IP: gonzo.qa.suse.de<br>
PVUSB_DEVICE: Logitech<br>
23:<br>
WORKER_CLASS: virt-mm-64bit-ipmi,virt-pvusb-64bit-ipmi,64bit-ipmi<br>
IPMI_HOSTNAME: sp.fozzie.qa.suse.de<br>
IPMI_PASSWORD: ADMIN<br>
IPMI_USER: ADMIN<br>
MAX_JOB_TIME: 32000<br>
SUT_IP: fozzie-1.qa.suse.de<br>
PVUSB_DEVICE: Logitech<br>
24:<br>
WORKER_CLASS: virt-mm-64bit-ipmi,virt-pvusb-64bit-ipmi,64bit-ipmi<br>
IPMI_HOSTNAME: sp.scooter.qa.suse.de<br>
IPMI_PASSWORD: ADMIN<br>
IPMI_USER: ADMIN<br>
MAX_JOB_TIME: 32000<br>
SUT_IP: scooter-1.qa.suse.de</p>
<pre><code> 16:
WORKER_CLASS: virt-pvusb-64bit-ipmi,64bit-ipmi
IPMI_HOSTNAME: 10.162.28.200
IPMI_PASSWORD: qatesting
IPMI_USER: admin
MAX_JOB_TIME: 32000
SUT_IP: 10.162.2.75
PVUSB_DEVICE: Winbond
17: # new worker quinn.qa.suse.de
WORKER_CLASS: virt-pvusb-64bit-ipmi,64bit-ipmi
IPMI_HOSTNAME: 10.162.3.87
IPMI_PASSWORD: ADMIN
IPMI_USER: ADMIN
SUT_IP: 10.162.2.87
MAX_JOB_TIME: 32000
PVUSB_DEVICE: Verbatim
</code></pre> openQA Project - action #46583 (Closed): [tools][dependency jobs][scheduling] Request to support ...https://progress.opensuse.org/issues/465832019-01-24T06:19:30Zxlaixlai@suse.com
<p>Currently for the dependent jobs with START_AFTER relationship, eg A START_AFTER B, openqa scheduling can ensure that A is after B, but can not promise that:</p>
<ul>
<li>A is just the next job of B(possible a job C(os installation) in between) </li>
<li>A and B run on the same worker</li>
</ul>
<p>This is fine for jobs on qemu workers. However for ipmi jobs, the above two constraints are needed. For example, a quite common request on ipmi machines is that first a host installation is done, and then launch various kinds of tests. This is a good example that should use START_AFTER relationship from a common tool's view. However due to above limitations, we can not do it on openqa.</p>
<p>With more and more users on ipmi workers to test physical machines, eg sebastian's team , QAM team and sle-virt team, we strongly recommend to add support for the stricter START_AFTER on ipmi machines. Especially currently we kind of partially rely on it to improve the openqa test efficiency to get prepare for future parallel developing products tested the same time on openqa.</p>
<p><a class="user active user-mention" href="https://progress.opensuse.org/users/15">@coolo</a>, <a class="user active user-mention" href="https://progress.opensuse.org/users/24624">@nicksinger</a>, would you please help to evaluate whether this is a reasonable request? If yes, would you please share the plan for it? Look forward to your reply!</p>
openQA Project - action #45749 (Resolved): [tools][scheduler] Multi-machine jobs with higher prio...https://progress.opensuse.org/issues/457492019-01-07T03:20:21Zxlaixlai@suse.com
<p>During 15sp1 beta1 test, the multi-machine jobs(2 sut) in virtualization job groups can not get workers to kickoff job until some low priority single machine jobs finish. This delays especially acceptance test(not able to finish within 24 hours).</p>
<p><a href="https://openqa.suse.de/tests/overview?distri=sle&version=15-SP1&build=125.1&groupid=115" class="external">https://openqa.suse.de/tests/overview?distri=sle&version=15-SP1&build=125.1&groupid=115</a> <br>
<a href="https://openqa.suse.de/tests/overview?distri=sle&version=15-SP1&build=125.1&groupid=213" class="external">https://openqa.suse.de/tests/overview?distri=sle&version=15-SP1&build=125.1&groupid=213</a> </p>
<p>I did not open a ticket when I found it, because I understood that:<br>
although they were with higher priority in our group, but possibly other job group ipmi jobs have even higher priority. So they got the machine first, and they did not finish at the same time, so our multi-machine jobs still could not be started and other lower priority single machine jobs started.</p>
<p><a class="user active user-mention" href="https://progress.opensuse.org/users/17668">@okurz</a> commented that openqa tool should make some enhancement for it in <a href="https://gitlab.suse.de/openqa/salt-pillars-openqa/merge_requests/148/" class="external">https://gitlab.suse.de/openqa/salt-pillars-openqa/merge_requests/148/</a>. </p>
<p>Please help to evaluate. Really appreciate.</p>
openQA Project - action #44978 (Rejected): [ipmi unstability] jobs got blue screen making openqa ...https://progress.opensuse.org/issues/449782018-12-11T03:25:09Zxlaixlai@suse.com
<p>The blue screen looks like <a href="https://openqa.suse.de/tests/2319257#step/boot_from_pxe/25" class="external">https://openqa.suse.de/tests/2319257#step/boot_from_pxe/25</a>, and when it happens, needle matching mechenism of openqa will not work(creating new needle based on the blue screen not work and with such needle when retrigger jobs, job will fail at any needle matching).</p>
openQA Infrastructure - action #44795 (Rejected): [tools] setup failure: Cache service not availa...https://progress.opensuse.org/issues/447952018-12-06T02:31:53Zxlaixlai@suse.com
<p>In latest build 108.1, most of virtualization jobs(several tens) fail as incomplete job with similar reason:</p>
<p>[2018-12-05T11:34:06.0806 CET] [info] +++ setup notes +++<br>
[2018-12-05T11:34:06.0806 CET] [info] start time: 2018-12-05 10:34:06<br>
[2018-12-05T11:34:06.0807 CET] [info] running on grenache-1:17 (Linux 4.4.138-94.39-default #1 SMP Mon Jun 18 13:27:26 UTC 2018 (baa07f9) ppc64le)<br>
[2018-12-05T11:34:06.0812 CET] [warn] job is missing files, releasing job<br>
[2018-12-05T11:34:06.0842 CET] [info] +++ worker notes +++<br>
[2018-12-05T11:34:06.0842 CET] [info] end time: 2018-12-05 10:34:06<br>
[2018-12-05T11:34:06.0842 CET] [info] result: setup failure: Cache service not available.<br>
[2018-12-05T11:34:06.0843 CET] [info] uploading autoinst-log.txt</p>
<p>Fail job link:<br>
<a href="https://openqa.suse.de/tests/2303781/file/autoinst-log.txt" class="external">https://openqa.suse.de/tests/2303781/file/autoinst-log.txt</a></p>
openQA Project - action #39356 (Resolved): [worker] openqaworker13:3 is not a proper 64bit-ipmi w...https://progress.opensuse.org/issues/393562018-08-09T05:51:43Zxlaixlai@suse.com
<p>The SUT that openqaworker13:3 controls is not properly set for pxe boot. It fails 3 virtualization jobs at boot_from_pxe. Would you please remove it from 64bit-ipmi workers firstly before the pxe setting ready?</p>
<p>Failure job link:<br>
<a href="https://openqa.suse.de/tests/1911033" class="external">https://openqa.suse.de/tests/1911033</a></p>
openQA Project - action #39074 (Closed): [OpenQA][API] upload_logs fails from time to time.https://progress.opensuse.org/issues/390742018-08-02T03:35:53Zxlaixlai@suse.com
<p>On ipmi backend, upload_logs fails from time to time in jobs. Fail reason is "curl: fail creating formpost data"</p>
<p>It happened since build 0312. </p>
<p>Failure job(build 0315 has several such failure jobs):<br>
<a href="http://openqa.suse.de/tests/1880375#step/update_package/24" class="external">http://openqa.suse.de/tests/1880375#step/update_package/24</a></p>
openQA Project - action #16602 (Resolved): Comment with unicode characters do not work because of...https://progress.opensuse.org/issues/166022017-02-09T02:24:15Zxlaixlai@suse.com
<a name="observation"></a>
<h2 >observation<a href="#observation" class="wiki-anchor">¶</a></h2>
<p>An error message is reported when I updated result for a build on virtualization group on openqa.suse.de</p>
<p>What was typed on comment:</p>
<pre><code>**Date:** <2017-02-09>
**Build:** 0234
**Arch:** x86_64
**Status: <font color="red">Red</font>
**New Product bugs:**
* No
**Requirement:**
* one more ipmi machine: poo#15376
**Existing Product bugs:**
* sles12sp3 pv guest installation: bsc#1012005
* pvusb bug on sles12sp3 xen: bsc#1023948
**Existing openqa issues:**
* guest migration job fail/multi-machine: poo#15376
* ipmi ikvm unstable: poo#14338
* ipmi serial console unstable: poo#13914
* ipmi keyboard simulator unstable: poo#12982 poo#16088
</code></pre>
<a name="steps-to-reproduce"></a>
<h2 >steps to reproduce<a href="#steps-to-reproduce" class="wiki-anchor">¶</a></h2>
<ul>
<li>on an openQA instance with enabled rabbitmq plugin</li>
<li>type character '/' in any comment</li>
<li>observe reported error</li>
</ul>
<a name="problem"></a>
<h2 >problem<a href="#problem" class="wiki-anchor">¶</a></h2>
<p>H1. Mojo::RabbitMQ::Client does not handle unicode input correctly<br>
<a href="https://github.com/inway/mojo-rabbitmq-client/commit/c043447d4fa53a1456515c4def1c4d1367cd87b0#commitcomment-20829251" class="external">https://github.com/inway/mojo-rabbitmq-client/commit/c043447d4fa53a1456515c4def1c4d1367cd87b0#commitcomment-20829251</a><br>
H2. <code>lib/OpenQA/WebAPI/Plugin/AMQP.pm</code> does not forward unicode correctly to Mojo::RabbitMQ::Client</p>
<a name="workaround"></a>
<h2 >workaround<a href="#workaround" class="wiki-anchor">¶</a></h2>
<p>only type ascii within comments.</p>
openQA Infrastructure - action #16088 (Rejected): [ipmi] Do not respond to send_key.https://progress.opensuse.org/issues/160882017-01-19T06:02:20Zxlaixlai@suse.com
<p>In job <a href="https://openqa.suse.de/tests/716647#step/reboot_and_wait_up_normal2/3" class="external">https://openqa.suse.de/tests/716647#step/reboot_and_wait_up_normal2/3</a>, we use send_key_until_needle_match api to select xen grub menuentry, however after catching a not matching screen, and a send_key is sent, screen does not change.</p>
openQA Tests - action #15376 (Resolved): Need one more stable ipmi physical machine for virtualiz...https://progress.opensuse.org/issues/153762016-12-08T08:44:37Zxlaixlai@suse.com
<p>Requirement: <br>
Please add one more ipmi machine with stable ipmi.</p>
<p>Background:<br>
Currently, we only get one stable ipmi machine which is pointed to by worker openqaw2:1. As we add more virtualization tests, we have multi machine jobs involving, including guest migration which was added into openqa already. Also we plan to add virt-v2v, post copy migration tests before sle12sp3 milestone starts. They all require 2 ipmi machines.</p>
openQA Tests - action #13916 (Rejected): [ipmi] What should be typed by type_string is not typed ...https://progress.opensuse.org/issues/139162016-09-27T01:38:14Zxlaixlai@suse.com
<p>In build <a href="https://openqa.suse.de/tests/overview?distri=sle&version=12-SP2&build=2144&groupid=46" class="external">https://openqa.suse.de/tests/overview?distri=sle&version=12-SP2&build=2144&groupid=46</a>, two tests failed because what should be typed by type_string is not actually typed at all to screen . There is no ipmi connection issue reported from either serial0.txt or autoinst log. But from screenshot, the string required is not typed at all.</p>
<p>Detailed failure:<br>
<a href="https://openqa.suse.de/tests/589661#comments" class="external">https://openqa.suse.de/tests/589661#comments</a><br>
<a href="https://openqa.suse.de/tests/589666#comments" class="external">https://openqa.suse.de/tests/589666#comments</a></p>
openQA Tests - action #13914 (New): [qe-core][functional][ipmi] wait_serial does not get expected...https://progress.opensuse.org/issues/139142016-09-27T01:36:22Zxlaixlai@suse.com
<p>Test failed due to wait_serial does not get output. From serial0.txt, the ipmi session was already closed due to "excess errors received"</p>
<p>Failure step:<br>
<a href="https://openqa.suse.de/tests/587781#step/install_package/4" class="external">https://openqa.suse.de/tests/587781#step/install_package/4</a></p>
<p>Build link:<br>
<a href="https://openqa.suse.de/tests/overview?distri=sle&version=12-SP2&build=2141&groupid=46" class="external">https://openqa.suse.de/tests/overview?distri=sle&version=12-SP2&build=2141&groupid=46</a></p>
<p>Serial output link:<br>
<a href="https://openqa.suse.de/tests/587781/file/serial0.txt" class="external">https://openqa.suse.de/tests/587781/file/serial0.txt</a></p>
<p>Key serial output errors:</p>
<pre><code>[�[0;32m OK �[0m] Started Serial Getty on ttyS1.
[�[0;32m OK �[0m] Started Serial Getty on hvc0.
Starting X Display Manager...
[�[0;32m OK �[0m] Started Getty on tty1.
[�[0;32m OK �[0m] Reached target Login Prompts.
[�[0;32m OK �[0m] Started /etc/init.d/after.local Compatibility.
[�[0;32m OK �[0m] Started Load dom0 backend drivers.
Starting The Xen xenstore...
[SOL established]
[error received]: excess errors received
[closing the connection]
</code></pre>