openSUSE Project Management Tool: Issueshttps://progress.opensuse.org/https://progress.opensuse.org/themes/openSUSE/favicon/favicon.ico?15829177842023-09-01T03:20:18ZopenSUSE Project Management Tool
Redmine openQA Tests - action #134969 (New): [qe-core] Require latest released quarterly update images fo...https://progress.opensuse.org/issues/1349692023-09-01T03:20:18Zxlaixlai@suse.com
<p>We recently get a suggestion and agreed in discussions with Heiko Rommel, Jan Stehlik, and Antoine Ginies, that when testing released products, it is better to use <code>latest QU image + scc updates</code>.</p>
<p>For detailed requirements,</p>
<ul>
<li>all supported products' latest QU image are needed,
<ul>
<li>sle12sp3~sle12sp5 (12sp3 has teradata tests after LTSS ends, 12sp4 seems ending but not final confirmed yet)</li>
<li>sle15sp2-sle15sp5 (and sle15sp6/7 when they are released)</li>
</ul></li>
<li>available in three locations' mirror server and pxe server so that local openqa servers' jobs can access them-- DE NUE2 lab(OSD), PRG1 lab (<a href="http://openqa.qam.suse.cz/" class="external">http://openqa.qam.suse.cz/</a> resides in) and BeiJing lab (<a href="http://openqa.qa2.suse.asia/" class="external">http://openqa.qa2.suse.asia/</a> resides in)
<ul>
<li>the QU images can be accessible via http(s) in repository way (unpacked from iso), and are put with FIXED names like SLExxSPx-QU-LATEST </li>
<li>the latest QU for all products are added in pxe server</li>
</ul></li>
</ul>
<p>I am not sure if this ticket's scope can be fully covered by openqa infrastructure (some should). It will help a lot, if anyone familiar with it can help point out which parts should go to which team. We will then split the ticket properly. Thanks in advance!</p>
openQA Tests - action #55100 (Resolved): [hyperv] Need to delete ISO with issue when checksum doe...https://progress.opensuse.org/issues/551002019-08-05T07:00:31Zxlaixlai@suse.com
<p>All vmware&hyperv jobs in virtualization job group fail by similar error <a href="https://openqa.suse.de/tests/3204511#step/welcome/10" class="external">https://openqa.suse.de/tests/3204511#step/welcome/10</a>.</p>
<p>Need to find why checksum does not match and fix it.</p>
openQA Tests - action #42899 (Resolved): Compilation failed in require at /usr/bin/isotovideo lin...https://progress.opensuse.org/issues/428992018-10-25T03:55:39Zxlaixlai@suse.com
<p>This makes openqa tests fail a lot in all job groups.</p>
<p>Job link:<br>
<a href="https://openqa.suse.de/tests/2208950/file/autoinst-log.txt" class="external">https://openqa.suse.de/tests/2208950/file/autoinst-log.txt</a></p>
<p>Key failure:<br>
[2018-10-25T00:43:11.0643 CEST] [debug] scheduling welcome tests/installation/welcome.pm<br>
[2018-10-25T00:43:11.0644 CEST] [debug] scheduling keyboard_selection tests/installation/keyboard_selection.pm<br>
[2018-10-25T00:43:11.0645 CEST] [debug] scheduling skip_registration tests/installation/skip_registration.pm<br>
[2018-10-25T00:43:11.0647 CEST] [debug] scheduling addon_products_sle tests/installation/addon_products_sle.pm<br>
Undefined subroutine &version_utils::is_server called at /var/lib/openqa/cache/tests/sle/lib/version_utils.pm line 320.<br>
Compilation failed in require at /usr/bin/isotovideo line 229.<br>
543: EXIT 1<br>
[2018-10-25T00:43:14.0671 CEST] [info] Isotovideo exit status: 1<br>
[2018-10-25T00:43:14.0698 CEST] [info] +++ worker notes +++<br>
[2018-10-25T00:43:14.0699 CEST] [info] end time: 2018-10-24 22:43:14<br>
[2018-10-25T00:43:14.0699 CEST] [info] result: died</p>
openQA Infrastructure - action #40544 (Resolved): [OpenQA][IPMI backend] IPMI worker can not surv...https://progress.opensuse.org/issues/405442018-09-04T07:29:51Zxlaixlai@suse.com
<p>We have two dell machines, vh003.qa2.suse.asia and vh004.qa2.suse.asia. When they are binded with ipmi worker, the jobs on those two machines can not survive reboot. For example, after host installation when it boots to the new os, the sol console can only get black screen, not reactive at all. So does any other simple reboot.</p>
<p>After debugging by john and jerry, it is found that the reset_console operation leads to this failure because the existing sol console connection is not properly cleaned up and result in failure in the new sol console setup.</p>
<p>John and jerry also have their 2 proposals as solutions which are open for discussions. I will let them describe in more details in later comments.</p>
openQA Project - action #40148 (Resolved): [OpenQA][64bit-ipmi worker] Three online 64bit-ipmi wo...https://progress.opensuse.org/issues/401482018-08-23T03:20:40Zxlaixlai@suse.com
<p>Currently there are 3 online 64bit-ipmi workers(openqaw1:2, openqaworker2:24, openqaworker2:25) which haven't take jobs for over 10 hours. However there are a lot queened jobs in virtualization job group in 12sp4 build 0351. Only 3 other workers are taking jobs.</p>
<p>Seems openqa scheduler has some problem? This delays tests a lot. Build 0351 has been running for about 2 days, but virtualization still has not finished yet. Generally it should finish within 1 day.</p>
openQA Tests - action #37408 (Resolved): [openqa]Please add sle15 GM image to http://openqa.nue.s...https://progress.opensuse.org/issues/374082018-06-15T02:52:33Zxlaixlai@suse.com
<p>In sle12sp4, virtualization job group will includes tests on sle15 hosts, please help to add sle15 GM image to <a href="http://openqa.nue.suse.com/assets/repo/fixed/" class="external">http://openqa.nue.suse.com/assets/repo/fixed/</a>, like sle12sp3/sle11sp4 .</p>
openQA Tests - action #25504 (Resolved): Support for changing test variables including needles du...https://progress.opensuse.org/issues/255042017-09-22T02:56:16Zxlaixlai@suse.com
<p>In sle15, many test files of installation uses function sle_version_at_least to do check for product version, so as to differenciate sle15 new behaviors from older products. </p>
<p>However, this may be not correct. </p>
<p>Since sle12sp2, a new var INSTALL_TO_OTHERS was introduced for tests that needed to install system to a different product(mostly former product), for example in sles12sp3 release, virtualization job group in openqa.suse.de already had tests that actually installed system to sle11sp4, sle12sp1, sle12sp2, like <a href="https://openqa.suse.de/tests/1058378" class="external">https://openqa.suse.de/tests/1058378</a>. </p>
<p>So IMHO, sle15 different behavior should be done for ONLY those <em>really</em> install to a product at least 15, that is for those tests with INSTALL_TO_OTHERS, we should check the version that it want to install is at least 15, and for those without INSTALL_TO_OTHERS, just do what sle_version_at_least does. This is my first thing to talk. Do you agree?</p>
<p>Currently in utils, there are two apis, install_to_other_at_least and sle_version_at_least to check versions for both situations. So the sle15 different behavior should be done only for a condition like "((install_this_version() && sle_version_at_least('15')) || install_to_other_at_least('15'))", rather than simple sle_version_at_least('15'). </p>
<p>However the above complex condition writing is absolutely not a good idea. So here comes the second topic. Solution to it. What comes up to me are:<br>
option 1) add a new api to stand the above complex checking for versions, using the apis install_to_other_at_least and sle_version_at_least, and replace all the usages of the api to the new one, and also notify all test writers about it.<br>
option 2) keep the api, but rewrite it to represent the complex checking. Good thing is no need to change various usages. All test writers does not needs to know about the change, and continue to regard the api as the assumed perfect one.</p>
<p>I personally prefer option 2. What's your choice? Or any other solutions you can figure out?</p>
openQA Tests - action #19994 (Resolved): Time out when install guest on sles12sp1 xen.https://progress.opensuse.org/issues/199942017-06-22T07:15:37Zxlaixlai@suse.com
<p>Fail job link <a href="https://openqa.suse.de/tests/1002358#step/guest_installation_run/12" class="external">https://openqa.suse.de/tests/1002358#step/guest_installation_run/12</a>.<br>
Failed due to timeout, but it should not. Something unexpected happen. Please fix it.</p>
openQA Tests - action #19992 (Resolved): [sle][virtualization]Guest installation on sle11sp4 both...https://progress.opensuse.org/issues/199922017-06-22T07:12:55Zxlaixlai@suse.com
<p>Refer to <a href="https://openqa.suse.de/tests/1004374" class="external">https://openqa.suse.de/tests/1004374</a> and <a href="https://openqa.suse.de/tests/1004375" class="external">https://openqa.suse.de/tests/1004375</a>, the guest installation is skipped. </p>
<p>Suspect to be automation issue in qa_lib_virtauto, in vm-install.sh script.</p>
openQA Tests - action #19744 (Resolved): Guest installation on sles12sp3 kvm fails due to not get...https://progress.opensuse.org/issues/197442017-06-12T05:17:21Zxlaixlai@suse.com
<p>Job link <a href="https://openqa.suse.de/tests/991716" class="external">https://openqa.suse.de/tests/991716</a>, key error screen <a href="https://openqa.suse.de/tests/991716#step/update_package/12" class="external">https://openqa.suse.de/tests/991716#step/update_package/12</a></p>
openQA Tests - action #19602 (Resolved): [tools] New ipmi backend can not switch back to sol cons...https://progress.opensuse.org/issues/196022017-06-06T03:37:58Zxlaixlai@suse.com
<p>In the new ipmi backend, root-ssh console can be successfully selected. And running commands under root-ssh console works fine too. However before reboot, when switching back to sol console, it fails. Job link <a href="http://147.2.212.149/tests/26" class="external">http://147.2.212.149/tests/26</a>. From autoinst log, it fails in finding sol needle. However the screen does not change to sol relative window, it just stays on the root-ssh screen window, so it is not able to create the relative needle.</p>
<p>Code to switch to sol:<br>
sub use_sol_serial_console() {<br>
console('root-ssh')->disable;<br>
select_console('sol'); //fails here<br>
set_var('SERIALDEV', '');<br>
bmwqemu::save_vars();<br>
resetup_console;<br>
}</p>
<p>Also I have a question, when using root-ssh console, the key board interaction is via ssh pipe, the serial console is from the simulated fifo device. When switching back to sol console, the key board interaction should be re-established via ipmi, and serial console should also via ipmiconsole, right? </p>
openQA Tests - action #17936 (Resolved): [tools]Add jenkins job to automatically trigger ipmi mai...https://progress.opensuse.org/issues/179362017-03-24T03:32:22Zxlaixlai@suse.com
<p>From the virtualization practice using ipmi machine, I find that the ipmi machine's ipmi main board become unstable after using constantly for some time like two weeks or long. It made the test results unreliable. I had to restart ipmi manually and retrigger the job after I found it.</p>
<p>So maybe we can add a new jenkins job to do such a thing. To avoid the restart affecting running jobs, it is better to be done on two constraints, one is to meet the time period check( machine is used for over two weeks or so), second is before a new build trigger starts.</p>
<p>@oliver, do you agree with such a resolution? Can we write a jenkins job to do it?</p>
openQA Tests - action #17694 (Resolved): [tools][virtualization]Please add sles11sp3/sles11sp4 GM...https://progress.opensuse.org/issues/176942017-03-14T08:19:31Zxlaixlai@suse.com
<p>We decide to deploy all guest installation tests into openqa and use it for sles12sp3 verification. It contains scenarios on sles11sp3/sp4 host. So we will need these images for host installation via pxe on ipmi machine.</p>
openQA Tests - action #13914 (New): [qe-core][functional][ipmi] wait_serial does not get expected...https://progress.opensuse.org/issues/139142016-09-27T01:36:22Zxlaixlai@suse.com
<p>Test failed due to wait_serial does not get output. From serial0.txt, the ipmi session was already closed due to "excess errors received"</p>
<p>Failure step:<br>
<a href="https://openqa.suse.de/tests/587781#step/install_package/4" class="external">https://openqa.suse.de/tests/587781#step/install_package/4</a></p>
<p>Build link:<br>
<a href="https://openqa.suse.de/tests/overview?distri=sle&version=12-SP2&build=2141&groupid=46" class="external">https://openqa.suse.de/tests/overview?distri=sle&version=12-SP2&build=2141&groupid=46</a></p>
<p>Serial output link:<br>
<a href="https://openqa.suse.de/tests/587781/file/serial0.txt" class="external">https://openqa.suse.de/tests/587781/file/serial0.txt</a></p>
<p>Key serial output errors:</p>
<pre><code>[�[0;32m OK �[0m] Started Serial Getty on ttyS1.
[�[0;32m OK �[0m] Started Serial Getty on hvc0.
Starting X Display Manager...
[�[0;32m OK �[0m] Started Getty on tty1.
[�[0;32m OK �[0m] Reached target Login Prompts.
[�[0;32m OK �[0m] Started /etc/init.d/after.local Compatibility.
[�[0;32m OK �[0m] Started Load dom0 backend drivers.
Starting The Xen xenstore...
[SOL established]
[error received]: excess errors received
[closing the connection]
</code></pre> openQA Tests - action #12926 (Resolved): Can not get ipmi serial output correctly.https://progress.opensuse.org/issues/129262016-07-29T07:11:57Zxlaixlai@suse.com
<p>CASE FAILURES:</p>
<p>Job link: <a href="https://openqa.suse.de/tests/overview?distri=sle&version=12-SP2&build=2016&groupid=46">https://openqa.suse.de/tests/overview?distri=sle&version=12-SP2&build=2016&groupid=46</a></p>
<p>Testsuites:</p>
<p>*gi-guest_sles12sp2-on-host_sles12sp2-kvm:</p>
<p>Fail stage: host installation<br>
Fail reason: what are typed out by type_string is not complete<br>
At the last step of installation , 'install and reboot', command 'save_y2logs /tmp/y2logs.tar.bz2 ' is typed to 'save_y2lgs /tmp/y2logs.tar.bz2' which results to 'command y2lgs not found' and exit </p>
<p>*gi-guest_sles11sp4-on-host_sles12sp2-kvm</p>
<p>Fail stage: run the first script after login on the newly installed system<br>
Fail reason: Seems unstable serial<br>
Did not get the check code on serial for upload logs, but the previous check code of our virt code was got.</p>
<p>Logs:<br>
20:01:14.5759 24169 <<< testapi::type_string(string='(source /usr/share/qa/virtautolib/lib/virtlib;update_virt_rpms off on off 2>&1 | tee /tmp/update_virt_rpms.log ; echo CMD_FINISHED-45346) 2>&1 | tee -a /dev/ttyS1<br>
', max_interval=250)<br>
20:01:26.2177 Debug: /var/lib/openqa/share/tests/sle/tests/virt_autotest/install_package.pm:44 called virt_autotest_base::execute_script_run<br>
20:01:26.2182 24169 <<< testapi::wait_serial(regex='CMD_FINISHED-45346', timeout=7200)<br>
20:02:02.2913 24169 >>> testapi::wait_serial: CMD_FINISHED-45346: ok</p>
<p>20:02:02.4125 24169 <<< testapi::type_string(string='curl --form upload=@/tmp/update_virt_rpms.log --form upname=install_package-update_virt_rpms.log <a href="http://10.162.0.12:20013/JI6Pc6TclLAoD8J5/uploadlog/update_virt_rpms.log">http://10.162.0.12:20013/JI6Pc6TclLAoD8J5/uploadlog/update_virt_rpms.log</a>; echo CUMYV-$?- > /dev/ttyS1', max_interval=250)<br>
20:02:15.7595 Debug: /var/lib/openqa/share/tests/sle/tests/virt_autotest/install_package.pm:47 called testapi::upload_logs<br>
20:02:15.7597 24169 <<< testapi::send_key(key='ret')<br>
20:02:15.9613 Debug: /var/lib/openqa/share/tests/sle/tests/virt_autotest/install_package.pm:47 called testapi::upload_logs<br>
20:02:15.9617 24169 <<< testapi::wait_serial(regex='CUMYV-\d+-', timeout=90)<br>
20:06:46.7188 24169 >>> testapi::wait_serial: CUMYV-\d+-: fail</p>
<p>*gi-guest_sles12sp1-on-host_sles12sp2-kvm</p>
<p>Fail stage: run the first script after login on the newly installed system<br>
Fail reason: seems serial decode problem, the serial output after the first reboot following installation are unknown character like @</p>
<p>Logs:</p>
<p>21:46:58.4681 Debug: /var/lib/openqa/share/tests/sle/tests/virt_autotest/guest_installation_run.pm:14 called testapi::script_output<br>
21:46:58.4683 24846 <<< testapi::script_run(name='curl -f -v <a href="http://10.162.0.12:20013/84p34tmDuUI38f1X/current_script">http://10.162.0.12:20013/84p34tmDuUI38f1X/current_script</a> > /tmp/scriptOdTh6.sh; echo dTNlY-$?- > /dev/ttyS1', wait=0)<br>
21:46:58.4684 Debug: /var/lib/openqa/share/tests/sle/tests/virt_autotest/guest_installation_run.pm:14 called testapi::script_output<br>
21:46:58.4685 24846 <<< testapi::type_string(string='curl -f -v <a href="http://10.162.0.12:20013/84p34tmDuUI38f1X/current_script">http://10.162.0.12:20013/84p34tmDuUI38f1X/current_script</a> > /tmp/scriptOdTh6.sh; echo dTNlY-$?- > /dev/ttyS1', max_interval=250)<br>
21:47:06.8974 Debug: /var/lib/openqa/share/tests/sle/tests/virt_autotest/guest_installation_run.pm:14 called testapi::script_output<br>
21:47:06.8976 24846 <<< testapi::send_key(key='ret')<br>
21:47:07.0989 Debug: /var/lib/openqa/share/tests/sle/tests/virt_autotest/guest_installation_run.pm:14 called testapi::script_output<br>
21:47:07.0992 24846 <<< testapi::wait_serial(regex='dTNlY-\d+-', timeout=90)<br>
21:51:38.4801 24846 >>> testapi::wait_serial: dTNlY-\d+-: fail</p>
<p>*gi-guest_sles12sp2-on-host_sles12sp1-kvm</p>
<p>Fail stage: run the first script after login on the newly installed system<br>
Fail reason: seems serial decode problem, the serial output after the first reboot following installation are unknown character like @</p>
<p>Logs:</p>
<p>00:06:36.8151 26070 <<< testapi::script_run(name='curl -f -v <a href="http://10.162.0.12:20013/vkUq43upmdPkRsHZ/current_script">http://10.162.0.12:20013/vkUq43upmdPkRsHZ/current_script</a> > /tmp/scriptOdTh6.sh; echo FSXE_-$?- > /dev/ttyS1', wait=0)<br>
00:06:36.8151 Debug: /var/lib/openqa/share/tests/sle/tests/virt_autotest/guest_installation_run.pm:14 called testapi::script_output<br>
00:06:36.8153 26070 <<< testapi::type_string(string='curl -f -v <a href="http://10.162.0.12:20013/vkUq43upmdPkRsHZ/current_script">http://10.162.0.12:20013/vkUq43upmdPkRsHZ/current_script</a> > /tmp/scriptOdTh6.sh; echo FSXE_-$?- > /dev/ttyS1', max_interval=250)<br>
00:06:45.2365 Debug: /var/lib/openqa/share/tests/sle/tests/virt_autotest/guest_installation_run.pm:14 called testapi::script_output<br>
00:06:45.2368 26070 <<< testapi::send_key(key='ret')<br>
00:06:45.4379 Debug: /var/lib/openqa/share/tests/sle/tests/virt_autotest/guest_installation_run.pm:14 called testapi::script_output<br>
00:06:45.4381 26070 <<< testapi::wait_serial(regex='FSXE_-\d+-', timeout=90)<br>
00:11:16.9354 26070 >>> testapi::wait_serial: FSXE_-\d+-: fail</p>