openSUSE Project Management Tool: Issueshttps://progress.opensuse.org/https://progress.opensuse.org/themes/openSUSE/favicon/favicon.ico?15829177842024-03-21T09:51:42ZopenSUSE Project Management Tool
Redmine openQA Tests - action #157657 (Feedback): PackageHub module activation while using SCC-proxyhttps://progress.opensuse.org/issues/1576572024-03-21T09:51:42Zjlausuchjalausuch@suse.com
<a name="Problem-statement"></a>
<h2 >Problem statement:<a href="#Problem-statement" class="wiki-anchor">¶</a></h2>
<p>When registering the system using ProxySCC, activating the PackageHub module returns repos in OSD that don't exist.<br>
This is expected, because the proxySCC replaces the repo URLs given by the real SCC by local openQA asset URL.</p>
<p>Normally, we use the proxySCC in the openQA jobs with <code>SCC_URL</code> variable, this will be used by the code to make the <code>SUSEConnect</code> call like this:</p>
<p><code>SUSEConnect -r $REGCODE --url http://all-67.1.proxy.scc.suse.de</code></p>
<p>This will make that all the repos returned look like this:<br>
<code>http://openqa.suse.de/assets/repo/...</code></p>
<p>And enabling any module, will have the same effect, for example PackageHub:</p>
<pre><code>
susetest:~ # suseconnect -p PackageHub/15.6/x86_64
Registering system to registration proxy http://all-67.1.proxy.scc.suse.de
Updating system details on http://all-67.1.proxy.scc.suse.de ...
Activating PackageHub 15.6 x86_64 ...
-> Adding service to system ...
-> Installing release package ...
Successfully registered system
</code></pre>
<p>And the repo will point to the wrong place:</p>
<pre><code>18 | SUSE_Package_Hub_15_SP6_x86_64:SLE-Module-Packagehub-Subpackages15-SP6-Pool | SLE-Module-Packagehub-Subpackages15-SP6-Pool | Yes | (r ) Yes | No | http://openqa.suse.de/assets/repo/SLE-15-SP6-Module-Packagehub-Subpackages-POOL-x86_64-Build67.1-Media1/
</code></pre>
<a name="Solution"></a>
<h2 >Solution<a href="#Solution" class="wiki-anchor">¶</a></h2>
<p>To cope with this, we would need to sync the PackageHub module for EVERY build we have for 15-SP6. Another solution would be to tweak ProxySCC code to return the real URL instead of local assets.</p>
<p>BUT there is an easier way:<br>
1) Register the system using the usual proxySCC</p>
<pre><code>susetest:~ # SUSEConnect -r $REGCODE --url http://all-67.1.proxy.scc.suse.de
Registering system to registration proxy http://all-67.1.proxy.scc.suse.de
Announcing system to http://all-67.1.proxy.scc.suse.de ...
Activating SLES 15.6 x86_64 ...
-> Adding service to system ...
Activating sle-module-basesystem 15.6 x86_64 ...
-> Adding service to system ...
-> Installing release package ...
Activating sle-module-server-applications 15.6 x86_64 ...
-> Adding service to system ...
-> Installing release package ...
Activating sle-module-python3 15.6 x86_64 ...
-> Adding service to system ...
-> Installing release package ...
Successfully registered system
</code></pre>
<p>2) Activate the desired module using <code>--url</code> pointing to real SCC:</p>
<pre><code>susetest:~ # SUSEConnect -p PackageHub/15.6/x86_64 --url https://scc.suse.com
Registering system to SUSE Customer Center
Updating system details on https://scc.suse.com ...
Activating PackageHub 15.6 x86_64 ...
-> Adding service to system ...
-> Installing release package ...
Successfully registered system
</code></pre>
<p>3) And voilá, you will get the repos from REAL SCC:</p>
<pre><code>susetest:~ # zypper lr -u |grep -i PackageHub-15-SP6-Pool
22 | SUSE_Package_Hub_15_SP6_x86_64:SUSE-PackageHub-15-SP6-Pool | SUSE-PackageHub-15-SP6-Pool | Yes | (r ) Yes | No | https://updates.suse.com/SUSE/Backports/SLE-15-SP6_x86_64/product?RMMzfgZAuv8Hy903n30FsW6hbWs5KExVGGIwF1w5nRMdkliFvRBscHmgomWpHwFbG8l2C7cmrcZead-ejTo_f0NKbts7ZNwGBw0WtMwuWeKXxK2mgknqmbcYZ22y3RB3rThU1AMZTw
</code></pre> openQA Tests - action #151738 (Feedback): [qe-core] Enable systemd-testsuite in 15-SP6https://progress.opensuse.org/issues/1517382023-11-30T09:41:09Zjlausuchjalausuch@suse.com
<a name="Background"></a>
<h2 >Background<a href="#Background" class="wiki-anchor">¶</a></h2>
<p>There is an important update for systemd in 15-SP6 to v254: <a href="https://jira.suse.com/browse/PED-4846" class="external">https://jira.suse.com/browse/PED-4846</a><br>
Some important partners are a bit worried about this update and have asked us to please make sure this doesn't break any application (e.g. SAP applications).<br>
We can't ensure 3rd party applications will be ok with this update, but what we can do is to provide good coverage for systemd testing (of course, SAP part will be covered by SAP squad). </p>
<p><code>systemd-testsuite</code> has been integrated in openQA and enabled in TW by Martin Loviska when he was on rotation. e.g. <a href="https://openqa.opensuse.org/tests/3760221" class="external">https://openqa.opensuse.org/tests/3760221</a><br>
It was also enabled in 15-SP4 and SP5 after GA, but the package is already in 15-SP6 channel <a href="http://download.suse.de/download/ibs/SUSE:/SLE-15-SP6:/GA/standard/x86_64/systemd-testsuite-254.5-150600.1.2.x86_64.rpm" class="external">http://download.suse.de/download/ibs/SUSE:/SLE-15-SP6:/GA/standard/x86_64/systemd-testsuite-254.5-150600.1.2.x86_64.rpm</a></p>
<a name="Acceptance-criteria"></a>
<h2 >Acceptance criteria<a href="#Acceptance-criteria" class="wiki-anchor">¶</a></h2>
<ul>
<li><code>systemd-testsuite</code> is scheduled in 15-SP6 functional group.</li>
</ul>
<p>Related slack thead: <a href="https://suse.slack.com/archives/C02CSAZLAR4/p1701087663691749" class="external">https://suse.slack.com/archives/C02CSAZLAR4/p1701087663691749</a></p>
openQA Tests - action #129601 (Feedback): bootloader_uefi fails too often in aarch64https://progress.opensuse.org/issues/1296012023-05-19T05:49:51Zjlausuchjalausuch@suse.com
<p><a href="https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/tests/installation/bootloader_uefi.pm" class="external">installation/bootloader_uefi.pm</a> times out quite often because openQA does not have time to enter uefi grub2 configuration screen before the 10s grub timeout.</p>
<p>e.g. <br>
<a href="https://openqa.suse.de/tests/11157624#step/bootloader_uefi/11">https://openqa.suse.de/tests/11157624#step/bootloader_uefi/11</a><br>
<a href="https://openqa.suse.de/tests/11157620#step/bootloader_uefi/11">https://openqa.suse.de/tests/11157620#step/bootloader_uefi/11</a><br>
<a href="https://openqa.suse.de/tests/11157622#step/bootloader_uefi/11">https://openqa.suse.de/tests/11157622#step/bootloader_uefi/11</a></p>
<p>The culprit is that the first <code>bootloader-grub2</code> takes too long to match.</p>
<p>This is a successful run:</p>
<pre><code>[2023-05-17T22:29:48.135632+02:00] [debug] [pid:78975] <<< testapi::assert_screen(mustmatch=[
"bootloader-shim-import-prompt",
"bootloader-grub2"
], timeout=90)
[2023-05-17T22:29:49.928958+02:00] [warn] [pid:79369] !!! backend::baseclass::check_asserted_screen: check_asserted_screen took 0.82 seconds for 14 candidate needles - make your needles more specific
[2023-05-17T22:29:49.929428+02:00] [debug] [pid:79369] no match: 89.0s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-17T22:29:50.104691+02:00] [debug] [pid:79369] no change: 88.0s
[2023-05-17T22:29:50.588948+02:00] [debug] [pid:79369] no match: 88.0s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-17T22:29:51.105278+02:00] [debug] [pid:79369] no change: 87.0s
[2023-05-17T22:29:51.589146+02:00] [debug] [pid:79369] no match: 87.0s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-17T22:29:52.105972+02:00] [debug] [pid:79369] no change: 86.0s
[2023-05-17T22:29:52.589983+02:00] [debug] [pid:79369] no match: 86.0s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-17T22:30:00.232487+02:00] [warn] [pid:79369] !!! backend::baseclass::check_asserted_screen: check_asserted_screen took 7.13 seconds for 14 candidate needles - make your needles more specific
[2023-05-17T22:30:00.232953+02:00] [debug] [pid:79369] no match: 85.0s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-17T22:30:00.248237+02:00] [debug] [pid:79369] no change: 77.8s
[2023-05-17T22:30:00.733227+02:00] [debug] [pid:79369] no match: 77.8s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-17T22:30:01.236512+02:00] [debug] [pid:79369] no change: 76.9s
[2023-05-17T22:30:01.743323+02:00] [warn] [pid:79369] !!! backend::baseclass::check_asserted_screen: check_asserted_screen took 0.51 seconds for 14 candidate needles - make your needles more specific
[2023-05-17T22:30:01.743772+02:00] [debug] [pid:79369] no match: 76.9s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-17T22:30:09.594737+02:00] [warn] [pid:79369] !!! backend::baseclass::check_asserted_screen: check_asserted_screen took 7.36 seconds for 14 candidate needles - make your needles more specific
[2023-05-17T22:30:09.595227+02:00] [debug] [pid:79369] no match: 75.9s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-17T22:30:10.332028+02:00] [warn] [pid:79369] !!! backend::baseclass::check_asserted_screen: check_asserted_screen took 0.71 seconds for 14 candidate needles - make your needles more specific
[2023-05-17T22:30:10.332520+02:00] [debug] [pid:79369] no match: 68.5s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-17T22:30:11.328004+02:00] [warn] [pid:79369] !!! backend::baseclass::check_asserted_screen: check_asserted_screen took 0.72 seconds for 14 candidate needles - make your needles more specific
[2023-05-17T22:30:11.328472+02:00] [debug] [pid:79369] no match: 67.5s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-17T22:30:12.205373+02:00] [warn] [pid:79369] !!! backend::baseclass::check_asserted_screen: check_asserted_screen took 0.59 seconds for 14 candidate needles - make your needles more specific
[2023-05-17T22:30:12.205867+02:00] [debug] [pid:79369] no match: 66.5s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-17T22:30:19.704649+02:00] [warn] [pid:79369] !!! backend::baseclass::check_asserted_screen: check_asserted_screen took 7.09 seconds for 14 candidate needles - make your needles more specific
[2023-05-17T22:30:19.705116+02:00] [debug] [pid:79369] no match: 65.5s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-17T22:30:20.417002+02:00] [debug] [pid:78975] >>> testapi::_handle_found_needle: found bootmenu-jeos-20201204, similarity 1.00 @ 101/208
[2023-05-17T22:30:20.419694+02:00] [debug] [pid:78975] tests/installation/bootloader_uefi.pm:124 called bootloader_setup::uefi_bootmenu_params -> lib/bootloader_setup.pm:383 called testapi::send_key_until_needlematch
[2023-05-17T22:30:20.420160+02:00] [debug] [pid:78975] <<< testapi::check_screen(mustmatch="grub2-enter-edit-mode", timeout=0)
</code></pre>
<p>And this is a failed run:</p>
<pre><code>[2023-05-18T22:31:23.112731+02:00] [debug] [pid:20579] <<< testapi::assert_screen(mustmatch=[
"bootloader-shim-import-prompt",
"bootloader-grub2"
], timeout=90)
[2023-05-18T22:31:24.904170+02:00] [warn] [pid:21382] !!! backend::baseclass::check_asserted_screen: check_asserted_screen took 0.82 seconds for 14 candidate needles - make your needles more specific
[2023-05-18T22:31:24.904637+02:00] [debug] [pid:21382] no match: 89.0s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-18T22:31:25.081871+02:00] [debug] [pid:21382] no change: 88.0s
[2023-05-18T22:31:25.580143+02:00] [debug] [pid:21382] no match: 88.0s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-18T22:31:26.094924+02:00] [debug] [pid:21382] no change: 87.0s
[2023-05-18T22:31:26.569793+02:00] [debug] [pid:21382] no match: 87.0s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-18T22:31:27.083393+02:00] [debug] [pid:21382] no change: 86.0s
[2023-05-18T22:31:27.553981+02:00] [debug] [pid:21382] no match: 86.0s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-18T22:31:35.276260+02:00] [warn] [pid:21382] !!! backend::baseclass::check_asserted_screen: check_asserted_screen took 7.19 seconds for 14 candidate needles - make your needles more specific
[2023-05-18T22:31:35.276731+02:00] [debug] [pid:21382] no match: 85.0s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-18T22:31:36.083159+02:00] [warn] [pid:21382] !!! backend::baseclass::check_asserted_screen: check_asserted_screen took 0.78 seconds for 14 candidate needles - make your needles more specific
[2023-05-18T22:31:36.083646+02:00] [debug] [pid:21382] no match: 77.8s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-18T22:31:36.279755+02:00] [debug] [pid:21382] no change: 76.8s
[2023-05-18T22:31:37.020384+02:00] [warn] [pid:21382] !!! backend::baseclass::check_asserted_screen: check_asserted_screen took 0.74 seconds for 14 candidate needles - make your needles more specific
[2023-05-18T22:31:37.020854+02:00] [debug] [pid:21382] no match: 76.8s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-18T22:31:44.628366+02:00] [warn] [pid:21382] !!! backend::baseclass::check_asserted_screen: check_asserted_screen took 7.35 seconds for 14 candidate needles - make your needles more specific
[2023-05-18T22:31:44.628834+02:00] [debug] [pid:21382] no match: 75.8s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-18T22:31:45.401250+02:00] [warn] [pid:21382] !!! backend::baseclass::check_asserted_screen: check_asserted_screen took 0.74 seconds for 14 candidate needles - make your needles more specific
[2023-05-18T22:31:45.401731+02:00] [debug] [pid:21382] no match: 68.4s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-18T22:31:46.386500+02:00] [warn] [pid:21382] !!! backend::baseclass::check_asserted_screen: check_asserted_screen took 0.74 seconds for 14 candidate needles - make your needles more specific
[2023-05-18T22:31:46.386994+02:00] [debug] [pid:21382] no match: 67.4s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-18T22:31:47.387270+02:00] [warn] [pid:21382] !!! backend::baseclass::check_asserted_screen: check_asserted_screen took 0.74 seconds for 14 candidate needles - make your needles more specific
[2023-05-18T22:31:47.387869+02:00] [debug] [pid:21382] no match: 66.4s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-18T22:31:55.130059+02:00] [warn] [pid:21382] !!! backend::baseclass::check_asserted_screen: check_asserted_screen took 7.48 seconds for 14 candidate needles - make your needles more specific
[2023-05-18T22:31:55.130529+02:00] [debug] [pid:21382] no match: 65.4s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-18T22:31:55.715449+02:00] [warn] [pid:21382] !!! backend::baseclass::check_asserted_screen: check_asserted_screen took 0.56 seconds for 14 candidate needles - make your needles more specific
[2023-05-18T22:31:55.715931+02:00] [debug] [pid:21382] no match: 57.9s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-18T22:31:56.143795+02:00] [debug] [pid:21382] no change: 56.9s
[2023-05-18T22:31:56.669176+02:00] [warn] [pid:21382] !!! backend::baseclass::check_asserted_screen: check_asserted_screen took 0.52 seconds for 14 candidate needles - make your needles more specific
[2023-05-18T22:31:56.669652+02:00] [debug] [pid:21382] no match: 56.9s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-18T22:32:04.606465+02:00] [debug] [pid:20579] >>> testapi::_handle_found_needle: found bootmenu-jeos-20201204, similarity 1.00 @ 101/208
[2023-05-18T22:32:04.610142+02:00] [debug] [pid:20579] tests/installation/bootloader_uefi.pm:124 called bootloader_setup::uefi_bootmenu_params -> lib/bootloader_setup.pm:383 called testapi::send_key_until_needlematch
[2023-05-18T22:32:04.610684+02:00] [debug] [pid:20579] <<< testapi::check_screen(mustmatch="grub2-enter-edit-mode", timeout=0)
</code></pre>
<p>So, in the first case, the needle takes around 25s to be matched after the VM is started, but in the second case it takes ~45s, which makes the grub to pick the default option and boot the system without a chance to go to the <code>grub2-enter-edit-mode</code> screen.</p>
ALP - coordination #125846 (Resolved): [epic] March Prototype testinghttps://progress.opensuse.org/issues/1258462023-03-13T07:20:30Zjlausuchjalausuch@suse.com
<p>For March prototype we have a 2 main deliverables:</p>
<ul>
<li><a href="https://confluence.suse.com/display/LEONG/01+SUSE+ALP+Bedrock" class="external">ALP Bedrock</a></li>
<li><a href="https://confluence.suse.com/display/LEONG/03+SUSE+ALP+Micro" class="external">ALP Micro</a></li>
</ul>
<p>Bedrock is like generic server-flavor OS and Micro is similar to SLE Micro or MicroOS flavor with limited set of packages.</p>
<p>QE Department has to provide test coverage for all the <a href="https://jira.suse.com/issues/?jql=project%20%3D%20PED%20AND%20fixVersion%20%3D%20%22ALP%20March%20Prototype%22" class="external">features</a> and all the images in the 2 flavors.</p>
<p>This epic aims to collect all the needed tickets related to March prototype testing.</p>
openQA Project - action #113219 (Closed): Create a master label at job group level to label all t...https://progress.opensuse.org/issues/1132192022-07-04T18:53:37Zjlausuchjalausuch@suse.com
<p>As a openQA user, I would like to have a feature to label all the jobs of the same build at the same time with certain <code>bsc</code> or <code>poo</code>, instead of labeling one by one.</p>
<p>A possible use case is in container image tests (BCI) where a new container image build is tested on multiple host versions and architectures to be validated. Sometimes, the image is faulty and we need to label all the jobs (~40) which makes thing very time consuming for the reviewer.<br>
For example, <a href="https://openqa.suse.de/tests/overview?distri=sle&version=15-SP4&build=7.12_pcp-image&groupid=445" class="external">this build</a> or <a href="https://openqa.suse.de/tests/overview?distri=sle&version=15-SP4&build=7.9_pcp-image&groupid=445" class="external">this</a> are mostly failed tests.</p>
<p>It would be nice to use the <code>tag</code> feature + the <code>build</code> number doing something like this:<br>
<img src="https://progress.opensuse.org/attachments/download/13487/Screenshot%202022-07-04%20at%2020.52.21.png" alt="" loading="lazy" /><br>
and automatically label all the jobs.</p>
ALP - action #112895 (Closed): [stale] SelfInstall ALP Images fail in image_checkshttps://progress.opensuse.org/issues/1128952022-06-22T18:07:31Zjlausuchjalausuch@suse.com
<p><a href="https://openqa.opensuse.org/tests/2430307#step/image_checks/8" class="external">https://openqa.opensuse.org/tests/2430307#step/image_checks/8</a><br>
<a href="https://openqa.opensuse.org/tests/2430327#step/image_checks/9" class="external">https://openqa.opensuse.org/tests/2430327#step/image_checks/9</a></p>
<p>It fails in the step: <br>
<code>validate_script_output("sfdisk --list-free /dev/$disk", qr/Unpartitioned space .* 0 sectors/);</code><br>
<a href="https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/tests/microos/image_checks.pm#L26" class="external">https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/tests/microos/image_checks.pm#L26</a></p>
<pre><code>Unpartitioned space /dev/vda: 27.85 GiB, 29906419200 bytes, 58410975 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
Start End Sectors Size
</code></pre>
<p>Expected output:<br>
<a href="https://openqa.opensuse.org/tests/2426647#step/image_checks/6" class="external">https://openqa.opensuse.org/tests/2426647#step/image_checks/6</a></p>
<pre><code>Unpartitioned space /dev/vda: 0 B, 0 bytes, 0 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
</code></pre> ALP - action #112829 (Closed): [stale] ALP uefi jobs fail in disk_boot https://progress.opensuse.org/issues/1128292022-06-21T21:32:51Zjlausuchjalausuch@suse.com
<p>UEFI jobs are not booting properly:<br>
<a href="https://openqa.opensuse.org/tests/2426185" class="external">https://openqa.opensuse.org/tests/2426185</a></p>
<p>Expected result (Leap Micro):<br>
<a href="https://openqa.opensuse.org/tests/2426642#step/disk_boot/1" class="external">https://openqa.opensuse.org/tests/2426642#step/disk_boot/1</a></p>
ALP - coordination #112409 (Resolved): [epic] PoC for openSUSE ALP testing in openQA https://progress.opensuse.org/issues/1124092022-06-14T15:51:37Zjlausuchjalausuch@suse.com
<p>This epic is a coordination effort for a first Proof of Concept enabling ALP testing in openqa.opensuse.org.</p>
<p>The goal is to re-use the existing tests we are running for MicroOS plus some adaptations and execute them against images.</p>
<p>The images that this PoC shall cover are:</p>
<ul>
<li>kvm-and-xen</li>
<li>kvm-and-xen_NonTransactional</li>
<li>SelfInstall</li>
<li>SelfInstall_NonTransactional</li>
</ul>
<p>Current available architecture is x86_64, but aarch64 might be available for the PoC as well.</p>
qe-yam - action #107959 (Closed): 12-SP5-Server-DVD-Updates-s390x installation job fails too oftenhttps://progress.opensuse.org/issues/1079592022-03-08T08:16:49Zjlausuchjalausuch@suse.com
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>openQA test in scenario sle-12-SP5-Server-DVD-Updates-s390x-mru-install-minimal-with-addons@s390x-kvm-sle12 fails in<br>
<a href="https://openqa.suse.de/tests/8286121/modules/await_install/steps/2" class="external">await_install</a></p>
<p>This job is a parent job which many child jobs depend on and are blocked if this fails,<br>
which delays the maintenance review some hours.</p>
<p>It fails 40-50% of the times with <strong>timeout_exceeded</strong> or <strong>incomplete</strong>.</p>
<p>Examples:<br>
<a href="https://openqa.suse.de/tests/8280085" class="external">https://openqa.suse.de/tests/8280085</a><br>
<a href="https://openqa.suse.de/tests/8277975" class="external">https://openqa.suse.de/tests/8277975</a><br>
<a href="https://openqa.suse.de/tests/8277320" class="external">https://openqa.suse.de/tests/8277320</a><br>
<a href="https://openqa.suse.de/tests/8261669" class="external">https://openqa.suse.de/tests/8261669</a><br>
<a href="https://openqa.suse.de/tests/8247241" class="external">https://openqa.suse.de/tests/8247241</a><br>
<a href="https://openqa.suse.de/tests/8239743" class="external">https://openqa.suse.de/tests/8239743</a><br>
<a href="https://openqa.suse.de/tests/8227540" class="external">https://openqa.suse.de/tests/8227540</a><br>
<a href="https://openqa.suse.de/tests/8226549" class="external">https://openqa.suse.de/tests/8226549</a><br>
<a href="https://openqa.suse.de/tests/8194333" class="external">https://openqa.suse.de/tests/8194333</a></p>
<a name="Test-suite-description"></a>
<h2 >Test suite description<a href="#Test-suite-description" class="wiki-anchor">¶</a></h2>
<p>Testsuite maintained at <a href="https://gitlab.suse.de/qa-maintenance/qam-openqa-yml" class="external">https://gitlab.suse.de/qa-maintenance/qam-openqa-yml</a>.</p>
<a name="Reproducible"></a>
<h2 >Reproducible<a href="#Reproducible" class="wiki-anchor">¶</a></h2>
<p>Fails since (at least) Build <a href="https://openqa.suse.de/tests/8286121" class="external">20220308-1</a> (current job)</p>
<a name="Expected-result"></a>
<h2 >Expected result<a href="#Expected-result" class="wiki-anchor">¶</a></h2>
<p>Last good: <a href="https://openqa.suse.de/tests/8281074" class="external">20220307-1</a> (or more recent)</p>
<a name="Further-details"></a>
<h2 >Further details<a href="#Further-details" class="wiki-anchor">¶</a></h2>
<p>Always latest result in this scenario: <a href="https://openqa.suse.de/tests/latest?arch=s390x&distri=sle&flavor=Server-DVD-Updates&machine=s390x-kvm-sle12&test=mru-install-minimal-with-addons&version=12-SP5" class="external">latest</a></p>
openQA Tests - action #107062 (Feedback): Multiple failures due to network issueshttps://progress.opensuse.org/issues/1070622022-02-18T08:27:41Zjlausuchjalausuch@suse.com
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>I will use this ticket to collect the different errors I observe in our tests (at least for QE-C squad) that fail due to network issues.<br>
Normally a restart helps to get the job green again (in case we need it green), but this is not the ideal solution.</p>
<p>The idea of this ticket is to collect more potential issues caught by reviewers and propose solutions for some of them, in the code (retry same command several times might help) or in the infra side. </p>
<p>There is an example for each error I found, but from my experience reviewing jobs every day, these failures happen multiple times a day and randomly (difficult to predict).</p>
<p>1) <strong>SUSEConnect timeouts</strong> -> <a href="https://openqa.suse.de/tests/8189768#step/docker/34">https://openqa.suse.de/tests/8189768#step/docker/34</a><br>
<code>Test died: command 'SUSEConnect -p sle-module-containers/${VERSION_ID}/${CPU} ' timed out at /usr/lib/os-autoinst/testapi.pm line 1039.</code></p>
<p>Or <a href="https://openqa.suse.de/tests/8193554#step/suseconnect_scc/8">https://openqa.suse.de/tests/8193554#step/suseconnect_scc/8</a><br>
<code>Test died: command 'SUSEConnect -r $regcode' timed out at /usr/lib/os-autoinst/testapi.pm line 950.</code></p>
<p>2) <strong><a href="updates.suse.com" class="external">updates.suse.com</a> not reachable</strong> -> <a href="https://openqa.suse.de/tests/8189697#step/image_docker/1110">https://openqa.suse.de/tests/8189697#step/image_docker/1110</a></p>
<pre><code>Retrieving: kmod-25-6.10.1.aarch64.rpm [.........error]
Abort, retry, ignore? [a/r/i/...? shows all options] (a): a
Download (curl) error for 'https://updates.suse.com/SUSE/Updates/SLE-Module-Basesystem/15-SP2/aarch64/update/aarch64/kmod-25-6.10.1.aarch64.rpm?nE0jiYdfiOdLYjH0o-llNN2xIDXncon0vYw8z1aBPGx00H9S1eN413vUsfSJnzFrVz-CoZoGtSdsPKIDRAOQy3Xw2Tac3Yx5_1i8TPomSNiqhDJ0Ayxro23n46NHHB-XHq669RlHs17wiUFSJiSMCSh-YzdGdFw':
Error code: Connection failed
Error message: Could not resolve host: updates.suse.com
Problem occurred during or after installation or removal of packages:
Installation has been aborted as directed.
Please see the above error message for a hint.
</code></pre>
<p>3) <strong>SCC timeouts</strong> -> <a href="https://openqa.suse.de/tests/8189613#step/image_docker/316">https://openqa.suse.de/tests/8189613#step/image_docker/316</a></p>
<pre><code>docker run --entrypoint /usr/lib/zypp/plugins/services/container-suseconnect-zypp -i zypper_docker_derived lp
...
2022/02/18 07:16:19 Installed product: SLES-12.3-x86_64
2022/02/18 07:16:19 Registration server set to https://scc.suse.com
2022/02/18 07:16:30 Get https://scc.suse.com/connect/subscriptions/products?arch=x86_64&identifier=SLES&version=12.3: dial tcp: lookup scc.suse.com on 10.0.2.3:53: read udp 172.17.0.2:37151->10.0.2.3:53: i/o timeout
</code></pre>
<p>4) <strong>zypper ref timeout or error</strong> -> <a href="https://openqa.opensuse.org/tests/2193730#step/image_podman/124">https://openqa.opensuse.org/tests/2193730#step/image_podman/124</a></p>
<pre><code>podman run -i --name 'refreshed' --entrypoint '' registry.opensuse.org/opensuse/leap/15.3/images/totest/containers/opensuse/leap:15.3 zypper -nv ref
...
Retrieving: cb71cb070e8aac79327e6f1b6edc5317122ca1f72970299c3cb2cf505e18b27f-deltainfo.xml.gz [........................done (82.3 KiB/s)]
Retrieving: 832729371fe20bc1a4d27e59d76c10ffe2c0b5a1ff71c4e934e7a11baa24a74b-primary.xml.gz [............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................error (87.0 KiB/s)]
WJdDM-124-
</code></pre>
<a name="Acceptance-criteria"></a>
<h2 >Acceptance criteria<a href="#Acceptance-criteria" class="wiki-anchor">¶</a></h2>
<ul>
<li><strong>AC1:</strong> All existing subtasks are resolved, no additional work needed on top</li>
</ul>
openQA Tests - action #62408 (Closed): [kernel][public cloud] Timeout when creating Azure-Standar...https://progress.opensuse.org/issues/624082020-01-21T07:59:24Zjlausuchjalausuch@suse.com
<p>All the Azure-Standard tests are failing with timeout in terraform apply. <br>
<a href="https://openqa.suse.de/tests/overview?distri=sle&version=15-SP2&build=0.9.3-1.28&groupid=219&flavor=Azure-Standard" class="external">https://openqa.suse.de/tests/overview?distri=sle&version=15-SP2&build=0.9.3-1.28&groupid=219&flavor=Azure-Standard</a></p>
<p>I did some tests increasing that timeout to 1 hour and I get this message:</p>
<pre><code>azurerm_virtual_machine.openqa-vm[0]: Still creating... [40m10s elapsed]
azurerm_virtual_machine.openqa-vm[0]: Still creating... [40m20s elapsed]
azurerm_virtual_machine.openqa-vm[0]: Still creating... [40m30s elapsed]
azurerm_virtual_machine.openqa-vm[0]: Still creating... [40m40s elapsed]
azurerm_virtual_machine.openqa-vm[0]: Still creating... [40m50s elapsed]
azurerm_virtual_machine.openqa-vm[0]: Still creating... [41m0s elapsed]
Error: Code="OSProvisioningTimedOut" Message="OS Provisioning for VM 'jlausuch-vm-b453e006c72d7fae' did not finish in the allotted time. The VM may still finish provisioning successfully. Please check provisioning state later. Also, make sure the image has been properly prepared (generalized).\r\n * Instructions for Windows: https://azure.microsoft.com/documentation/articles/virtual-machines-windows-upload-image/ \r\n * Instructions for Linux: https://azure.microsoft.com/documentation/articles/virtual-machines-linux-capture-image/ "
on plan.tf line 129, in resource "azurerm_virtual_machine" "openqa-vm":
129: resource "azurerm_virtual_machine" "openqa-vm" {
</code></pre> openQA Tests - action #59339 (Closed): [functional] Support Server based on SLE12-SP3 is brokenhttps://progress.opensuse.org/issues/593392019-11-12T11:05:59Zjlausuchjalausuch@suse.com
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>Support Server fails to install NFS related packages in ARM.<br>
<a href="https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/tests/support_server/setup.pm#L530" class="external">https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/tests/support_server/setup.pm#L530</a></p>
<a name="Test-suite-description"></a>
<h2 >Test suite description<a href="#Test-suite-description" class="wiki-anchor">¶</a></h2>
<p>Maintainer: Pavel Sladek <a href="mailto:psladek@suse.com">psladek@suse.com</a><br>
according to <a href="https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/tests/support_server/setup.pm#L35" class="external">https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/tests/support_server/setup.pm#L35</a></p>
<a name="Reproducible"></a>
<h2 >Reproducible<a href="#Reproducible" class="wiki-anchor">¶</a></h2>
<p>Fails since (at least) Build <a href="https://openqa.suse.de/tests/3583082" class="external">0372</a></p>
<pre><code># zypper -n in rpcbind nfs-kernel-server | cat; ( exit ${PIPESTATUS[0]} ); echo yjD1U-$?-
File '/content' not found on medium 'http://download.suse.de/install/SLP/SLE-12-SP3-Server-GM/x86_64/DVD1/'
</code></pre>
<a name="Expected-result"></a>
<h2 >Expected result<a href="#Expected-result" class="wiki-anchor">¶</a></h2>
<p>Last good: <a href="https://openqa.suse.de/tests/3563133" class="external">0369</a></p>
<pre><code># zypper -n in rpcbind nfs-kernel-server | cat; ( exit ${PIPESTATUS[0]} ); echo yjD1U-$?-
Loading repository data...
Reading installed packages...
'rpcbind' is already installed.
Package 'rpcbind' is not available in your repositories. Cannot reinstall, upgrade, or downgrade.
'nfs-kernel-server' is already installed.
Package 'nfs-kernel-server' is not available in your repositories. Cannot reinstall, upgrade, or downgrade.
Resolving package dependencies...
Nothing to do.
</code></pre>
<a name="Further-details"></a>
<h2 >Further details<a href="#Further-details" class="wiki-anchor">¶</a></h2>
<p>The tests are using the same qcow2 image since a bunch of builds, and the reason why it got broken recently is because<br>
<a href="http://download.suse.de/install/SLP/SLE-12-SP3-Server-GM/x86_64/DVD1" class="external">http://download.suse.de/install/SLP/SLE-12-SP3-Server-GM/x86_64/DVD1</a> does not longer exist.</p>
<p>We should build a new support server image using a newer repo. E.g. SP4:<br>
<a href="http://download.suse.de/install/SLP/SLE-12-SP4-Server-GM/x86_64/DVD1" class="external">http://download.suse.de/install/SLP/SLE-12-SP4-Server-GM/x86_64/DVD1</a></p>
openQA Tests - action #58220 (Resolved): [kernel] fadump LVM test failshttps://progress.opensuse.org/issues/582202019-10-15T20:25:44Zjlausuchjalausuch@suse.com
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>Failure when going into grub screen<br>
<a href="https://openqa.suse.de/tests/3479900#step/kdump_and_crash/64" class="external">grub screen</a></p>
<a name="Test-suite-description"></a>
<h2 >Test suite description<a href="#Test-suite-description" class="wiki-anchor">¶</a></h2>
<p>Maintainer: Petr Cervinka <a href="mailto:pcervinka@suse.com">pcervinka@suse.com</a></p>
<a name="Reproducible"></a>
<h2 >Reproducible<a href="#Reproducible" class="wiki-anchor">¶</a></h2>
<p>Fails since (at least) Build <a href="https://openqa.suse.de/tests/3479900" class="external">0358</a><br>
But it also failed in some <a href="https://openqa.suse.de/tests/3395111" class="external">older run</a></p>
<a name="Expected-result"></a>
<h2 >Expected result<a href="#Expected-result" class="wiki-anchor">¶</a></h2>
<p>This is an example of <a href="https://openqa.suse.de/tests/3470330#step/kdump_and_crash/64" class="external">successful run</a> in the previous build.</p>
<a name="Further-details"></a>
<h2 >Further details<a href="#Further-details" class="wiki-anchor">¶</a></h2>
openQA Tests - action #58148 (Rejected): [kernel][multipath] Timeout in qa_kernel_multipathhttps://progress.opensuse.org/issues/581482019-10-14T15:19:53Zjlausuchjalausuch@suse.com
<p>The test times out after 2 hours. Need to investigate why.</p>
<p><a href="https://openqa.suse.de/tests/3470142" class="external">https://openqa.suse.de/tests/3470142</a><br>
<a href="https://openqa.suse.de/tests/3470021" class="external">https://openqa.suse.de/tests/3470021</a></p>
openQA Project - action #31417 (Rejected): Add support for SSH from Host to VMhttps://progress.opensuse.org/issues/314172018-02-06T14:21:54Zjlausuchjalausuch@suse.com
<p>The VMs that OpenQA launches have an internal IP that is not reachable from the Host. Therefore, there is no way to SSH into them, only VNC is available. </p>
<p>The problem of VNC is that it limits the user comfort when debugging. SSH allows much better experience as you can use your own console and use SCP, bidireccional copy/paste, mouse scrolling, etc. </p>
<p>However, it is possible to SSH from the VM to the Host, therefore Reverse SSH can be used, but it is just a workaround. Having ssh supported directly from the Host would be more convenient.</p>
<p>I think this could be achieved by changing the qemu command line to launch the VM adding a parameter <em>-net user,hostfwd=tcp::7777-:8001</em><br>
Not sure if this is the right place, but it could help to look at this line: <a href="https://github.com/os-autoinst/os-autoinst/blob/master/backend/qemu.pm#L581" class="external">https://github.com/os-autoinst/os-autoinst/blob/master/backend/qemu.pm#L581</a></p>