openSUSE Project Management Tool: Issueshttps://progress.opensuse.org/https://progress.opensuse.org/themes/openSUSE/favicon/favicon.ico?15829177842024-03-21T09:51:42ZopenSUSE Project Management Tool
Redmine openQA Tests - action #157657 (Feedback): PackageHub module activation while using SCC-proxyhttps://progress.opensuse.org/issues/1576572024-03-21T09:51:42Zjlausuchjalausuch@suse.com
<a name="Problem-statement"></a>
<h2 >Problem statement:<a href="#Problem-statement" class="wiki-anchor">¶</a></h2>
<p>When registering the system using ProxySCC, activating the PackageHub module returns repos in OSD that don't exist.<br>
This is expected, because the proxySCC replaces the repo URLs given by the real SCC by local openQA asset URL.</p>
<p>Normally, we use the proxySCC in the openQA jobs with <code>SCC_URL</code> variable, this will be used by the code to make the <code>SUSEConnect</code> call like this:</p>
<p><code>SUSEConnect -r $REGCODE --url http://all-67.1.proxy.scc.suse.de</code></p>
<p>This will make that all the repos returned look like this:<br>
<code>http://openqa.suse.de/assets/repo/...</code></p>
<p>And enabling any module, will have the same effect, for example PackageHub:</p>
<pre><code>
susetest:~ # suseconnect -p PackageHub/15.6/x86_64
Registering system to registration proxy http://all-67.1.proxy.scc.suse.de
Updating system details on http://all-67.1.proxy.scc.suse.de ...
Activating PackageHub 15.6 x86_64 ...
-> Adding service to system ...
-> Installing release package ...
Successfully registered system
</code></pre>
<p>And the repo will point to the wrong place:</p>
<pre><code>18 | SUSE_Package_Hub_15_SP6_x86_64:SLE-Module-Packagehub-Subpackages15-SP6-Pool | SLE-Module-Packagehub-Subpackages15-SP6-Pool | Yes | (r ) Yes | No | http://openqa.suse.de/assets/repo/SLE-15-SP6-Module-Packagehub-Subpackages-POOL-x86_64-Build67.1-Media1/
</code></pre>
<a name="Solution"></a>
<h2 >Solution<a href="#Solution" class="wiki-anchor">¶</a></h2>
<p>To cope with this, we would need to sync the PackageHub module for EVERY build we have for 15-SP6. Another solution would be to tweak ProxySCC code to return the real URL instead of local assets.</p>
<p>BUT there is an easier way:<br>
1) Register the system using the usual proxySCC</p>
<pre><code>susetest:~ # SUSEConnect -r $REGCODE --url http://all-67.1.proxy.scc.suse.de
Registering system to registration proxy http://all-67.1.proxy.scc.suse.de
Announcing system to http://all-67.1.proxy.scc.suse.de ...
Activating SLES 15.6 x86_64 ...
-> Adding service to system ...
Activating sle-module-basesystem 15.6 x86_64 ...
-> Adding service to system ...
-> Installing release package ...
Activating sle-module-server-applications 15.6 x86_64 ...
-> Adding service to system ...
-> Installing release package ...
Activating sle-module-python3 15.6 x86_64 ...
-> Adding service to system ...
-> Installing release package ...
Successfully registered system
</code></pre>
<p>2) Activate the desired module using <code>--url</code> pointing to real SCC:</p>
<pre><code>susetest:~ # SUSEConnect -p PackageHub/15.6/x86_64 --url https://scc.suse.com
Registering system to SUSE Customer Center
Updating system details on https://scc.suse.com ...
Activating PackageHub 15.6 x86_64 ...
-> Adding service to system ...
-> Installing release package ...
Successfully registered system
</code></pre>
<p>3) And voilá, you will get the repos from REAL SCC:</p>
<pre><code>susetest:~ # zypper lr -u |grep -i PackageHub-15-SP6-Pool
22 | SUSE_Package_Hub_15_SP6_x86_64:SUSE-PackageHub-15-SP6-Pool | SUSE-PackageHub-15-SP6-Pool | Yes | (r ) Yes | No | https://updates.suse.com/SUSE/Backports/SLE-15-SP6_x86_64/product?RMMzfgZAuv8Hy903n30FsW6hbWs5KExVGGIwF1w5nRMdkliFvRBscHmgomWpHwFbG8l2C7cmrcZead-ejTo_f0NKbts7ZNwGBw0WtMwuWeKXxK2mgknqmbcYZ22y3RB3rThU1AMZTw
</code></pre> openQA Tests - action #136298 (Resolved): Leap Micro test fails in transactional_updatehttps://progress.opensuse.org/issues/1362982023-09-22T06:11:38Zjlausuchjalausuch@suse.com
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>openQA test in scenario leap-micro-5.5-MicroOS-Image-x86_64-microos_image_default@64bit fails in<br>
<a href="https://openqa.opensuse.org/tests/3591732/modules/transactional_update/steps/96" class="external">transactional_update</a></p>
<p>As the job says, there should be an expected change when <a href="https://openqa.opensuse.org/tests/3591732#step/transactional_update/23" class="external">installing the local rpm</a><br>
<code>transactional-update -n ptf install update-test-trivial/update-test-security-5.1-1.20.x86_64.rpm</code><br>
and <a href="https://openqa.opensuse.org/tests/3591732#step/transactional_update/73" class="external">adding the utt repo</a> along with <code>transactional-update -n cleanup up</code>. </p>
<p>The local package is <code>/data/microos/utt-opensuse-x86_64.tgz</code> and the repo is <code>/data/microos/utt-leap.repo</code>, both in the data directory:<br>
<a href="https://github.com/os-autoinst/os-autoinst-distri-opensuse/tree/master/data/microos" class="external">https://github.com/os-autoinst/os-autoinst-distri-opensuse/tree/master/data/microos</a></p>
<p>This was working on Leap Micro 5.4 (no existing jobs), so for some reason now the packages seem to be the same or the local rpm is already a higher version than the one in the repo, and zypper says <code>zypper: nothing to update</code>.</p>
<a name="Test-suite-description"></a>
<h2 >Test suite description<a href="#Test-suite-description" class="wiki-anchor">¶</a></h2>
<p>MicroOS image boot with ignition disk and default tests. Default tests are transactional-update, rebootmgr, health_check, cockpit service and some other checks specific to MicroOS.</p>
<a name="Reproducible"></a>
<h2 >Reproducible<a href="#Reproducible" class="wiki-anchor">¶</a></h2>
<p>Fails since the beginning.</p>
openQA Tests - action #129601 (Feedback): bootloader_uefi fails too often in aarch64https://progress.opensuse.org/issues/1296012023-05-19T05:49:51Zjlausuchjalausuch@suse.com
<p><a href="https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/tests/installation/bootloader_uefi.pm" class="external">installation/bootloader_uefi.pm</a> times out quite often because openQA does not have time to enter uefi grub2 configuration screen before the 10s grub timeout.</p>
<p>e.g. <br>
<a href="https://openqa.suse.de/tests/11157624#step/bootloader_uefi/11">https://openqa.suse.de/tests/11157624#step/bootloader_uefi/11</a><br>
<a href="https://openqa.suse.de/tests/11157620#step/bootloader_uefi/11">https://openqa.suse.de/tests/11157620#step/bootloader_uefi/11</a><br>
<a href="https://openqa.suse.de/tests/11157622#step/bootloader_uefi/11">https://openqa.suse.de/tests/11157622#step/bootloader_uefi/11</a></p>
<p>The culprit is that the first <code>bootloader-grub2</code> takes too long to match.</p>
<p>This is a successful run:</p>
<pre><code>[2023-05-17T22:29:48.135632+02:00] [debug] [pid:78975] <<< testapi::assert_screen(mustmatch=[
"bootloader-shim-import-prompt",
"bootloader-grub2"
], timeout=90)
[2023-05-17T22:29:49.928958+02:00] [warn] [pid:79369] !!! backend::baseclass::check_asserted_screen: check_asserted_screen took 0.82 seconds for 14 candidate needles - make your needles more specific
[2023-05-17T22:29:49.929428+02:00] [debug] [pid:79369] no match: 89.0s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-17T22:29:50.104691+02:00] [debug] [pid:79369] no change: 88.0s
[2023-05-17T22:29:50.588948+02:00] [debug] [pid:79369] no match: 88.0s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-17T22:29:51.105278+02:00] [debug] [pid:79369] no change: 87.0s
[2023-05-17T22:29:51.589146+02:00] [debug] [pid:79369] no match: 87.0s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-17T22:29:52.105972+02:00] [debug] [pid:79369] no change: 86.0s
[2023-05-17T22:29:52.589983+02:00] [debug] [pid:79369] no match: 86.0s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-17T22:30:00.232487+02:00] [warn] [pid:79369] !!! backend::baseclass::check_asserted_screen: check_asserted_screen took 7.13 seconds for 14 candidate needles - make your needles more specific
[2023-05-17T22:30:00.232953+02:00] [debug] [pid:79369] no match: 85.0s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-17T22:30:00.248237+02:00] [debug] [pid:79369] no change: 77.8s
[2023-05-17T22:30:00.733227+02:00] [debug] [pid:79369] no match: 77.8s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-17T22:30:01.236512+02:00] [debug] [pid:79369] no change: 76.9s
[2023-05-17T22:30:01.743323+02:00] [warn] [pid:79369] !!! backend::baseclass::check_asserted_screen: check_asserted_screen took 0.51 seconds for 14 candidate needles - make your needles more specific
[2023-05-17T22:30:01.743772+02:00] [debug] [pid:79369] no match: 76.9s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-17T22:30:09.594737+02:00] [warn] [pid:79369] !!! backend::baseclass::check_asserted_screen: check_asserted_screen took 7.36 seconds for 14 candidate needles - make your needles more specific
[2023-05-17T22:30:09.595227+02:00] [debug] [pid:79369] no match: 75.9s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-17T22:30:10.332028+02:00] [warn] [pid:79369] !!! backend::baseclass::check_asserted_screen: check_asserted_screen took 0.71 seconds for 14 candidate needles - make your needles more specific
[2023-05-17T22:30:10.332520+02:00] [debug] [pid:79369] no match: 68.5s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-17T22:30:11.328004+02:00] [warn] [pid:79369] !!! backend::baseclass::check_asserted_screen: check_asserted_screen took 0.72 seconds for 14 candidate needles - make your needles more specific
[2023-05-17T22:30:11.328472+02:00] [debug] [pid:79369] no match: 67.5s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-17T22:30:12.205373+02:00] [warn] [pid:79369] !!! backend::baseclass::check_asserted_screen: check_asserted_screen took 0.59 seconds for 14 candidate needles - make your needles more specific
[2023-05-17T22:30:12.205867+02:00] [debug] [pid:79369] no match: 66.5s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-17T22:30:19.704649+02:00] [warn] [pid:79369] !!! backend::baseclass::check_asserted_screen: check_asserted_screen took 7.09 seconds for 14 candidate needles - make your needles more specific
[2023-05-17T22:30:19.705116+02:00] [debug] [pid:79369] no match: 65.5s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-17T22:30:20.417002+02:00] [debug] [pid:78975] >>> testapi::_handle_found_needle: found bootmenu-jeos-20201204, similarity 1.00 @ 101/208
[2023-05-17T22:30:20.419694+02:00] [debug] [pid:78975] tests/installation/bootloader_uefi.pm:124 called bootloader_setup::uefi_bootmenu_params -> lib/bootloader_setup.pm:383 called testapi::send_key_until_needlematch
[2023-05-17T22:30:20.420160+02:00] [debug] [pid:78975] <<< testapi::check_screen(mustmatch="grub2-enter-edit-mode", timeout=0)
</code></pre>
<p>And this is a failed run:</p>
<pre><code>[2023-05-18T22:31:23.112731+02:00] [debug] [pid:20579] <<< testapi::assert_screen(mustmatch=[
"bootloader-shim-import-prompt",
"bootloader-grub2"
], timeout=90)
[2023-05-18T22:31:24.904170+02:00] [warn] [pid:21382] !!! backend::baseclass::check_asserted_screen: check_asserted_screen took 0.82 seconds for 14 candidate needles - make your needles more specific
[2023-05-18T22:31:24.904637+02:00] [debug] [pid:21382] no match: 89.0s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-18T22:31:25.081871+02:00] [debug] [pid:21382] no change: 88.0s
[2023-05-18T22:31:25.580143+02:00] [debug] [pid:21382] no match: 88.0s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-18T22:31:26.094924+02:00] [debug] [pid:21382] no change: 87.0s
[2023-05-18T22:31:26.569793+02:00] [debug] [pid:21382] no match: 87.0s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-18T22:31:27.083393+02:00] [debug] [pid:21382] no change: 86.0s
[2023-05-18T22:31:27.553981+02:00] [debug] [pid:21382] no match: 86.0s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-18T22:31:35.276260+02:00] [warn] [pid:21382] !!! backend::baseclass::check_asserted_screen: check_asserted_screen took 7.19 seconds for 14 candidate needles - make your needles more specific
[2023-05-18T22:31:35.276731+02:00] [debug] [pid:21382] no match: 85.0s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-18T22:31:36.083159+02:00] [warn] [pid:21382] !!! backend::baseclass::check_asserted_screen: check_asserted_screen took 0.78 seconds for 14 candidate needles - make your needles more specific
[2023-05-18T22:31:36.083646+02:00] [debug] [pid:21382] no match: 77.8s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-18T22:31:36.279755+02:00] [debug] [pid:21382] no change: 76.8s
[2023-05-18T22:31:37.020384+02:00] [warn] [pid:21382] !!! backend::baseclass::check_asserted_screen: check_asserted_screen took 0.74 seconds for 14 candidate needles - make your needles more specific
[2023-05-18T22:31:37.020854+02:00] [debug] [pid:21382] no match: 76.8s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-18T22:31:44.628366+02:00] [warn] [pid:21382] !!! backend::baseclass::check_asserted_screen: check_asserted_screen took 7.35 seconds for 14 candidate needles - make your needles more specific
[2023-05-18T22:31:44.628834+02:00] [debug] [pid:21382] no match: 75.8s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-18T22:31:45.401250+02:00] [warn] [pid:21382] !!! backend::baseclass::check_asserted_screen: check_asserted_screen took 0.74 seconds for 14 candidate needles - make your needles more specific
[2023-05-18T22:31:45.401731+02:00] [debug] [pid:21382] no match: 68.4s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-18T22:31:46.386500+02:00] [warn] [pid:21382] !!! backend::baseclass::check_asserted_screen: check_asserted_screen took 0.74 seconds for 14 candidate needles - make your needles more specific
[2023-05-18T22:31:46.386994+02:00] [debug] [pid:21382] no match: 67.4s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-18T22:31:47.387270+02:00] [warn] [pid:21382] !!! backend::baseclass::check_asserted_screen: check_asserted_screen took 0.74 seconds for 14 candidate needles - make your needles more specific
[2023-05-18T22:31:47.387869+02:00] [debug] [pid:21382] no match: 66.4s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-18T22:31:55.130059+02:00] [warn] [pid:21382] !!! backend::baseclass::check_asserted_screen: check_asserted_screen took 7.48 seconds for 14 candidate needles - make your needles more specific
[2023-05-18T22:31:55.130529+02:00] [debug] [pid:21382] no match: 65.4s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-18T22:31:55.715449+02:00] [warn] [pid:21382] !!! backend::baseclass::check_asserted_screen: check_asserted_screen took 0.56 seconds for 14 candidate needles - make your needles more specific
[2023-05-18T22:31:55.715931+02:00] [debug] [pid:21382] no match: 57.9s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-18T22:31:56.143795+02:00] [debug] [pid:21382] no change: 56.9s
[2023-05-18T22:31:56.669176+02:00] [warn] [pid:21382] !!! backend::baseclass::check_asserted_screen: check_asserted_screen took 0.52 seconds for 14 candidate needles - make your needles more specific
[2023-05-18T22:31:56.669652+02:00] [debug] [pid:21382] no match: 56.9s, best candidate: bootloader_uefi-20210917 (0.00)
[2023-05-18T22:32:04.606465+02:00] [debug] [pid:20579] >>> testapi::_handle_found_needle: found bootmenu-jeos-20201204, similarity 1.00 @ 101/208
[2023-05-18T22:32:04.610142+02:00] [debug] [pid:20579] tests/installation/bootloader_uefi.pm:124 called bootloader_setup::uefi_bootmenu_params -> lib/bootloader_setup.pm:383 called testapi::send_key_until_needlematch
[2023-05-18T22:32:04.610684+02:00] [debug] [pid:20579] <<< testapi::check_screen(mustmatch="grub2-enter-edit-mode", timeout=0)
</code></pre>
<p>So, in the first case, the needle takes around 25s to be matched after the VM is started, but in the second case it takes ~45s, which makes the grub to pick the default option and boot the system without a chance to go to the <code>grub2-enter-edit-mode</code> screen.</p>
openQA Tests - action #116257 (New): [virtualization][svirt] Some workers in openqaworker2 time o...https://progress.opensuse.org/issues/1162572022-09-06T06:57:39Zjlausuchjalausuch@suse.com
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>openQA test in scenario sle-12-SP5-JeOS-for-kvm-and-xen-Updates-x86_64-jeos-extratest@svirt-xen-hvm fails in<br>
<a href="https://openqa.suse.de/tests/9459035/modules/bootloader_svirt/steps/25" class="external">bootloader_svirt</a></p>
<p>It hits the <code>MAX_JOB_TIMEOUT</code> while trying to copy the image. </p>
<p>The affected workers are:<br>
<a href="https://openqa.suse.de/admin/workers/366" class="external">openqaworker2:9</a><br>
<a href="https://openqa.suse.de/admin/workers/980" class="external">openqaworker2:10</a><br>
<a href="https://openqa.suse.de/admin/workers/1252" class="external">openqaworker2:16</a></p>
<p>Most jobs using these workers time out during this step. Other examples:<br>
<a href="https://openqa.suse.de/tests/9459036" class="external">https://openqa.suse.de/tests/9459036</a><br>
<a href="https://openqa.suse.de/tests/9459031" class="external">https://openqa.suse.de/tests/9459031</a><br>
<a href="https://openqa.suse.de/tests/9459037" class="external">https://openqa.suse.de/tests/9459037</a><br>
<a href="https://openqa.suse.de/tests/9459064" class="external">https://openqa.suse.de/tests/9459064</a><br>
<a href="https://openqa.suse.de/tests/9459069" class="external">https://openqa.suse.de/tests/9459069</a></p>
<a name="Reproducible"></a>
<h2 >Reproducible<a href="#Reproducible" class="wiki-anchor">¶</a></h2>
<p>Fails since (at least) Build <a href="https://openqa.suse.de/tests/9459035" class="external">20220905-1</a> (current job)</p>
<a name="Expected-result"></a>
<h2 >Expected result<a href="#Expected-result" class="wiki-anchor">¶</a></h2>
<p>Last good: <a href="https://openqa.suse.de/tests/9450196" class="external">20220903-1</a> (or more recent)</p>
<a name="Further-details"></a>
<h2 >Further details<a href="#Further-details" class="wiki-anchor">¶</a></h2>
<p>Always latest result in this scenario: <a href="https://openqa.suse.de/tests/latest?arch=x86_64&distri=sle&flavor=JeOS-for-kvm-and-xen-Updates&machine=svirt-xen-hvm&test=jeos-extratest&version=12-SP5" class="external">latest</a></p>
openQA Tests - action #111093 (New): [containers][sporadic][s389x] test fails in boot_to_desktop ...https://progress.opensuse.org/issues/1110932022-05-13T13:22:29Zjlausuchjalausuch@suse.com
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>openQA test in scenario sle-15-SP4-BCI-Updates-s390x-bci_on_SLES_15-SP2_host_docker@s390x-kvm-sle12 fails in<br>
<a href="https://openqa.suse.de/tests/8753928/modules/boot_to_desktop/steps/28" class="external">boot_to_desktop</a></p>
<a name="Test-suite-description"></a>
<h2 >Test suite description<a href="#Test-suite-description" class="wiki-anchor">¶</a></h2>
<p>The base test suite is used for job templates defined in YAML documents. It has no settings of its own.</p>
<a name="Reproducible"></a>
<h2 >Reproducible<a href="#Reproducible" class="wiki-anchor">¶</a></h2>
<p>Fails since (at least) Build <a href="https://openqa.suse.de/tests/8753928" class="external">_15-SP4_10.47_minimal-image</a> (current job)</p>
<a name="Expected-result"></a>
<h2 >Expected result<a href="#Expected-result" class="wiki-anchor">¶</a></h2>
<p>Last good: <a href="https://openqa.suse.de/tests/8744352" class="external">_15-SP4_3.9_python-3.10-image</a> (or more recent)</p>
<a name="Further-details"></a>
<h2 >Further details<a href="#Further-details" class="wiki-anchor">¶</a></h2>
<p>Always latest result in this scenario: <a href="https://openqa.suse.de/tests/latest?arch=s390x&distri=sle&flavor=BCI-Updates&machine=s390x-kvm-sle12&test=bci_on_SLES_15-SP2_host_docker&version=15-SP4" class="external">latest</a></p>
openQA Tests - action #111010 (Resolved): type_password does not hide the string in autoinst-log.txthttps://progress.opensuse.org/issues/1110102022-05-12T11:30:11Zjlausuchjalausuch@suse.com
<p>I am doing some tests using <code>type_password</code> command to put some sensitive information into a file in a running system.<br>
According to the documentation [1]: <code>A convenience wrapper around type_string, which doesn’t log the string.</code></p>
<p>However, if I do:<br>
<code>type_password("echo 'yada yada' > /root/influxdb_conf\n");</code></p>
<p>This info is shown in autoinst-log.txt:</p>
<pre><code>[2022-05-12T13:05:09.897059+02:00] [debug] tests/jeos/image_info.pm:130 called testapi::type_password
[2022-05-12T13:05:09.897329+02:00] [debug] <<< testapi::type_string(string="echo 'yada yada' > /root/influxdb_conf\n", max_interval=100, wait_screen_changes=0, wait_still_screen=0, timeout=30, similarity_level=47)
</code></pre>
<p>This is using root-console. If I use serial_terminal, I get it in the <code>serial_terminal.txt</code> log:</p>
<pre><code>...
SCRIPT_FINISHEDGItcm-0-
# echo 'yada yada' > /root/influxdb_conf
</code></pre>
<p>Also, openQA tests are showing this </p>
<p>2 months ago (before <a href="https://github.com/os-autoinst/os-autoinst/pull/2002" class="external">https://github.com/os-autoinst/os-autoinst/pull/2002</a>)<br>
<a href="https://openqa.suse.de/tests/8405529/logfile?filename=autoinst-log.txt" class="external">https://openqa.suse.de/tests/8405529/logfile?filename=autoinst-log.txt</a></p>
<pre><code>[2022-03-26T12:33:40.095557+01:00] [debug] tests/jeos/firstrun.pm:116 called testapi::type_password
[2022-03-26T12:33:40.095850+01:00] [debug] <<< testapi::type_string(string="SECRET STRING", max_interval=100, wait_screen_changes=0, wait_still_screen=0, timeout=30, similarity_level=47)
</code></pre>
<p>Now:<br>
<a href="https://openqa.suse.de/tests/8738443/logfile?filename=autoinst-log.txt" class="external">https://openqa.suse.de/tests/8738443/logfile?filename=autoinst-log.txt</a></p>
<pre><code>[2022-05-12T06:09:58.666172+02:00] [debug] tests/jeos/firstrun.pm:116 called testapi::type_password
[2022-05-12T06:09:58.666561+02:00] [debug] <<< testapi::type_string(string="nots3cr3t", max_interval=100, wait_screen_changes=0, wait_still_screen=0, timeout=30, similarity_level=47)
</code></pre>
<p>[1] <a href="http://open.qa/api/testapi/#_type_password" class="external">http://open.qa/api/testapi/#_type_password</a></p>
openQA Tests - action #109882 (Resolved): Leap Micro 5.2 autoyast installation misses the repo-ma...https://progress.opensuse.org/issues/1098822022-04-12T22:14:27Zjlausuchjalausuch@suse.com
<p>After doing an <a href="https://openqa.opensuse.org/tests/2293631" class="external">autoyast installation</a>, the system doesn't have the repository </p>
<pre><code>install:~ # zypper lr -u
Repository priorities are without effect. All enabled repositories share the same priority.
# | Alias | Name | Enabled | GPG Check | Refresh | URI
--+---------------------------+---------------------------+---------+-----------+---------+--------------------------------------------------------
1 | openSUSE-Leap-Micro-5.2-1 | openSUSE-Leap-Micro-5.2-1 | No | ---- | ---- | cd:/?devices=/dev/disk/by-id/scsi-0QEMU_QEMU_CD-ROM_cd0
</code></pre>
<p>And because of this, the installation of some packages fail (e.g. <a href="https://openqa.opensuse.org/tests/2293631#step/cockpit_service/8" class="external">cockpit</a>)</p>
<p>This is how it should look like from the regular <a href="https://openqa.opensuse.org/tests/2294237" class="external">DVD installation</a>:</p>
<pre><code>localhost:~ # zypper lr -u
Repository priorities are without effect. All enabled repositories share the same priority.
# | Alias | Name | Enabled | GPG Check | Refresh | URI
--+---------------------------+----------------------------+---------+-----------+---------+----------------------------------------------------------------------------------------------------
1 | openSUSE-Leap-Micro-5.2-1 | openSUSE-Leap-Micro-5.2-1 | No | ---- | ---- | cd:/?devices=/dev/disk/by-id/scsi-0QEMU_QEMU_CD-ROM_cd0
2 | repo-main | Leap Micro Main Repository | Yes | (r ) Yes | Yes | https://download.opensuse.org/distribution/leap-micro/5.2/product/repo/Leap-Micro-5.2-x86_64-Media/
</code></pre>
<p>The used autoyast profile is <a href="https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/data/autoyast_opensuse/autoyast_leap-micro.xml.ep" class="external">https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/data/autoyast_opensuse/autoyast_leap-micro.xml.ep</a></p>
qe-yam - action #107959 (Closed): 12-SP5-Server-DVD-Updates-s390x installation job fails too oftenhttps://progress.opensuse.org/issues/1079592022-03-08T08:16:49Zjlausuchjalausuch@suse.com
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>openQA test in scenario sle-12-SP5-Server-DVD-Updates-s390x-mru-install-minimal-with-addons@s390x-kvm-sle12 fails in<br>
<a href="https://openqa.suse.de/tests/8286121/modules/await_install/steps/2" class="external">await_install</a></p>
<p>This job is a parent job which many child jobs depend on and are blocked if this fails,<br>
which delays the maintenance review some hours.</p>
<p>It fails 40-50% of the times with <strong>timeout_exceeded</strong> or <strong>incomplete</strong>.</p>
<p>Examples:<br>
<a href="https://openqa.suse.de/tests/8280085" class="external">https://openqa.suse.de/tests/8280085</a><br>
<a href="https://openqa.suse.de/tests/8277975" class="external">https://openqa.suse.de/tests/8277975</a><br>
<a href="https://openqa.suse.de/tests/8277320" class="external">https://openqa.suse.de/tests/8277320</a><br>
<a href="https://openqa.suse.de/tests/8261669" class="external">https://openqa.suse.de/tests/8261669</a><br>
<a href="https://openqa.suse.de/tests/8247241" class="external">https://openqa.suse.de/tests/8247241</a><br>
<a href="https://openqa.suse.de/tests/8239743" class="external">https://openqa.suse.de/tests/8239743</a><br>
<a href="https://openqa.suse.de/tests/8227540" class="external">https://openqa.suse.de/tests/8227540</a><br>
<a href="https://openqa.suse.de/tests/8226549" class="external">https://openqa.suse.de/tests/8226549</a><br>
<a href="https://openqa.suse.de/tests/8194333" class="external">https://openqa.suse.de/tests/8194333</a></p>
<a name="Test-suite-description"></a>
<h2 >Test suite description<a href="#Test-suite-description" class="wiki-anchor">¶</a></h2>
<p>Testsuite maintained at <a href="https://gitlab.suse.de/qa-maintenance/qam-openqa-yml" class="external">https://gitlab.suse.de/qa-maintenance/qam-openqa-yml</a>.</p>
<a name="Reproducible"></a>
<h2 >Reproducible<a href="#Reproducible" class="wiki-anchor">¶</a></h2>
<p>Fails since (at least) Build <a href="https://openqa.suse.de/tests/8286121" class="external">20220308-1</a> (current job)</p>
<a name="Expected-result"></a>
<h2 >Expected result<a href="#Expected-result" class="wiki-anchor">¶</a></h2>
<p>Last good: <a href="https://openqa.suse.de/tests/8281074" class="external">20220307-1</a> (or more recent)</p>
<a name="Further-details"></a>
<h2 >Further details<a href="#Further-details" class="wiki-anchor">¶</a></h2>
<p>Always latest result in this scenario: <a href="https://openqa.suse.de/tests/latest?arch=s390x&distri=sle&flavor=Server-DVD-Updates&machine=s390x-kvm-sle12&test=mru-install-minimal-with-addons&version=12-SP5" class="external">latest</a></p>
openQA Tests - action #107623 (Resolved): Investigate which update causes rootless_podman regress...https://progress.opensuse.org/issues/1076232022-02-25T08:13:10Zjlausuchjalausuch@suse.com
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>Since 24 Feb, one of the updates in <code>BASE_TEST_ISSUES</code> is causing a regression in rootless podman.<br>
<img src="https://progress.opensuse.org/attachments/download/12904/rootless%20podman.png" alt="" loading="lazy" /></p>
<p>It only happens in 15-SP3 Update jobs. <br>
<code>BASE_TEST_ISSUES=21897,22201,22540,22559,22637,22670,22688,22700,22721,22726,22748,22752,22756,22834,22837,22848,22857,22863,22881,22884,22909,22914,22921,22926,22933,22948,22951,22955,22961,22965,22970,22971,22973,22977,22990</code></p>
<p>podman version: 2.1.1</p>
<p>I have started doing some small investigation and neither <a href="https://smelt.suse.de/incident/22201/" class="external">https://smelt.suse.de/incident/22201/</a> or <a href="https://smelt.suse.de/incident/22843/" class="external">https://smelt.suse.de/incident/22843/</a> (buildah update) cause the issue.</p>
<p>openQA test in scenario sle-15-SP3-JeOS-for-kvm-and-xen-Updates-x86_64-jeos-containers@64bit-virtio-vga fails in<br>
<a href="https://openqa.suse.de/tests/8225835/modules/rootless_podman/steps/102" class="external">rootless_podman</a></p>
<a name="Test-suite-description"></a>
<h2 >Test suite description<a href="#Test-suite-description" class="wiki-anchor">¶</a></h2>
<a name="Reproducible"></a>
<h2 >Reproducible<a href="#Reproducible" class="wiki-anchor">¶</a></h2>
<p>Fails since (at least) Build <a href="https://openqa.suse.de/tests/8219750" class="external">20220224-1</a></p>
<a name="Expected-result"></a>
<h2 >Expected result<a href="#Expected-result" class="wiki-anchor">¶</a></h2>
<p>Last good: <a href="https://openqa.suse.de/tests/8214690" class="external">20220223-1</a> (or more recent)</p>
<a name="Further-details"></a>
<h2 >Further details<a href="#Further-details" class="wiki-anchor">¶</a></h2>
<p>Always latest result in this scenario: <a href="https://openqa.suse.de/tests/latest?arch=x86_64&distri=sle&flavor=JeOS-for-kvm-and-xen-Updates&machine=64bit-virtio-vga&test=jeos-containers&version=15-SP3" class="external">latest</a></p>
openQA Tests - action #107062 (Feedback): Multiple failures due to network issueshttps://progress.opensuse.org/issues/1070622022-02-18T08:27:41Zjlausuchjalausuch@suse.com
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>I will use this ticket to collect the different errors I observe in our tests (at least for QE-C squad) that fail due to network issues.<br>
Normally a restart helps to get the job green again (in case we need it green), but this is not the ideal solution.</p>
<p>The idea of this ticket is to collect more potential issues caught by reviewers and propose solutions for some of them, in the code (retry same command several times might help) or in the infra side. </p>
<p>There is an example for each error I found, but from my experience reviewing jobs every day, these failures happen multiple times a day and randomly (difficult to predict).</p>
<p>1) <strong>SUSEConnect timeouts</strong> -> <a href="https://openqa.suse.de/tests/8189768#step/docker/34">https://openqa.suse.de/tests/8189768#step/docker/34</a><br>
<code>Test died: command 'SUSEConnect -p sle-module-containers/${VERSION_ID}/${CPU} ' timed out at /usr/lib/os-autoinst/testapi.pm line 1039.</code></p>
<p>Or <a href="https://openqa.suse.de/tests/8193554#step/suseconnect_scc/8">https://openqa.suse.de/tests/8193554#step/suseconnect_scc/8</a><br>
<code>Test died: command 'SUSEConnect -r $regcode' timed out at /usr/lib/os-autoinst/testapi.pm line 950.</code></p>
<p>2) <strong><a href="updates.suse.com" class="external">updates.suse.com</a> not reachable</strong> -> <a href="https://openqa.suse.de/tests/8189697#step/image_docker/1110">https://openqa.suse.de/tests/8189697#step/image_docker/1110</a></p>
<pre><code>Retrieving: kmod-25-6.10.1.aarch64.rpm [.........error]
Abort, retry, ignore? [a/r/i/...? shows all options] (a): a
Download (curl) error for 'https://updates.suse.com/SUSE/Updates/SLE-Module-Basesystem/15-SP2/aarch64/update/aarch64/kmod-25-6.10.1.aarch64.rpm?nE0jiYdfiOdLYjH0o-llNN2xIDXncon0vYw8z1aBPGx00H9S1eN413vUsfSJnzFrVz-CoZoGtSdsPKIDRAOQy3Xw2Tac3Yx5_1i8TPomSNiqhDJ0Ayxro23n46NHHB-XHq669RlHs17wiUFSJiSMCSh-YzdGdFw':
Error code: Connection failed
Error message: Could not resolve host: updates.suse.com
Problem occurred during or after installation or removal of packages:
Installation has been aborted as directed.
Please see the above error message for a hint.
</code></pre>
<p>3) <strong>SCC timeouts</strong> -> <a href="https://openqa.suse.de/tests/8189613#step/image_docker/316">https://openqa.suse.de/tests/8189613#step/image_docker/316</a></p>
<pre><code>docker run --entrypoint /usr/lib/zypp/plugins/services/container-suseconnect-zypp -i zypper_docker_derived lp
...
2022/02/18 07:16:19 Installed product: SLES-12.3-x86_64
2022/02/18 07:16:19 Registration server set to https://scc.suse.com
2022/02/18 07:16:30 Get https://scc.suse.com/connect/subscriptions/products?arch=x86_64&identifier=SLES&version=12.3: dial tcp: lookup scc.suse.com on 10.0.2.3:53: read udp 172.17.0.2:37151->10.0.2.3:53: i/o timeout
</code></pre>
<p>4) <strong>zypper ref timeout or error</strong> -> <a href="https://openqa.opensuse.org/tests/2193730#step/image_podman/124">https://openqa.opensuse.org/tests/2193730#step/image_podman/124</a></p>
<pre><code>podman run -i --name 'refreshed' --entrypoint '' registry.opensuse.org/opensuse/leap/15.3/images/totest/containers/opensuse/leap:15.3 zypper -nv ref
...
Retrieving: cb71cb070e8aac79327e6f1b6edc5317122ca1f72970299c3cb2cf505e18b27f-deltainfo.xml.gz [........................done (82.3 KiB/s)]
Retrieving: 832729371fe20bc1a4d27e59d76c10ffe2c0b5a1ff71c4e934e7a11baa24a74b-primary.xml.gz [............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................error (87.0 KiB/s)]
WJdDM-124-
</code></pre>
<a name="Acceptance-criteria"></a>
<h2 >Acceptance criteria<a href="#Acceptance-criteria" class="wiki-anchor">¶</a></h2>
<ul>
<li><strong>AC1:</strong> All existing subtasks are resolved, no additional work needed on top</li>
</ul>
openQA Tests - action #95697 (New): [kernel][jeos][opensuse] Have a common way to add LTP reposit...https://progress.opensuse.org/issues/956972021-07-20T08:34:37Zjlausuchjalausuch@suse.com
<p>Currently, LTP tests in SLE use QA_HEAD_REPO variable<br>
<a href="https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/tests/kernel/install_ltp.pm" class="external">https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/tests/kernel/install_ltp.pm</a></p>
<pre><code> if (is_sle) {
add_qa_head_repo;
return;
}
</code></pre>
<p>Then, for openSUSE, the openSUSE tests, the condition is a bit complex:</p>
<pre><code> my $arch = '';
$arch = "_PowerPC" if is_ppc64le();
$arch = "_zSystems" if is_s390x();
$arch = ((is_x86_64 || is_aarch64) ? "Tumbleweed" : "Factory") . $arch;
$repo = "https://download.opensuse.org/repositories/benchmark:/ltp:/devel/openSUSE_$arch/";
</code></pre>
<p>and even more complex after <a href="https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/12902" class="external">this PR</a>.</p>
<p>The idea behind this ticket is to use the same function (e.g. <code>add_qa_head_repo</code>) for ALL distri/versions using a single variable (e.g. <code>QA_HEAD_REPO</code>) pointing to the repository to be used, instead of hardcoding the repository with several conditions in the code. This would affect all the kernel jobs (also for JeOS-kernel jobs) for TW and Leap in O3.</p>
openQA Tests - action #93339 (Resolved): test fails in validate_btrfshttps://progress.opensuse.org/issues/933392021-06-01T09:16:42Zjlausuchjalausuch@suse.com
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>openQA test in scenario sle-15-SP3-Server-DVD-Updates-x86_64-sle_image_on_sle_host@64bit fails in<br>
<a href="https://openqa.suse.de/tests/6149331/modules/validate_btrfs/steps/39" class="external">validate_btrfs</a><br>
Other failures:<br>
All Container jobs, e.g. <a href="https://openqa.suse.de/tests/6151606" class="external">https://openqa.suse.de/tests/6151606</a><br>
All Public Cloud jobs, e.g. <a href="https://openqa.suse.de/tests/6149926#step/validate_btrfs/39" class="external">https://openqa.suse.de/tests/6149926#step/validate_btrfs/39</a></p>
<a name="Test-suite-description"></a>
<h2 >Test suite description<a href="#Test-suite-description" class="wiki-anchor">¶</a></h2>
<p>The base test suite is used for job templates defined in YAML documents. It has no settings of its own.</p>
<a name="Reproducible"></a>
<h2 >Reproducible<a href="#Reproducible" class="wiki-anchor">¶</a></h2>
<p>Fails since (at least) Build <a href="https://openqa.suse.de/tests/6149331" class="external">20210601-1</a> (current job)</p>
<a name="Expected-result"></a>
<h2 >Expected result<a href="#Expected-result" class="wiki-anchor">¶</a></h2>
<p>Last good: (unknown) (or more recent)</p>
<a name="Further-details"></a>
<h2 >Further details<a href="#Further-details" class="wiki-anchor">¶</a></h2>
<p>Always latest result in this scenario: <a href="https://openqa.suse.de/tests/latest?arch=x86_64&distri=sle&flavor=Server-DVD-Updates&machine=64bit&test=sle_image_on_sle_host&version=15-SP3" class="external">latest</a></p>
openQA Tests - action #89287 (Resolved): test fails in yast2_lanhttps://progress.opensuse.org/issues/892872021-03-01T14:48:17Zjlausuchjalausuch@suse.com
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>openQA test in scenario sle-15-SP3-JeOS-for-MS-HyperV-x86_64-jeos-main@svirt-hyperv fails in<br>
<a href="https://openqa.suse.de/tests/5552124/modules/yast2_lan/steps/21" class="external">yast2_lan</a></p>
<a name="Test-suite-description"></a>
<h2 >Test suite description<a href="#Test-suite-description" class="wiki-anchor">¶</a></h2>
<a name="Reproducible"></a>
<h2 >Reproducible<a href="#Reproducible" class="wiki-anchor">¶</a></h2>
<p>Fails since (at least) Build <a href="https://openqa.suse.de/tests/5462492" class="external">23.32</a></p>
<a name="Expected-result"></a>
<h2 >Expected result<a href="#Expected-result" class="wiki-anchor">¶</a></h2>
<p>Last good: (unknown) (or more recent)</p>
<a name="Further-details"></a>
<h2 >Further details<a href="#Further-details" class="wiki-anchor">¶</a></h2>
<p>Always latest result in this scenario: <a href="https://openqa.suse.de/tests/latest?arch=x86_64&distri=sle&flavor=JeOS-for-MS-HyperV&machine=svirt-hyperv&test=jeos-main&version=15-SP3" class="external">latest</a></p>
openQA Tests - action #66949 (Resolved): [qac][wicked] New test: wicked ifreload all fails to det...https://progress.opensuse.org/issues/669492020-05-18T06:42:57Zjlausuchjalausuch@suse.com
<p>Provide a new test covering bridges and basic wicked function <code>ifreload all</code>.</p>
<p>Basically, we should create a test to cover this bug: <br>
<a href="https://gitlab.suse.de/wicked-maintainers/wicked/-/issues/156" class="external">https://gitlab.suse.de/wicked-maintainers/wicked/-/issues/156</a><br>
<a href="http://bugzilla.suse.com/show_bug.cgi?id=1168155" class="external">http://bugzilla.suse.com/show_bug.cgi?id=1168155</a></p>
openQA Tests - action #65121 (Resolved): [qac][public cloud][ltp] ioctl08 test from LTP syscalls ...https://progress.opensuse.org/issues/651212020-04-01T08:06:25Zjlausuchjalausuch@suse.com
<p>Some failed jobs: <br>
<a href="https://openqa.suse.de/tests/4070442" class="external">https://openqa.suse.de/tests/4070442</a><br>
<a href="https://openqa.suse.de/tests/4070449" class="external">https://openqa.suse.de/tests/4070449</a></p>
<pre><code>Summary:
passed 1
failed 0
skipped 0
warnings 0
cmd-exit-406-0
sh-4.4# ioctl08; echo cmd-exit-407-$?
tst_device.c:88: INFO: Found free device 0 '/dev/loop0'
tst_mkfs.c:90: INFO: Formatting /dev/loop0 with btrfs opts='' extra opts=''
cmd-exit-407-255
sh-4.4# printf tainted-; cat /proc/sys/kernel/tainted; echo cmd-exit-408-$?
tainted-0
cmd-exit-408-0
sh-4.4# ioctl_ns01; echo cmd-exit-409-$?
tst_test.c:1241: INFO: Timeout per run is 0h 05m 00s
ioctl_ns01.c:57: PASS: NS_GET_PARENT fails with EPERM
ioctl_ns01.c:57: PASS: NS_GET_PARENT fails with EPERM
</code></pre>