openSUSE Project Management Tool: Issueshttps://progress.opensuse.org/https://progress.opensuse.org/themes/openSUSE/favicon/favicon.ico?15829177842024-03-21T09:51:42ZopenSUSE Project Management Tool
Redmine openQA Tests - action #157657 (Feedback): PackageHub module activation while using SCC-proxyhttps://progress.opensuse.org/issues/1576572024-03-21T09:51:42Zjlausuchjalausuch@suse.com
<a name="Problem-statement"></a>
<h2 >Problem statement:<a href="#Problem-statement" class="wiki-anchor">¶</a></h2>
<p>When registering the system using ProxySCC, activating the PackageHub module returns repos in OSD that don't exist.<br>
This is expected, because the proxySCC replaces the repo URLs given by the real SCC by local openQA asset URL.</p>
<p>Normally, we use the proxySCC in the openQA jobs with <code>SCC_URL</code> variable, this will be used by the code to make the <code>SUSEConnect</code> call like this:</p>
<p><code>SUSEConnect -r $REGCODE --url http://all-67.1.proxy.scc.suse.de</code></p>
<p>This will make that all the repos returned look like this:<br>
<code>http://openqa.suse.de/assets/repo/...</code></p>
<p>And enabling any module, will have the same effect, for example PackageHub:</p>
<pre><code>
susetest:~ # suseconnect -p PackageHub/15.6/x86_64
Registering system to registration proxy http://all-67.1.proxy.scc.suse.de
Updating system details on http://all-67.1.proxy.scc.suse.de ...
Activating PackageHub 15.6 x86_64 ...
-> Adding service to system ...
-> Installing release package ...
Successfully registered system
</code></pre>
<p>And the repo will point to the wrong place:</p>
<pre><code>18 | SUSE_Package_Hub_15_SP6_x86_64:SLE-Module-Packagehub-Subpackages15-SP6-Pool | SLE-Module-Packagehub-Subpackages15-SP6-Pool | Yes | (r ) Yes | No | http://openqa.suse.de/assets/repo/SLE-15-SP6-Module-Packagehub-Subpackages-POOL-x86_64-Build67.1-Media1/
</code></pre>
<a name="Solution"></a>
<h2 >Solution<a href="#Solution" class="wiki-anchor">¶</a></h2>
<p>To cope with this, we would need to sync the PackageHub module for EVERY build we have for 15-SP6. Another solution would be to tweak ProxySCC code to return the real URL instead of local assets.</p>
<p>BUT there is an easier way:<br>
1) Register the system using the usual proxySCC</p>
<pre><code>susetest:~ # SUSEConnect -r $REGCODE --url http://all-67.1.proxy.scc.suse.de
Registering system to registration proxy http://all-67.1.proxy.scc.suse.de
Announcing system to http://all-67.1.proxy.scc.suse.de ...
Activating SLES 15.6 x86_64 ...
-> Adding service to system ...
Activating sle-module-basesystem 15.6 x86_64 ...
-> Adding service to system ...
-> Installing release package ...
Activating sle-module-server-applications 15.6 x86_64 ...
-> Adding service to system ...
-> Installing release package ...
Activating sle-module-python3 15.6 x86_64 ...
-> Adding service to system ...
-> Installing release package ...
Successfully registered system
</code></pre>
<p>2) Activate the desired module using <code>--url</code> pointing to real SCC:</p>
<pre><code>susetest:~ # SUSEConnect -p PackageHub/15.6/x86_64 --url https://scc.suse.com
Registering system to SUSE Customer Center
Updating system details on https://scc.suse.com ...
Activating PackageHub 15.6 x86_64 ...
-> Adding service to system ...
-> Installing release package ...
Successfully registered system
</code></pre>
<p>3) And voilá, you will get the repos from REAL SCC:</p>
<pre><code>susetest:~ # zypper lr -u |grep -i PackageHub-15-SP6-Pool
22 | SUSE_Package_Hub_15_SP6_x86_64:SUSE-PackageHub-15-SP6-Pool | SUSE-PackageHub-15-SP6-Pool | Yes | (r ) Yes | No | https://updates.suse.com/SUSE/Backports/SLE-15-SP6_x86_64/product?RMMzfgZAuv8Hy903n30FsW6hbWs5KExVGGIwF1w5nRMdkliFvRBscHmgomWpHwFbG8l2C7cmrcZead-ejTo_f0NKbts7ZNwGBw0WtMwuWeKXxK2mgknqmbcYZ22y3RB3rThU1AMZTw
</code></pre> openQA Tests - action #136298 (Resolved): Leap Micro test fails in transactional_updatehttps://progress.opensuse.org/issues/1362982023-09-22T06:11:38Zjlausuchjalausuch@suse.com
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>openQA test in scenario leap-micro-5.5-MicroOS-Image-x86_64-microos_image_default@64bit fails in<br>
<a href="https://openqa.opensuse.org/tests/3591732/modules/transactional_update/steps/96" class="external">transactional_update</a></p>
<p>As the job says, there should be an expected change when <a href="https://openqa.opensuse.org/tests/3591732#step/transactional_update/23" class="external">installing the local rpm</a><br>
<code>transactional-update -n ptf install update-test-trivial/update-test-security-5.1-1.20.x86_64.rpm</code><br>
and <a href="https://openqa.opensuse.org/tests/3591732#step/transactional_update/73" class="external">adding the utt repo</a> along with <code>transactional-update -n cleanup up</code>. </p>
<p>The local package is <code>/data/microos/utt-opensuse-x86_64.tgz</code> and the repo is <code>/data/microos/utt-leap.repo</code>, both in the data directory:<br>
<a href="https://github.com/os-autoinst/os-autoinst-distri-opensuse/tree/master/data/microos" class="external">https://github.com/os-autoinst/os-autoinst-distri-opensuse/tree/master/data/microos</a></p>
<p>This was working on Leap Micro 5.4 (no existing jobs), so for some reason now the packages seem to be the same or the local rpm is already a higher version than the one in the repo, and zypper says <code>zypper: nothing to update</code>.</p>
<a name="Test-suite-description"></a>
<h2 >Test suite description<a href="#Test-suite-description" class="wiki-anchor">¶</a></h2>
<p>MicroOS image boot with ignition disk and default tests. Default tests are transactional-update, rebootmgr, health_check, cockpit service and some other checks specific to MicroOS.</p>
<a name="Reproducible"></a>
<h2 >Reproducible<a href="#Reproducible" class="wiki-anchor">¶</a></h2>
<p>Fails since the beginning.</p>
ALP - action #126716 (Resolved): test fails in run_container_in_k3s - /usr/bin/k3s-install: No su...https://progress.opensuse.org/issues/1267162023-03-27T13:48:34Zjlausuchjalausuch@suse.com
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>openQA test in scenario alp-micro-0.1-Default-x86_64-k3s@64bit fails in<br>
<a href="https://openqa.opensuse.org/tests/3194054/modules/run_container_in_k3s/steps/17" class="external">run_container_in_k3s</a></p>
<a name="Test-suite-description"></a>
<h2 >Test suite description<a href="#Test-suite-description" class="wiki-anchor">¶</a></h2>
<p>The base test suite is used for job templates defined in YAML documents. It has no settings of its own.</p>
<a name="Reproducible"></a>
<h2 >Reproducible<a href="#Reproducible" class="wiki-anchor">¶</a></h2>
<p>Fails since (at least) Build <a href="https://openqa.opensuse.org/tests/3183089" class="external">5.2</a></p>
<a name="Expected-result"></a>
<h2 >Expected result<a href="#Expected-result" class="wiki-anchor">¶</a></h2>
<p>Last good: (unknown) (or more recent)</p>
<a name="Further-details"></a>
<h2 >Further details<a href="#Further-details" class="wiki-anchor">¶</a></h2>
<p>Always latest result in this scenario: <a href="https://openqa.opensuse.org/tests/latest?arch=x86_64&distri=alp&flavor=Default&machine=64bit&test=k3s&version=micro-0.1" class="external">latest</a></p>
ALP - action #125855 (Resolved): Create OBS-sync scripts for ALP Micro and Bedrockhttps://progress.opensuse.org/issues/1258552023-03-13T07:52:34Zjlausuchjalausuch@suse.com
<p>Create new sync scripts in <a href="https://github.com/os-autoinst/openqa-trigger-from-obs" class="external">openqa-trigger-from-obs</a></p>
<p>Currently we have 2 sync scripts that we used for the ALP December prototype:<br>
<a href="https://github.com/os-autoinst/openqa-trigger-from-obs/blob/master/xml/obs/SUSE:ALP:ToTest.xml" class="external">https://github.com/os-autoinst/openqa-trigger-from-obs/blob/master/xml/obs/SUSE:ALP:ToTest.xml</a><br>
<a href="https://github.com/os-autoinst/openqa-trigger-from-obs/blob/master/xml/obs/SUSE:ALP:Staging.xml" class="external">https://github.com/os-autoinst/openqa-trigger-from-obs/blob/master/xml/obs/SUSE:ALP:Staging.xml</a></p>
<p>But this project in IBS is obsolete and we should now sync the new images from the 2 new IBS projects:<br>
<a href="https://build.opensuse.org/project/show/SUSE:ALP:Products:Micro:0.1" class="external">https://build.opensuse.org/project/show/SUSE:ALP:Products:Micro:0.1</a><br>
<a href="https://build.opensuse.org/project/show/SUSE:ALP:Products:Bedrock:0.1" class="external">https://build.opensuse.org/project/show/SUSE:ALP:Products:Bedrock:0.1</a></p>
<p><a href="https://download.opensuse.org/repositories/SUSE:/ALP:/Products:/" class="external">https://download.opensuse.org/repositories/SUSE:/ALP:/Products:/</a></p>
<a name="Acceptance-Criteria"></a>
<h2 >Acceptance Criteria<a href="#Acceptance-Criteria" class="wiki-anchor">¶</a></h2>
<ul>
<li>Create new sync script for ALP Micro in <a href="https://github.com/os-autoinst/openqa-trigger-from-obs" class="external">openqa-trigger-from-obs</a></li>
<li>Create new sync script for ALP Bedrock in <a href="https://github.com/os-autoinst/openqa-trigger-from-obs" class="external">openqa-trigger-from-obs</a></li>
</ul>
ALP - action #125852 (Resolved): Create 2 new job groups in openqa.opensuse.org to hold the 2 new...https://progress.opensuse.org/issues/1258522023-03-13T07:44:11Zjlausuchjalausuch@suse.com
<p>So far, we have been using a <a href="https://openqa.opensuse.org/group_overview/100" class="external">job group</a> for ALP images, but that product in IBS is obsolete.</p>
<p>Instead, we now have 2 flavors corresponding to the 2 new products (see epic).</p>
<p>For now, we can keep the set of test suites the same as we had for ALP December, and we can do modifications as needed.</p>
<a name="Acceptance-criteria"></a>
<h2 >Acceptance criteria<a href="#Acceptance-criteria" class="wiki-anchor">¶</a></h2>
<ul>
<li>Create 2 new jobs in O3: <code>ALP-Micro</code> and <code>ALP-Bedrock</code></li>
<li>Create corresponding mediums</li>
<li>Create the content of the groups in <a href="https://github.com/os-autoinst/opensuse-jobgroups" class="external">opensuse-jobgroups</a></li>
</ul>
openQA Project - action #113219 (Closed): Create a master label at job group level to label all t...https://progress.opensuse.org/issues/1132192022-07-04T18:53:37Zjlausuchjalausuch@suse.com
<p>As a openQA user, I would like to have a feature to label all the jobs of the same build at the same time with certain <code>bsc</code> or <code>poo</code>, instead of labeling one by one.</p>
<p>A possible use case is in container image tests (BCI) where a new container image build is tested on multiple host versions and architectures to be validated. Sometimes, the image is faulty and we need to label all the jobs (~40) which makes thing very time consuming for the reviewer.<br>
For example, <a href="https://openqa.suse.de/tests/overview?distri=sle&version=15-SP4&build=7.12_pcp-image&groupid=445" class="external">this build</a> or <a href="https://openqa.suse.de/tests/overview?distri=sle&version=15-SP4&build=7.9_pcp-image&groupid=445" class="external">this</a> are mostly failed tests.</p>
<p>It would be nice to use the <code>tag</code> feature + the <code>build</code> number doing something like this:<br>
<img src="https://progress.opensuse.org/attachments/download/13487/Screenshot%202022-07-04%20at%2020.52.21.png" alt="" loading="lazy" /><br>
and automatically label all the jobs.</p>
ALP - action #112826 (Resolved): Fix ALP SelfInstall jobshttps://progress.opensuse.org/issues/1128262022-06-21T21:27:06Zjlausuchjalausuch@suse.com
<p>These jobs fail booting the image:<br>
<a href="https://openqa.opensuse.org/tests/2427880#step/bootloader_uefi/2" class="external">https://openqa.opensuse.org/tests/2427880#step/bootloader_uefi/2</a></p>
<p>Expected result (Leap Micro):<br>
<a href="https://openqa.opensuse.org/tests/2426647#step/bootloader_uefi/3" class="external">https://openqa.opensuse.org/tests/2426647#step/bootloader_uefi/3</a></p>
openQA Project - action #111135 (New): Enhance email notification message content for about faile...https://progress.opensuse.org/issues/1111352022-05-16T07:57:05Zjlausuchjalausuch@suse.com
<p>The new feature introduced by <a href="https://progress.opensuse.org/issues/91605" class="external">https://progress.opensuse.org/issues/91605</a> is very useful to notify people in different ways (direct email or to Slack, which will turn into Slack message). However, those messages could be improved adding some extra information about the test name, the group name, etc.</p>
<p>This is an example of how a message in Slack looks like:<br>
<img src="https://progress.opensuse.org/attachments/download/13239/slack_message.png" alt="" loading="lazy" /></p>
<p>So, a proposal from my side could be:</p>
<pre><code>Unknown issue to be reviewed.
OpenQA test https://openqa.suse.de/tests/8762619 fails with
"Test died: script timeout: docker info at /usr/lib/os-autoinst/distribution.pm line 296."
</code></pre>
<p>I guess it's difficult to include the reason given by the failure, so something like this could be also helpful:</p>
<pre><code>Unknown issue to be reviewed.
OpenQA test https://openqa.suse.de/tests/8762619 fails in docker.
Job Group: 427 - Maintenance: Test Repo / Public Cloud Maintenance Updates
Build: 20220515-1
Flavor: AZURE-CHOST-BYOS-Updates
</code></pre>
<p>similar to when you report a poo ticket directly from the UI. </p>
openQA Tests - action #111010 (Resolved): type_password does not hide the string in autoinst-log.txthttps://progress.opensuse.org/issues/1110102022-05-12T11:30:11Zjlausuchjalausuch@suse.com
<p>I am doing some tests using <code>type_password</code> command to put some sensitive information into a file in a running system.<br>
According to the documentation [1]: <code>A convenience wrapper around type_string, which doesn’t log the string.</code></p>
<p>However, if I do:<br>
<code>type_password("echo 'yada yada' > /root/influxdb_conf\n");</code></p>
<p>This info is shown in autoinst-log.txt:</p>
<pre><code>[2022-05-12T13:05:09.897059+02:00] [debug] tests/jeos/image_info.pm:130 called testapi::type_password
[2022-05-12T13:05:09.897329+02:00] [debug] <<< testapi::type_string(string="echo 'yada yada' > /root/influxdb_conf\n", max_interval=100, wait_screen_changes=0, wait_still_screen=0, timeout=30, similarity_level=47)
</code></pre>
<p>This is using root-console. If I use serial_terminal, I get it in the <code>serial_terminal.txt</code> log:</p>
<pre><code>...
SCRIPT_FINISHEDGItcm-0-
# echo 'yada yada' > /root/influxdb_conf
</code></pre>
<p>Also, openQA tests are showing this </p>
<p>2 months ago (before <a href="https://github.com/os-autoinst/os-autoinst/pull/2002" class="external">https://github.com/os-autoinst/os-autoinst/pull/2002</a>)<br>
<a href="https://openqa.suse.de/tests/8405529/logfile?filename=autoinst-log.txt" class="external">https://openqa.suse.de/tests/8405529/logfile?filename=autoinst-log.txt</a></p>
<pre><code>[2022-03-26T12:33:40.095557+01:00] [debug] tests/jeos/firstrun.pm:116 called testapi::type_password
[2022-03-26T12:33:40.095850+01:00] [debug] <<< testapi::type_string(string="SECRET STRING", max_interval=100, wait_screen_changes=0, wait_still_screen=0, timeout=30, similarity_level=47)
</code></pre>
<p>Now:<br>
<a href="https://openqa.suse.de/tests/8738443/logfile?filename=autoinst-log.txt" class="external">https://openqa.suse.de/tests/8738443/logfile?filename=autoinst-log.txt</a></p>
<pre><code>[2022-05-12T06:09:58.666172+02:00] [debug] tests/jeos/firstrun.pm:116 called testapi::type_password
[2022-05-12T06:09:58.666561+02:00] [debug] <<< testapi::type_string(string="nots3cr3t", max_interval=100, wait_screen_changes=0, wait_still_screen=0, timeout=30, similarity_level=47)
</code></pre>
<p>[1] <a href="http://open.qa/api/testapi/#_type_password" class="external">http://open.qa/api/testapi/#_type_password</a></p>
openQA Tests - action #109882 (Resolved): Leap Micro 5.2 autoyast installation misses the repo-ma...https://progress.opensuse.org/issues/1098822022-04-12T22:14:27Zjlausuchjalausuch@suse.com
<p>After doing an <a href="https://openqa.opensuse.org/tests/2293631" class="external">autoyast installation</a>, the system doesn't have the repository </p>
<pre><code>install:~ # zypper lr -u
Repository priorities are without effect. All enabled repositories share the same priority.
# | Alias | Name | Enabled | GPG Check | Refresh | URI
--+---------------------------+---------------------------+---------+-----------+---------+--------------------------------------------------------
1 | openSUSE-Leap-Micro-5.2-1 | openSUSE-Leap-Micro-5.2-1 | No | ---- | ---- | cd:/?devices=/dev/disk/by-id/scsi-0QEMU_QEMU_CD-ROM_cd0
</code></pre>
<p>And because of this, the installation of some packages fail (e.g. <a href="https://openqa.opensuse.org/tests/2293631#step/cockpit_service/8" class="external">cockpit</a>)</p>
<p>This is how it should look like from the regular <a href="https://openqa.opensuse.org/tests/2294237" class="external">DVD installation</a>:</p>
<pre><code>localhost:~ # zypper lr -u
Repository priorities are without effect. All enabled repositories share the same priority.
# | Alias | Name | Enabled | GPG Check | Refresh | URI
--+---------------------------+----------------------------+---------+-----------+---------+----------------------------------------------------------------------------------------------------
1 | openSUSE-Leap-Micro-5.2-1 | openSUSE-Leap-Micro-5.2-1 | No | ---- | ---- | cd:/?devices=/dev/disk/by-id/scsi-0QEMU_QEMU_CD-ROM_cd0
2 | repo-main | Leap Micro Main Repository | Yes | (r ) Yes | Yes | https://download.opensuse.org/distribution/leap-micro/5.2/product/repo/Leap-Micro-5.2-x86_64-Media/
</code></pre>
<p>The used autoyast profile is <a href="https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/data/autoyast_opensuse/autoyast_leap-micro.xml.ep" class="external">https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/data/autoyast_opensuse/autoyast_leap-micro.xml.ep</a></p>
openQA Project - action #108824 (Resolved): Some of the daily aggregate tests are cancelled witho...https://progress.opensuse.org/issues/1088242022-03-24T07:02:27Zjlausuchjalausuch@suse.com
<p>Some of the tests that are triggered by the bot-ng are cancelled without any apparent reason and missing logs.</p>
<p>Examples:<br>
<a href="https://openqa.suse.de/tests/8379473" class="external">https://openqa.suse.de/tests/8379473</a><br>
<a href="https://openqa.suse.de/tests/8379476" class="external">https://openqa.suse.de/tests/8379476</a></p>
<p>I have observed this a few times in past days, but I thought it was a sporadic error.<br>
Let's use this ticket to collect these kind of failures.</p>
<a name="Acceptance-criteria"></a>
<h2 >Acceptance criteria<a href="#Acceptance-criteria" class="wiki-anchor">¶</a></h2>
<ul>
<li><strong>AC1</strong>: It is clear what the expected scheduling behavior is (e.g. is VERSION <em>supposed</em> to be 5.1 in the job despite VERSION 5.0 being specified when scheduling the product)</li>
</ul>
<a name="Suggestions"></a>
<h2 >Suggestions<a href="#Suggestions" class="wiki-anchor">¶</a></h2>
<ul>
<li>Clarify with the author of the test - this test is very new
<ul>
<li>Initial hypothesis: Have these jobs <em>not</em> been scheduled by bot-ng?</li>
<li>Find a way to distinguish bot jobs better</li>
<li>Find out why affected jobs are scheduled with 5.0 but show VERSION 5.1 in the settings</li>
</ul></li>
<li><a href="https://gitlab.suse.de/qac/qac-openqa-yaml/-/blob/master/sle-micro/updates.yaml#L170" class="external">https://gitlab.suse.de/qac/qac-openqa-yaml/-/blob/master/sle-micro/updates.yaml#L170</a>
<ul>
<li>Examine bot-ng logs for scheduling the sle-micro, look for both 5.0 and 5.1 versions (the other one could obsolete the first one)</li>
<li>Product log for 5.0 scheduling: <a href="https://openqa.suse.de/admin/productlog?id=886973" class="external">https://openqa.suse.de/admin/productlog?id=886973</a></li>
<li>Product log for 5.1 scheduling: <a href="https://openqa.suse.de/admin/productlog?id=886978" class="external">https://openqa.suse.de/admin/productlog?id=886978</a></li>
</ul></li>
<li>If the bot cancells jobs as expected, comment "Cancelled because of job #foo" on relevant jobs</li>
</ul>
openQA Tests - action #107623 (Resolved): Investigate which update causes rootless_podman regress...https://progress.opensuse.org/issues/1076232022-02-25T08:13:10Zjlausuchjalausuch@suse.com
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>Since 24 Feb, one of the updates in <code>BASE_TEST_ISSUES</code> is causing a regression in rootless podman.<br>
<img src="https://progress.opensuse.org/attachments/download/12904/rootless%20podman.png" alt="" loading="lazy" /></p>
<p>It only happens in 15-SP3 Update jobs. <br>
<code>BASE_TEST_ISSUES=21897,22201,22540,22559,22637,22670,22688,22700,22721,22726,22748,22752,22756,22834,22837,22848,22857,22863,22881,22884,22909,22914,22921,22926,22933,22948,22951,22955,22961,22965,22970,22971,22973,22977,22990</code></p>
<p>podman version: 2.1.1</p>
<p>I have started doing some small investigation and neither <a href="https://smelt.suse.de/incident/22201/" class="external">https://smelt.suse.de/incident/22201/</a> or <a href="https://smelt.suse.de/incident/22843/" class="external">https://smelt.suse.de/incident/22843/</a> (buildah update) cause the issue.</p>
<p>openQA test in scenario sle-15-SP3-JeOS-for-kvm-and-xen-Updates-x86_64-jeos-containers@64bit-virtio-vga fails in<br>
<a href="https://openqa.suse.de/tests/8225835/modules/rootless_podman/steps/102" class="external">rootless_podman</a></p>
<a name="Test-suite-description"></a>
<h2 >Test suite description<a href="#Test-suite-description" class="wiki-anchor">¶</a></h2>
<a name="Reproducible"></a>
<h2 >Reproducible<a href="#Reproducible" class="wiki-anchor">¶</a></h2>
<p>Fails since (at least) Build <a href="https://openqa.suse.de/tests/8219750" class="external">20220224-1</a></p>
<a name="Expected-result"></a>
<h2 >Expected result<a href="#Expected-result" class="wiki-anchor">¶</a></h2>
<p>Last good: <a href="https://openqa.suse.de/tests/8214690" class="external">20220223-1</a> (or more recent)</p>
<a name="Further-details"></a>
<h2 >Further details<a href="#Further-details" class="wiki-anchor">¶</a></h2>
<p>Always latest result in this scenario: <a href="https://openqa.suse.de/tests/latest?arch=x86_64&distri=sle&flavor=JeOS-for-kvm-and-xen-Updates&machine=64bit-virtio-vga&test=jeos-containers&version=15-SP3" class="external">latest</a></p>
openQA Tests - action #107062 (Feedback): Multiple failures due to network issueshttps://progress.opensuse.org/issues/1070622022-02-18T08:27:41Zjlausuchjalausuch@suse.com
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>I will use this ticket to collect the different errors I observe in our tests (at least for QE-C squad) that fail due to network issues.<br>
Normally a restart helps to get the job green again (in case we need it green), but this is not the ideal solution.</p>
<p>The idea of this ticket is to collect more potential issues caught by reviewers and propose solutions for some of them, in the code (retry same command several times might help) or in the infra side. </p>
<p>There is an example for each error I found, but from my experience reviewing jobs every day, these failures happen multiple times a day and randomly (difficult to predict).</p>
<p>1) <strong>SUSEConnect timeouts</strong> -> <a href="https://openqa.suse.de/tests/8189768#step/docker/34">https://openqa.suse.de/tests/8189768#step/docker/34</a><br>
<code>Test died: command 'SUSEConnect -p sle-module-containers/${VERSION_ID}/${CPU} ' timed out at /usr/lib/os-autoinst/testapi.pm line 1039.</code></p>
<p>Or <a href="https://openqa.suse.de/tests/8193554#step/suseconnect_scc/8">https://openqa.suse.de/tests/8193554#step/suseconnect_scc/8</a><br>
<code>Test died: command 'SUSEConnect -r $regcode' timed out at /usr/lib/os-autoinst/testapi.pm line 950.</code></p>
<p>2) <strong><a href="updates.suse.com" class="external">updates.suse.com</a> not reachable</strong> -> <a href="https://openqa.suse.de/tests/8189697#step/image_docker/1110">https://openqa.suse.de/tests/8189697#step/image_docker/1110</a></p>
<pre><code>Retrieving: kmod-25-6.10.1.aarch64.rpm [.........error]
Abort, retry, ignore? [a/r/i/...? shows all options] (a): a
Download (curl) error for 'https://updates.suse.com/SUSE/Updates/SLE-Module-Basesystem/15-SP2/aarch64/update/aarch64/kmod-25-6.10.1.aarch64.rpm?nE0jiYdfiOdLYjH0o-llNN2xIDXncon0vYw8z1aBPGx00H9S1eN413vUsfSJnzFrVz-CoZoGtSdsPKIDRAOQy3Xw2Tac3Yx5_1i8TPomSNiqhDJ0Ayxro23n46NHHB-XHq669RlHs17wiUFSJiSMCSh-YzdGdFw':
Error code: Connection failed
Error message: Could not resolve host: updates.suse.com
Problem occurred during or after installation or removal of packages:
Installation has been aborted as directed.
Please see the above error message for a hint.
</code></pre>
<p>3) <strong>SCC timeouts</strong> -> <a href="https://openqa.suse.de/tests/8189613#step/image_docker/316">https://openqa.suse.de/tests/8189613#step/image_docker/316</a></p>
<pre><code>docker run --entrypoint /usr/lib/zypp/plugins/services/container-suseconnect-zypp -i zypper_docker_derived lp
...
2022/02/18 07:16:19 Installed product: SLES-12.3-x86_64
2022/02/18 07:16:19 Registration server set to https://scc.suse.com
2022/02/18 07:16:30 Get https://scc.suse.com/connect/subscriptions/products?arch=x86_64&identifier=SLES&version=12.3: dial tcp: lookup scc.suse.com on 10.0.2.3:53: read udp 172.17.0.2:37151->10.0.2.3:53: i/o timeout
</code></pre>
<p>4) <strong>zypper ref timeout or error</strong> -> <a href="https://openqa.opensuse.org/tests/2193730#step/image_podman/124">https://openqa.opensuse.org/tests/2193730#step/image_podman/124</a></p>
<pre><code>podman run -i --name 'refreshed' --entrypoint '' registry.opensuse.org/opensuse/leap/15.3/images/totest/containers/opensuse/leap:15.3 zypper -nv ref
...
Retrieving: cb71cb070e8aac79327e6f1b6edc5317122ca1f72970299c3cb2cf505e18b27f-deltainfo.xml.gz [........................done (82.3 KiB/s)]
Retrieving: 832729371fe20bc1a4d27e59d76c10ffe2c0b5a1ff71c4e934e7a11baa24a74b-primary.xml.gz [............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................error (87.0 KiB/s)]
WJdDM-124-
</code></pre>
<a name="Acceptance-criteria"></a>
<h2 >Acceptance criteria<a href="#Acceptance-criteria" class="wiki-anchor">¶</a></h2>
<ul>
<li><strong>AC1:</strong> All existing subtasks are resolved, no additional work needed on top</li>
</ul>
openQA Tests - action #93339 (Resolved): test fails in validate_btrfshttps://progress.opensuse.org/issues/933392021-06-01T09:16:42Zjlausuchjalausuch@suse.com
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>openQA test in scenario sle-15-SP3-Server-DVD-Updates-x86_64-sle_image_on_sle_host@64bit fails in<br>
<a href="https://openqa.suse.de/tests/6149331/modules/validate_btrfs/steps/39" class="external">validate_btrfs</a><br>
Other failures:<br>
All Container jobs, e.g. <a href="https://openqa.suse.de/tests/6151606" class="external">https://openqa.suse.de/tests/6151606</a><br>
All Public Cloud jobs, e.g. <a href="https://openqa.suse.de/tests/6149926#step/validate_btrfs/39" class="external">https://openqa.suse.de/tests/6149926#step/validate_btrfs/39</a></p>
<a name="Test-suite-description"></a>
<h2 >Test suite description<a href="#Test-suite-description" class="wiki-anchor">¶</a></h2>
<p>The base test suite is used for job templates defined in YAML documents. It has no settings of its own.</p>
<a name="Reproducible"></a>
<h2 >Reproducible<a href="#Reproducible" class="wiki-anchor">¶</a></h2>
<p>Fails since (at least) Build <a href="https://openqa.suse.de/tests/6149331" class="external">20210601-1</a> (current job)</p>
<a name="Expected-result"></a>
<h2 >Expected result<a href="#Expected-result" class="wiki-anchor">¶</a></h2>
<p>Last good: (unknown) (or more recent)</p>
<a name="Further-details"></a>
<h2 >Further details<a href="#Further-details" class="wiki-anchor">¶</a></h2>
<p>Always latest result in this scenario: <a href="https://openqa.suse.de/tests/latest?arch=x86_64&distri=sle&flavor=Server-DVD-Updates&machine=64bit&test=sle_image_on_sle_host&version=15-SP3" class="external">latest</a></p>
openQA Tests - action #89287 (Resolved): test fails in yast2_lanhttps://progress.opensuse.org/issues/892872021-03-01T14:48:17Zjlausuchjalausuch@suse.com
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>openQA test in scenario sle-15-SP3-JeOS-for-MS-HyperV-x86_64-jeos-main@svirt-hyperv fails in<br>
<a href="https://openqa.suse.de/tests/5552124/modules/yast2_lan/steps/21" class="external">yast2_lan</a></p>
<a name="Test-suite-description"></a>
<h2 >Test suite description<a href="#Test-suite-description" class="wiki-anchor">¶</a></h2>
<a name="Reproducible"></a>
<h2 >Reproducible<a href="#Reproducible" class="wiki-anchor">¶</a></h2>
<p>Fails since (at least) Build <a href="https://openqa.suse.de/tests/5462492" class="external">23.32</a></p>
<a name="Expected-result"></a>
<h2 >Expected result<a href="#Expected-result" class="wiki-anchor">¶</a></h2>
<p>Last good: (unknown) (or more recent)</p>
<a name="Further-details"></a>
<h2 >Further details<a href="#Further-details" class="wiki-anchor">¶</a></h2>
<p>Always latest result in this scenario: <a href="https://openqa.suse.de/tests/latest?arch=x86_64&distri=sle&flavor=JeOS-for-MS-HyperV&machine=svirt-hyperv&test=jeos-main&version=15-SP3" class="external">latest</a></p>