openSUSE Project Management Tool: Issueshttps://progress.opensuse.org/https://progress.opensuse.org/themes/openSUSE/favicon/favicon.ico?15829177842021-02-17T10:11:04ZopenSUSE Project Management Tool
Redmine openQA Tests - action #88687 (New): [qe-tools] Use the TEST_SUITE_SUFFICIENT value to mark packag...https://progress.opensuse.org/issues/886872021-02-17T10:11:04Ztjyrinki_susetjyrinki+redmine@suse.de
<p>Reviewing and actually using the collected TEST_SUITE_SUFFICIENT data together with making template generator aware of it to have templates automatically acknowledge when build time regression test suite is enough.</p>
<p>The current machine readable data for automation is at <a href="https://pes.suse.de/QA_Maintenance/Automation_Progress_Tracking_in_QAM/" class="external">https://pes.suse.de/QA_Maintenance/Automation_Progress_Tracking_in_QAM/</a> and so that should be appended - probably with new sections for indirect and build time ones - as needed with packages that would be now marked as automatically tested.</p>
openQA Tests - coordination #88684 (Feedback): [qe-core][epic][qem] Identify packages that we are...https://progress.opensuse.org/issues/886842021-02-17T10:08:30Ztjyrinki_susetjyrinki+redmine@suse.de
<p>We test some packages indirectly as part of other packages' test. The template generator (<a href="https://gitlab.suse.de/qa-maintenance/metadata/-/tree/master/template_generator" class="external">https://gitlab.suse.de/qa-maintenance/metadata/-/tree/master/template_generator</a>) Update Squad uses does not detect these cases, and does not then tell the update tester there's automatic openQA regression testing.</p>
<p>For example, webkit is used in evolution - if there have been maintenance updates of webkit, was the manual testing more complex than what the evolution automatic test case would be, or not? If the evolution test tests webkit to an enough extent, webkit could be marked as automatically regression tested, and manual testing would no longer be required.</p>
<a name="Acceptance-Criteria"></a>
<h2 >Acceptance Criteria<a href="#Acceptance-Criteria" class="wiki-anchor">¶</a></h2>
<p>AC1: Identify what packages we are regression testing indirectly, and via which tests, check if it's "sufficient" for regression testing of that package: sufficient means at least similar level as in manual testing done in case of updates, see <a href="http://qam.suse.de/" class="external">http://qam.suse.de/</a> database for how the packages was teded manually.<br>
AC2: Mark sufficiently indirectly regression tested packages as automatically tested. This can be done, in the current way, with an entry at PES wiki page (<a href="https://pes.suse.de/QA_Maintenance/Automation_Progress_Tracking_in_QAM/" class="external">https://pes.suse.de/QA_Maintenance/Automation_Progress_Tracking_in_QAM/</a>) - the template generator parses that page to know if there has been automatic regression testing. Add a separate header for indirectly tested packages.</p>
openQA Tests - action #88231 (Feedback): [qe-core][qem][kiwi][regression] Extend kiwi testing to ...https://progress.opensuse.org/issues/882312021-01-26T12:35:53Ztjyrinki_susetjyrinki+redmine@suse.de
<p>There was an lvm update that broke kiwi build:</p>
<p><a href="https://github.com/OSInside/kiwi/issues/1665" class="external">https://github.com/OSInside/kiwi/issues/1665</a></p>
<p>Our basic kiwi testing currently tests simply that kiwi is able to generate an image on x86_64, not enough to catch this kind of problem.</p>
<p>The ticket is not workable at the moment, to be needed to be added to this ticket:</p>
<ul>
<li>link to LVM update with the change that broke kiwi building</li>
<li>description of how to use kiwi in the way that would reveal the brokenness</li>
</ul>
<p>For more information contact mwilck, hmzhao. hmzhao also mentioned we would need specifically someone from kiwi team.</p>
openQA Tests - action #87713 (Rejected): [qe-yast][qe-core][qem][QU] test fails in scc_registrati...https://progress.opensuse.org/issues/877132021-01-13T15:57:47Ztjyrinki_susetjyrinki+redmine@suse.de
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>openQA test in scenario sle-15-SP2-Full-QR-ppc64le-multipath@ppc64le-no-tmpfs fails in<br>
<a href="https://openqa.suse.de/tests/5281463/modules/scc_registration/steps/5" class="external">scc_registration</a></p>
<a name="Test-suite-description"></a>
<h2 >Test suite description<a href="#Test-suite-description" class="wiki-anchor">¶</a></h2>
<p>Maintainer: okurz</p>
<p>Test installation on machine with virtual multipath hardware. Only tests succesful detection of multipath and installation. No functional testing of multipath itself.</p>
<a name="Reproducible"></a>
<h2 >Reproducible<a href="#Reproducible" class="wiki-anchor">¶</a></h2>
<p>Fails since (at least) Build <a href="https://openqa.suse.de/tests/5182046" class="external">376.10</a></p>
<a name="Expected-result"></a>
<h2 >Expected result<a href="#Expected-result" class="wiki-anchor">¶</a></h2>
<p>See Online-QR's result at: <a href="https://openqa.suse.de/tests/5259394" class="external">https://openqa.suse.de/tests/5259394</a><br>
This is the first time the test is tried to run on Full images.<br>
Also SP3 Yast tests do not test multipath on Full: <a href="https://openqa.suse.de/tests/overview?distri=sle&version=15-SP3&build=124.5&groupid=129" class="external">https://openqa.suse.de/tests/overview?distri=sle&version=15-SP3&build=124.5&groupid=129</a></p>
<p>This task might be for QE Yast to enable multipath testing on Full images at some point.</p>
<a name="Further-details"></a>
<h2 >Further details<a href="#Further-details" class="wiki-anchor">¶</a></h2>
<p>Always latest result in this scenario: <a href="https://openqa.suse.de/tests/latest?arch=ppc64le&distri=sle&flavor=Full-QR&machine=ppc64le-no-tmpfs&test=multipath&version=15-SP2" class="external">latest</a></p>
openQA Tests - action #87710 (Resolved): [qe-yast][qe-core][qem][QU] test fails in scc_registrati...https://progress.opensuse.org/issues/877102021-01-13T15:57:00Ztjyrinki_susetjyrinki+redmine@suse.de
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>openQA test in scenario sle-15-SP2-Full-QR-x86_64-multipath@64bit-no-tmpfs fails in<br>
<a href="https://openqa.suse.de/tests/5281462/modules/scc_registration/steps/5" class="external">scc_registration</a></p>
<a name="Test-suite-description"></a>
<h2 >Test suite description<a href="#Test-suite-description" class="wiki-anchor">¶</a></h2>
<p>Maintainer: okurz</p>
<p>Test installation on machine with virtual multipath hardware. Only tests succesful detection of multipath and installation. No functional testing of multipath itself.</p>
<a name="Reproducible"></a>
<h2 >Reproducible<a href="#Reproducible" class="wiki-anchor">¶</a></h2>
<p>Fails since (at least) Build <a href="https://openqa.suse.de/tests/5182070" class="external">376.10</a></p>
<a name="Expected-result"></a>
<h2 >Expected result<a href="#Expected-result" class="wiki-anchor">¶</a></h2>
<p>See Online-QR's result at: <a href="https://openqa.suse.de/tests/5259366" class="external">https://openqa.suse.de/tests/5259366</a><br>
This is the first time the test is tried to run on Full images.<br>
Also SP3 Yast tests do not test multipath on Full: <a href="https://openqa.suse.de/tests/overview?distri=sle&version=15-SP3&build=124.5&groupid=129" class="external">https://openqa.suse.de/tests/overview?distri=sle&version=15-SP3&build=124.5&groupid=129</a></p>
<p>This task might be for QE Yast to enable multipath testing on Full images at some point.</p>
<a name="Further-details"></a>
<h2 >Further details<a href="#Further-details" class="wiki-anchor">¶</a></h2>
<p>Always latest result in this scenario: <a href="https://openqa.suse.de/tests/latest?arch=x86_64&distri=sle&flavor=Full-QR&machine=64bit-no-tmpfs&test=multipath&version=15-SP2" class="external">latest</a></p>
openQA Tests - action #87709 (Resolved): [qe-core][qem][QU] test fails in bootloaderhttps://progress.opensuse.org/issues/877092021-01-13T15:55:43Ztjyrinki_susetjyrinki+redmine@suse.de
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>openQA test in scenario sle-15-SP2-Full-QR-x86_64-gnome_smb@64bit fails in<br>
<a href="https://openqa.suse.de/tests/5281460/modules/bootloader/steps/5" class="external">bootloader</a></p>
<a name="Test-suite-description"></a>
<h2 >Test suite description<a href="#Test-suite-description" class="wiki-anchor">¶</a></h2>
<p>Maintainer: okurz, mgriessmeier</p>
<p>Install gnome using smb and MIRROR_SMB variable available in Server-MINI-ISO flavor (only)</p>
<a name="Reproducible"></a>
<h2 >Reproducible<a href="#Reproducible" class="wiki-anchor">¶</a></h2>
<p>Fails since (at least) Build <a href="https://openqa.suse.de/tests/5182082" class="external">376.10</a></p>
<a name="Expected-result"></a>
<h2 >Expected result<a href="#Expected-result" class="wiki-anchor">¶</a></h2>
<p>See Online-QR's result at: <a href="https://openqa.suse.de/tests/5259405" class="external">https://openqa.suse.de/tests/5259405</a><br>
This is the first time the test is tried to run on Full images.</p>
<a name="Further-details"></a>
<h2 >Further details<a href="#Further-details" class="wiki-anchor">¶</a></h2>
<p>Always latest result in this scenario: <a href="https://openqa.suse.de/tests/latest?arch=x86_64&distri=sle&flavor=Full-QR&machine=64bit&test=gnome_smb&version=15-SP2" class="external">latest</a></p>
openQA Tests - action #87707 (Resolved): [qe-core][qem][QU] test fails in bootloaderhttps://progress.opensuse.org/issues/877072021-01-13T15:54:41Ztjyrinki_susetjyrinki+redmine@suse.de
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>openQA test in scenario sle-15-SP2-Full-QR-x86_64-gnome_http@64bit fails in<br>
<a href="https://openqa.suse.de/tests/5281459/modules/bootloader/steps/5" class="external">bootloader</a></p>
<a name="Test-suite-description"></a>
<h2 >Test suite description<a href="#Test-suite-description" class="wiki-anchor">¶</a></h2>
<p>Maintainer: okurz, mgriessmeier</p>
<p>Install default system using the gnome desktop using a remote repository over http.</p>
<a name="Reproducible"></a>
<h2 >Reproducible<a href="#Reproducible" class="wiki-anchor">¶</a></h2>
<p>Fails since (at least) Build <a href="https://openqa.suse.de/tests/5182081" class="external">376.10</a></p>
<a name="Expected-result"></a>
<h2 >Expected result<a href="#Expected-result" class="wiki-anchor">¶</a></h2>
<p>See Online-QR's result at: <a href="https://openqa.suse.de/tests/5259404" class="external">https://openqa.suse.de/tests/5259404</a><br>
This is the first time the test is tried to run on Full images.</p>
<a name="Further-details"></a>
<h2 >Further details<a href="#Further-details" class="wiki-anchor">¶</a></h2>
<p>Always latest result in this scenario: <a href="https://openqa.suse.de/tests/latest?arch=x86_64&distri=sle&flavor=Full-QR&machine=64bit&test=gnome_http&version=15-SP2" class="external">latest</a></p>
openQA Tests - action #87704 (Resolved): [qe-core][qem][QU] test fails in boot_encrypt on s390x (...https://progress.opensuse.org/issues/877042021-01-13T15:52:23Ztjyrinki_susetjyrinki+redmine@suse.de
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>openQA test in scenario sle-15-SP2-Online-QR-s390x-lvm-encrypt-separate-boot@s390x-kvm-sle12 fails in<br>
<a href="https://openqa.suse.de/tests/5259382/modules/boot_encrypt/steps/10" class="external">boot_encrypt</a></p>
<a name="Test-suite-description"></a>
<h2 >Test suite description<a href="#Test-suite-description" class="wiki-anchor">¶</a></h2>
<p>Same as lvm-full-encrypt, but with separate boot not encrypted partition, only installation to not repeat everything again with small risk.<br>
Maintainer: riafarov</p>
<a name="Reproducible"></a>
<h2 >Reproducible<a href="#Reproducible" class="wiki-anchor">¶</a></h2>
<p>Fails since (at least) Build <a href="https://openqa.suse.de/tests/5256009" class="external">377.1</a></p>
<a name="Expected-result"></a>
<h2 >Expected result<a href="#Expected-result" class="wiki-anchor">¶</a></h2>
<p>Last good: <a href="https://openqa.suse.de/tests/5239855" class="external">376.16</a> (or more recent)</p>
<p>--<br>
Note: also this passing is already using the separated QAM YAML schedule schedule/qam/QR/lvm_encrypt_separate_boot.yaml<br>
--</p>
<a name="Further-details"></a>
<h2 >Further details<a href="#Further-details" class="wiki-anchor">¶</a></h2>
<p>Always latest result in this scenario: <a href="https://openqa.suse.de/tests/latest?arch=s390x&distri=sle&flavor=Online-QR&machine=s390x-kvm-sle12&test=lvm-encrypt-separate-boot&version=15-SP2" class="external">latest</a></p>
openQA Tests - action #80452 (Resolved): [qe-core][qem] Problems with aarch64 RAID 15SP1/SP2 QU t...https://progress.opensuse.org/issues/804522020-11-26T12:13:47Ztjyrinki_susetjyrinki+redmine@suse.de
<p>Passed for 15SP1 16 days ago:<br>
<a href="https://openqa.suse.de/tests/overview?distri=sle&version=15-SP1&build=45.41&groupid=249" class="external">https://openqa.suse.de/tests/overview?distri=sle&version=15-SP1&build=45.41&groupid=249</a></p>
<p>Failed in a later build:<br>
<a href="https://openqa.suse.de/tests/overview?distri=sle&version=15-SP1&build=47.3&groupid=249" class="external">https://openqa.suse.de/tests/overview?distri=sle&version=15-SP1&build=47.3&groupid=249</a></p>
<p>(same build, aarch64 specific for playground)<br>
<a href="https://openqa.suse.de/tests/overview?distri=sle&version=15-SP1&build=47.3rerun1&groupid=249" class="external">https://openqa.suse.de/tests/overview?distri=sle&version=15-SP1&build=47.3rerun1&groupid=249</a></p>
<p>But associated with change last Thursday like:<br>
<a href="https://github.com/os-autoinst/os-autoinst-distri-opensuse/commit/16631d3018a1cddae2e5825670d1a066e10fb015" class="external">https://github.com/os-autoinst/os-autoinst-distri-opensuse/commit/16631d3018a1cddae2e5825670d1a066e10fb015</a><br>
that led to setup_libyui error:<br>
<a href="https://openqa.suse.de/tests/5060842#step/setup_libyui/1" class="external">https://openqa.suse.de/tests/5060842#step/setup_libyui/1</a><br>
and with manual schedule omitting the setup_libyui to raid_gpt error:<br>
<a href="https://openqa.suse.de/tests/5063512#step/raid_gpt/1" class="external">https://openqa.suse.de/tests/5063512#step/raid_gpt/1</a></p>
<p>Then Rodion mentioned YAML schedule should not be used and moved back to non-YAML (even though the passing tests 16 days earlier were using the YAML schedule):<br>
<a href="https://gitlab.suse.de/qa-maintenance/qam-openqa-yml/-/commit/0a9c964daa754911478d01a29ffb4d9148c79fdf" class="external">https://gitlab.suse.de/qa-maintenance/qam-openqa-yml/-/commit/0a9c964daa754911478d01a29ffb4d9148c79fdf</a></p>
<p>But that lead to different errors, which in turn were partially fixed by:<br>
<a href="https://github.com/os-autoinst/os-autoinst-distri-opensuse/commit/96c4dca21c8bd7d9b11cffd58c2f9985ed319f53" class="external">https://github.com/os-autoinst/os-autoinst-distri-opensuse/commit/96c4dca21c8bd7d9b11cffd58c2f9985ed319f53</a></p>
<p>Before now George started looking at, at least RAID 0, 5 and 10 were proven to have passed at least once for 15SP1 Build 47.3, while 1 and 6 remained problemtic.</p>
<p>For 15SP2, everything passed 5 days ago at <a href="https://openqa.suse.de/tests/overview?distri=sle&version=15-SP2&build=375.8&groupid=321" class="external">https://openqa.suse.de/tests/overview?distri=sle&version=15-SP2&build=375.8&groupid=321</a> but similarly failures with latest build (rerun can get it further though): <a href="https://openqa.suse.de/tests/overview?distri=sle&version=15-SP2&build=376.1&groupid=321" class="external">https://openqa.suse.de/tests/overview?distri=sle&version=15-SP2&build=376.1&groupid=321</a></p>
<p>SP2 is still using YAML.</p>
openSUSE admin - tickets #80354 (Rejected): ns3.opensuse.org reports wrong IP to meet.opensuse.orghttps://progress.opensuse.org/issues/803542020-11-25T09:58:45Ztjyrinki_susetjyrinki+redmine@suse.de
<p>The new meet.o.o server was put into place, and it's at 195.135.221.174. However, ns3.opensuse.org disagrees and reports 173, which breaks meet.o.o for everyone (unless using direct IP).</p>
<p>Meanwhile, ns2 and ns4 are also down, possibly something to look at as well and verify if they'll eventually agree on the IP of meet.o.o.</p>
<p>-Timo </p>
qe-yam - action #80244 (Closed): [qam] setup_libyui fails on aarch64 on 15SP1https://progress.opensuse.org/issues/802442020-11-24T07:33:04Ztjyrinki_susetjyrinki+redmine@suse.de
<p>Commit b391b84490214cfd27387353a5bd18fba1d8e2fe (Adjust qemu backend scheduled for RAID0/1/5/6/10 scenarios) added setup_libyui which fails on 15SP1 on aarch64:</p>
<p><a href="https://openqa.suse.de/tests/overview?distri=sle&version=15-SP1&build=47.3&groupid=249" class="external">https://openqa.suse.de/tests/overview?distri=sle&version=15-SP1&build=47.3&groupid=249</a></p>
<p><a href="https://openqa.suse.de/tests/5060842" class="external">https://openqa.suse.de/tests/5060842</a></p>
<p>The test is part of the QU5 validation being done today.</p>
<p>Note: There is no more QUs to 15SP1 after QU5, possibly finalized today, so this ticket is more for information. Trying to run with customs schedule now.</p>
openQA Tests - coordination #64126 (New): [qe-core][epic] Identify packages that have automated i...https://progress.opensuse.org/issues/641262020-03-03T13:23:14Ztjyrinki_susetjyrinki+redmine@suse.de
<p>There's both 1) potentially indirect testing of packages that could be marked as tested automatically, and 2) database of TEST_SUITE_SUFFICIENT information from Updates Squad that could be used likewise.</p>
<p>Let's use this ticket to define guidelines for us.</p>
<a name="How"></a>
<h1 >How<a href="#How" class="wiki-anchor">¶</a></h1>
<ul>
<li>How to own our testing better (Where our needs for tooling end, and where the tooling should be provided by external teams)</li>
<li>How to decide whether it should be a package test, an openQA test, or a separate test ran within openQA</li>
<li>How to match this with our test plan</li>
</ul>
openQA Tests - action #58673 (Resolved): libqt5_qtbase fails on 15 SP2https://progress.opensuse.org/issues/586732019-10-25T06:31:45Ztjyrinki_susetjyrinki+redmine@suse.de
<p>The new test just merged seems to be working elsewhere but needles fail on 15 SP2.</p>
<p><a href="https://openqa.suse.de/tests/3527794" class="external">https://openqa.suse.de/tests/3527794</a></p>
openQA Tests - action #57584 (Resolved): [qam] test fails in evolution_setup_servers - dovecot n...https://progress.opensuse.org/issues/575842019-10-01T11:52:32Ztjyrinki_susetjyrinki+redmine@suse.de
<p>After <a href="https://progress.opensuse.org/issues/56273" class="external">https://progress.opensuse.org/issues/56273</a> got fixed, the evolution_setup_servers is now failing, a rerun done first at <a href="https://openqa.suse.de/tests/3397700" class="external">https://openqa.suse.de/tests/3397700</a> and a newer one at <a href="https://openqa.suse.de/tests/3424806" class="external">https://openqa.suse.de/tests/3424806</a></p>
<p>"No provider of 'dovecot' found." at <a href="https://openqa.suse.de/tests/3397700#step/evolution_prepare_servers/18" class="external">https://openqa.suse.de/tests/3397700#step/evolution_prepare_servers/18</a></p>
<p>qam-regression-message is currently disabled, <a href="https://progress.opensuse.org/issues/57260" class="external">https://progress.opensuse.org/issues/57260</a> is about re-enabling it once the issues are all fixed.</p>
openQA Tests - action #50867 (Resolved): [qam] nfsidmap part of the autofs tests fail on SLE12https://progress.opensuse.org/issues/508672019-04-30T07:30:08Ztjyrinki_susetjyrinki+redmine@suse.de
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<pre><code class="bash syntaxhl" data-language="bash">nfsidmap <span class="nt">-c</span>
<span class="s1">'id_resolver'</span> keyring was not found
</code></pre>
<a name="Reproducible"></a>
<h2 >Reproducible<a href="#Reproducible" class="wiki-anchor">¶</a></h2>
<ul>
<li>Build <a href="https://openqa.suse.de/tests/2844872#step/autofs_client/39" class="external">20190430</a>
In scenario sle-12-SP3-Server-DVD-Updates-x86_64-1-mau-autofs-server@64bit</li>
</ul>
<a name="Expected-results"></a>
<h2 >Expected results<a href="#Expected-results" class="wiki-anchor">¶</a></h2>
<p>I cannot find any expected result in OSD, so I consider this a new test.</p>
<a name="Further-information"></a>
<h2 >Further information<a href="#Further-information" class="wiki-anchor">¶</a></h2>
<p>Latest job in this scenario: <a href="https://openqa.suse.de/tests/latest?test=mau-autofs-server&flavor=Server-DVD-Updates&arch=x86_64&distri=sle&version=12-SP3&machine=64bit" class="external">https://openqa.suse.de/tests/latest?test=mau-autofs-server&flavor=Server-DVD-Updates&arch=x86_64&distri=sle&version=12-SP3&machine=64bit</a></p>