openSUSE Project Management Tool: Issueshttps://progress.opensuse.org/https://progress.opensuse.org/themes/openSUSE/favicon/favicon.ico?15829177842024-03-08T14:07:43ZopenSUSE Project Management Tool
Redmine openQA Tests - action #156928 (Resolved): [qe-core] test did not fail in sudo when it should havehttps://progress.opensuse.org/issues/1569282024-03-08T14:07:43Zmgrifalconi
<a name="Context"></a>
<h2 >Context<a href="#Context" class="wiki-anchor">¶</a></h2>
<p>See: <a href="https://suse.slack.com/archives/C02CLB8TZP1/p1709889781138779" class="external">https://suse.slack.com/archives/C02CLB8TZP1/p1709889781138779</a><br>
And: <a href="https://bugzilla.suse.com/show_bug.cgi?id=1221134" class="external">https://bugzilla.suse.com/show_bug.cgi?id=1221134</a></p>
<p>Using the broken sudo update on 15-SP5: <a href="https://download.suse.de/ibs/SUSE:/Maintenance:/32785/SUSE_Updates_SLE-Module-Basesystem_15-SP5_x86_64/x86_64/" class="external">https://download.suse.de/ibs/SUSE:/Maintenance:/32785/SUSE_Updates_SLE-Module-Basesystem_15-SP5_x86_64/x86_64/</a><br>
(I downloaded sudo-1.9.12p1-150500.7.7.1.x86_64.rpm in case it gets removed: <a href="https://gitlab.suse.de/mgrifalconi/broken-sudo-mirror/-/blob/main/sudo-1.9.12p1-150500.7.7.1.x86_64.rpm" class="external">https://gitlab.suse.de/mgrifalconi/broken-sudo-mirror/-/blob/main/sudo-1.9.12p1-150500.7.7.1.x86_64.rpm</a>)</p>
<p>Clone a test in openQA or try it locally. Using this exact command you get the image with the broken sudo file already. Will not remain for long I guess.</p>
<pre><code>/usr/share/openqa/script/clone_job.pl \
--from https://openqa.suse.de \
--host https://openqa.suse.de \
-v 13722839 \
--skip-chained-deps \
--ignore-missing-assets \
--skip-download \
--skip-checks \
_SKIP_POST_FAIL_HOOKS=1 \
SCHEDULE=tests/installation/bootloader_start,tests/boot/boot_to_desktop,tests/console/prepare_test_data,tests/console/consoletest_setup,tests/console/sudo \
TEST=developer-13722839-2 \
BUILD=developer-13722839-2 \
_GROUP="developer-13722839-2"
</code></pre>
<p>Use developer tab in openQA and stop execution before sudo test.</p>
<p>Login to VNC</p>
<pre><code>visudo -f /etc/sudoers.d/bernhard
bernhard ALL = NOPASSWD: /usr/bin/ls
</code></pre>
<p>save and exit</p>
<pre><code>su - bernhard
sudo -k
sudo ls
</code></pre>
<p>will ask for password when it should not.</p>
<p>Intead:</p>
<pre><code>bernhard ALL = (root) NOPASSWD: /usr/bin/ls
</code></pre>
<p>Will work as expected.</p>
<p>We should improve the current sudo test: <a href="https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/tests/console/sudo.pm" class="external">https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/tests/console/sudo.pm</a><br>
to catch this issue.</p>
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>openQA test in scenario sle-15-SP5-Server-DVD-Updates-x86_64-mau-extratests2@64bit <del>fails</del> in<br>
<a href="https://openqa.suse.de/tests/13722839/modules/sudo/steps/1" class="external">sudo</a></p>
<a name="Test-suite-description"></a>
<h2 >Test suite description<a href="#Test-suite-description" class="wiki-anchor">¶</a></h2>
<p>Testsuite maintained at <a href="https://gitlab.suse.de/qa-maintenance/qam-openqa-yml" class="external">https://gitlab.suse.de/qa-maintenance/qam-openqa-yml</a>. Run console tests against aggregated test repo</p>
<a name="Reproducible"></a>
<h2 >Reproducible<a href="#Reproducible" class="wiki-anchor">¶</a></h2>
<p><del>Fails</del> since (at least) Build <a href="https://openqa.suse.de/tests/13721240" class="external">20240305-1</a></p>
<a name="Expected-result"></a>
<h2 >Expected result<a href="#Expected-result" class="wiki-anchor">¶</a></h2>
<p>Last good: <a href="https://openqa.suse.de/tests/13710699" class="external">20240304-2</a> (or more recent)</p>
<a name="Further-details"></a>
<h2 >Further details<a href="#Further-details" class="wiki-anchor">¶</a></h2>
<p>Always latest result in this scenario: <a href="https://openqa.suse.de/tests/latest?arch=x86_64&distri=sle&flavor=Server-DVD-Updates&machine=64bit&test=mau-extratests2&version=15-SP5" class="external">latest</a></p>
openQA Tests - action #134591 (Resolved): [qe-core] Unschedule gpg for all arch and all SLE versi...https://progress.opensuse.org/issues/1345912023-08-24T09:48:39Zmgrifalconi
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>It's owned by qe-security and they are already running it, let's get rid of it :)</p>
<p>openQA test in scenario sle-15-SP5-Server-DVD-Updates-s390x-mau-extratests2@s390x-kvm-sle12 fails in<br>
<a href="https://openqa.suse.de/tests/11908566/modules/gpg/steps/18" class="external">gpg</a></p>
<a name="Test-suite-description"></a>
<h2 >Test suite description<a href="#Test-suite-description" class="wiki-anchor">¶</a></h2>
<p>Testsuite maintained at <a href="https://gitlab.suse.de/qa-maintenance/qam-openqa-yml" class="external">https://gitlab.suse.de/qa-maintenance/qam-openqa-yml</a>. Run console tests against aggregated test repo</p>
<a name="Reproducible"></a>
<h2 >Reproducible<a href="#Reproducible" class="wiki-anchor">¶</a></h2>
<p>Fails since (at least) Build <a href="https://openqa.suse.de/tests/11906821" class="external">20230823-1</a></p>
<a name="Expected-result"></a>
<h2 >Expected result<a href="#Expected-result" class="wiki-anchor">¶</a></h2>
<p>Last good: <a href="https://openqa.suse.de/tests/11901405" class="external">20230822-1</a> (or more recent)</p>
<a name="Further-details"></a>
<h2 >Further details<a href="#Further-details" class="wiki-anchor">¶</a></h2>
<p>Always latest result in this scenario: <a href="https://openqa.suse.de/tests/latest?arch=s390x&distri=sle&flavor=Server-DVD-Updates&machine=s390x-kvm-sle12&test=mau-extratests2&version=15-SP5" class="external">latest</a></p>
openQA Tests - action #132449 (Resolved): [qe-core] remove fips test from qe-core aggregate job g...https://progress.opensuse.org/issues/1324492023-07-07T11:45:38Zmgrifalconi
<p>According to <a href="https://suse.slack.com/archives/C02D16TCP99/p1688721103274659" class="external">https://suse.slack.com/archives/C02D16TCP99/p1688721103274659</a> fips tests are maintained by security squad.</p>
<p>Known fips test in qe-core are:</p>
<ul>
<li><a href="https://openqa.suse.de/tests/11529860" class="external">https://openqa.suse.de/tests/11529860</a> qam-regression-firefox-SLED_fips_kernelmode</li>
<li><a href="https://openqa.suse.de/tests/11529864" class="external">https://openqa.suse.de/tests/11529864</a> qam-regression-other_fips_kernelmode</li>
<li><a href="https://openqa.suse.de/tests/11530950" class="external">https://openqa.suse.de/tests/11530950</a> mau-extratests_fips_kernelmode</li>
</ul>
<p>AC:</p>
<ul>
<li>Check for other places where we run fips test</li>
<li>Sync with Security Squad to check if our tests are duplicate to theirs so we can just remove them or they should take over these as well to their job groups</li>
<li>Remove fips test from Core Maintenance job group</li>
</ul>
openQA Tests - action #132269 (Resolved): [qe-core] test fails in Leap patch_and_reboothttps://progress.opensuse.org/issues/1322692023-07-04T07:01:42Zmgrifalconi
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>After reboot it enters GRUB recovery with errors like:</p>
<ul>
<li>no key data</li>
<li>no such cryptodisk found, perhaps a needed disk or cryptodisk module is not found</li>
</ul>
<p>openQA test in scenario opensuse-15.4-Leap-DVD-Incidents-x86_64-cryptlvm@64bit fails in<br>
<a href="https://openqa.suse.de/tests/11504711/modules/patch_and_reboot/steps/65" class="external">patch_and_reboot</a></p>
<a name="Test-suite-description"></a>
<h2 >Test suite description<a href="#Test-suite-description" class="wiki-anchor">¶</a></h2>
<p>Testsuite maintained at <a href="https://gitlab.suse.de/qa-maintenance/qam-openqa-yml" class="external">https://gitlab.suse.de/qa-maintenance/qam-openqa-yml</a>. Maintainers: QE Yast</p>
<p>Conduct installation with encrypted LVM selected during installation. Generated disk image used in downstream jobs.</p>
<p>(crypt-)LVM installations can take longer, especially on non-x86_64 architectures.</p>
<p>YAML_SCHEDULE=schedule/yaml/cryptlvm/cryptlvm_sle.yaml</p>
<a name="Reproducible"></a>
<h2 >Reproducible<a href="#Reproducible" class="wiki-anchor">¶</a></h2>
<p>Fails since (at least) Build <a href="https://openqa.suse.de/tests/11499130" class="external">:28679:jing-trang</a></p>
<a name="Expected-result"></a>
<h2 >Expected result<a href="#Expected-result" class="wiki-anchor">¶</a></h2>
<p>Last good: <a href="https://openqa.suse.de/tests/11497998" class="external">:29695:mozilla-nss</a> (or more recent)</p>
<a name="Further-details"></a>
<h2 >Further details<a href="#Further-details" class="wiki-anchor">¶</a></h2>
<p>Always latest result in this scenario: <a href="https://openqa.suse.de/tests/latest?arch=x86_64&distri=opensuse&flavor=Leap-DVD-Incidents&machine=64bit&test=cryptlvm&version=15.4" class="external">latest</a></p>
openQA Tests - action #131213 (Resolved): [qe-core] test fails in update_minimalhttps://progress.opensuse.org/issues/1312132023-06-21T12:18:02Zmgrifalconi
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<pre><code> [zypp-core] Exception.cc(log):186 ZYppFactory.cc(getZYpp):415 THROW: System management is locked by the application with pid 676 (/usr/bin/zypper).
</code></pre>
<p>openQA test in scenario sle-15-SP4-Server-DVD-Incidents-Minimal-x86_64-qam-minimal-lvm@64bit fails in<br>
<a href="https://openqa.suse.de/tests/11360320/modules/update_minimal/steps/49" class="external">update_minimal</a></p>
<a name="Test-suite-description"></a>
<h2 >Test suite description<a href="#Test-suite-description" class="wiki-anchor">¶</a></h2>
<p>Testsuite maintained at <a href="https://gitlab.suse.de/qa-maintenance/qam-openqa-yml" class="external">https://gitlab.suse.de/qa-maintenance/qam-openqa-yml</a>. disk image for grub2_lvm<br>
minimal = base pattern, minimal (enhanced base) pattern are additional convenience paclkages</p>
<a name="Reproducible"></a>
<h2 >Reproducible<a href="#Reproducible" class="wiki-anchor">¶</a></h2>
<p>Fails since (at least) Build <a href="https://openqa.suse.de/tests/11360320" class="external">:29299:dtb-armv7l</a> (current job)</p>
<a name="Expected-result"></a>
<h2 >Expected result<a href="#Expected-result" class="wiki-anchor">¶</a></h2>
<p>Last good: <a href="https://openqa.suse.de/tests/11344169" class="external">:28554:zypper</a> (or more recent)</p>
<a name="Further-details"></a>
<h2 >Further details<a href="#Further-details" class="wiki-anchor">¶</a></h2>
<p>Always latest result in this scenario: <a href="https://openqa.suse.de/tests/latest?arch=x86_64&distri=sle&flavor=Server-DVD-Incidents-Minimal&machine=64bit&test=qam-minimal-lvm&version=15-SP4" class="external">latest</a></p>
openQA Tests - action #128933 (Resolved): test fails in acceptance_sched_stresshttps://progress.opensuse.org/issues/1289332023-05-08T14:34:27Zmgrifalconi
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>One more QA-RUN test that we should have removed long time ago. <br>
Other than finding the issue, we should convert this test to a 'normal' one instead of using qe-run</p>
<p>openQA test in scenario sle-12-SP5-Server-DVD-Updates-x86_64-mau-qa_acceptance_sched_stress@64bit fails in<br>
<a href="https://openqa.suse.de/tests/11063719/modules/acceptance_sched_stress/steps/97" class="external">acceptance_sched_stress</a></p>
<a name="Test-suite-description"></a>
<h2 >Test suite description<a href="#Test-suite-description" class="wiki-anchor">¶</a></h2>
<p>Testsuite maintained at <a href="https://gitlab.suse.de/qa-maintenance/qam-openqa-yml" class="external">https://gitlab.suse.de/qa-maintenance/qam-openqa-yml</a>.</p>
<a name="Reproducible"></a>
<h2 >Reproducible<a href="#Reproducible" class="wiki-anchor">¶</a></h2>
<p>Fails since (at least) Build <a href="https://openqa.suse.de/tests/11061611" class="external">20230507-1</a></p>
<a name="Expected-result"></a>
<h2 >Expected result<a href="#Expected-result" class="wiki-anchor">¶</a></h2>
<p>Last good: <a href="https://openqa.suse.de/tests/11057394" class="external">20230506-1</a> (or more recent)</p>
<a name="Further-details"></a>
<h2 >Further details<a href="#Further-details" class="wiki-anchor">¶</a></h2>
<p>Always latest result in this scenario: <a href="https://openqa.suse.de/tests/latest?arch=x86_64&distri=sle&flavor=Server-DVD-Updates&machine=64bit&test=mau-qa_acceptance_sched_stress&version=12-SP5" class="external">latest</a></p>
openQA Tests - action #124292 (Resolved): [qe-core] Schedule missing Desktop tests for 15-SP4https://progress.opensuse.org/issues/1242922023-02-10T10:36:22Zmgrifalconi
<p>According to <a href="https://suse.slack.com/archives/C02CANHLANP/p1675782032119079" class="external">https://suse.slack.com/archives/C02CANHLANP/p1675782032119079</a></p>
<p>We are missing 15-SP4 Desktop tests and they should be scheduled in the qe-core jobgroup.<br>
We could use <a href="https://gitlab.suse.de/qa-maintenance/metadata/-/commit/be42df50b478c40a024ccdb76844c036a92bbf34" class="external">https://gitlab.suse.de/qa-maintenance/metadata/-/commit/be42df50b478c40a024ccdb76844c036a92bbf34</a> as blueprint and change SP3 to SP4</p>
QA - action #123286 (Resolved): Bot and dashboard reference to wrong data and block update approv...https://progress.opensuse.org/issues/1232862023-01-18T09:19:08Zmgrifalconi
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>Hello, there is some inconsistency with the dashboard data about 27130:dragonbox</p>
<p>Link of the red SLE 15 SP4 box in blocked page points to <a href="https://openqa.suse.de/tests/overview?build=%3A27130%3Afixmath&distri=sle&groupid=439" class="external">https://openqa.suse.de/tests/overview?build=%3A27130%3Afixmath&distri=sle&groupid=439</a><br>
with no failures</p>
<p>Link inside the update request page <a href="http://dashboard.qam.suse.de/incident/27130" class="external">http://dashboard.qam.suse.de/incident/27130</a> points to a different incidents results <a href="https://openqa.suse.de/tests/overview?build=%3A27130%3Alibmwaw" class="external">https://openqa.suse.de/tests/overview?build=%3A27130%3Alibmwaw</a> with this time a failure</p>
<p>Bot approval job log:</p>
<pre><code> 2023-01-17 08:05:34 INFO Found failed, not-ignored job 10166069 for incident 27130
</code></pre>
<p>Interestingly enough, I restarted the month-old job and now even that is green.<br>
But still, the bot does not like it and keeps the 'box' red.<br>
<a href="https://openqa.suse.de/tests/10166069" class="external">https://openqa.suse.de/tests/10166069</a><br>
even if its clone is green: <a href="https://openqa.suse.de/tests/10331221" class="external">https://openqa.suse.de/tests/10331221</a></p>
<a name="Problem"></a>
<h2 >Problem<a href="#Problem" class="wiki-anchor">¶</a></h2>
<p>The problem here seems to be that the incident 27130 was modified multiple times and references multiple package as visible in <a href="https://smelt.suse.de/incident/27130/" class="external">https://smelt.suse.de/incident/27130/</a></p>
<a name="Acceptance-criteria"></a>
<h2 >Acceptance criteria<a href="#Acceptance-criteria" class="wiki-anchor">¶</a></h2>
<ul>
<li><strong>AC1:</strong> The dashboard page and all links to openQA tests from dashboard reference the same consistent package(s) or no package at all, i.e. no "dragonbox" in dashboard but then pointing to "libmwaw" in openQA</li>
</ul>
<a name="Suggestions"></a>
<h2 >Suggestions<a href="#Suggestions" class="wiki-anchor">¶</a></h2>
<ul>
<li>Investigate if this is maybe <em>just a display issue</em> and in that case fix it</li>
<li>Ask mgrifalconi to update the ticket according to our ticket templates to help us understand what he really expects because we are not clear about that</li>
<li>Reconsider how we test maintenance requests before a release request is created while still supporting the "shift left" endeavour</li>
<li>Check if the data in the dashboard database regarding packages is consistent with SMELT (to rule out qem-bot involvement)</li>
</ul>
openQA Tests - action #117622 (Resolved): [qe-core] Unable to clone Git repository for wheelshttps://progress.opensuse.org/issues/1176222022-10-06T09:30:52Zmgrifalconi
<p><a href="https://openqa.suse.de/tests/9668977#" class="external">https://openqa.suse.de/tests/9668977#</a></p>
<pre><code>[2022-10-06T02:01:46.761898+02:00] [info] ::: OpenQA::Isotovideo::Utils::checkout_git_repo_and_branch: Cloning git URL 'https://github.com/Zaoliang/functional_wheel'
[2022-10-06T02:01:52.973889+02:00] [debug] Cloning into 'functional_wheel'...
fatal: unable to access 'https://github.com/Zaoliang/functional_wheel/': OpenSSL SSL_connect: Connection reset by peer in connection to github.com:443
</code></pre>
<p>Regardless of the cause of the issue, I am against using personal repos for test. What happens if owner changes team or company or simply on vacation? <br>
It's great for trying things out but when using on production, it should go into company owned github accounts, and squad owned repos.</p>
<a name="Acceptance-Criteria"></a>
<h4 >Acceptance Criteria<a href="#Acceptance-Criteria" class="wiki-anchor">¶</a></h4>
<ol>
<li><a href="https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/15336" class="external">#os-autoinst/os-autoinst-distri-opensuse/15336</a> is reverted (this removes the urgency of this ticket)</li>
<li>Fork <a href="https://github.com/Zaoliang/functional_wheel" class="external">https://github.com/Zaoliang/functional_wheel</a> into <a href="https://github.com/os-autoinst/test-core-modules" class="external">https://github.com/os-autoinst/test-core-modules</a> cleanning up unnecesary files (i.e only lib/**.pm should stay)</li>
</ol>
QA - action #117619 (Resolved): Bot approved update request with failing tests size:Mhttps://progress.opensuse.org/issues/1176192022-10-06T09:26:57Zmgrifalconi
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>Incident <a href="https://smelt.suse.de/incident/25982/" class="external">https://smelt.suse.de/incident/25982/</a><br>
Request that was approved by sle-qam-openqa: <a href="https://build.suse.de/request/show/280720" class="external">https://build.suse.de/request/show/280720</a><br>
Bot job: <a href="https://gitlab.suse.de/qa-maintenance/bot-ng/-/jobs/1166058#L279" class="external">https://gitlab.suse.de/qa-maintenance/bot-ng/-/jobs/1166058#L279</a><br>
<code>INFO: SUSE:Maintenance:25982:280720</code></p>
<p>Failing test: <a href="https://openqa.suse.de/tests/9642631#settings" class="external">https://openqa.suse.de/tests/9642631#settings</a><br>
Dashboard: <a href="https://dashboard.qam.suse.de/incident/25982" class="external">https://dashboard.qam.suse.de/incident/25982</a></p>
<p>Context on slack: <a href="https://suse.slack.com/archives/C02CANHLANP/p1665043765153419" class="external">https://suse.slack.com/archives/C02CANHLANP/p1665043765153419</a></p>
<a name="Acceptance-criteria"></a>
<h2 >Acceptance criteria<a href="#Acceptance-criteria" class="wiki-anchor">¶</a></h2>
<ul>
<li><strong>AC1</strong>: We know the reason why the bot approved the request and didn't see the test failure</li>
</ul>
<a name="Suggestions"></a>
<h2 >Suggestions<a href="#Suggestions" class="wiki-anchor">¶</a></h2>
<ul>
<li>Run <code>./qem-bot/bot-ng.py -c /etc/openqabot --token [MASKED] inc-approve --dry</code> (see <a href="https://github.com/openSUSE/qem-bot/#usage" class="external">https://github.com/openSUSE/qem-bot/#usage</a> for more info)</li>
<li>Look into the dashboard logs on qam2.suse.de <code>journalctl -u dashboard.service</code></li>
<li>Note: The journal only goes back 3 days currently (Oct 3), so for the incident in question it's too late.
Consider increasing the journal size as a first step</li>
<li>Consider adding code that only runs the bot on a single incident</li>
</ul>
QA - action #113345 (Resolved): qem-bot does not ignore Development/Leap job groups as it should ...https://progress.opensuse.org/issues/1133452022-07-07T08:37:58Zmgrifalconi
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>Bot does not ignore Development/Leap job groups and could block update approvals due to broken development tests.<br>
See <a href="http://dashboard.qam.suse.de/blocked" class="external">http://dashboard.qam.suse.de/blocked</a> and look for "leap"</p>
<a name="Problem"></a>
<h2 >Problem<a href="#Problem" class="wiki-anchor">¶</a></h2>
<p>Likely regression due to <a href="https://github.com/openSUSE/qem-bot/commit/d4d33720d183ba30b63529577e2bbad700b238cd" class="external">https://github.com/openSUSE/qem-bot/commit/d4d33720d183ba30b63529577e2bbad700b238cd</a><br>
or <a href="https://github.com/openSUSE/qem-bot/commit/c869a5cb7a56cdb5c3ba33f64e086f34c64ce5b9#diff-dbb33d499407c366ab760f232[…]e02dad0dd506c87b478b8007cf496ad" class="external">https://github.com/openSUSE/qem-bot/commit/c869a5cb7a56cdb5c3ba33f64e086f34c64ce5b9#diff-dbb33d499407c366ab760f232[…]e02dad0dd506c87b478b8007cf496ad</a></p>
<a name="Suggestions"></a>
<h2 >Suggestions<a href="#Suggestions" class="wiki-anchor">¶</a></h2>
<p>Look hard at the above commits and try it out with "--dry-run" and fix it. We at least know that jobs are still ignored, e.g. <a href="https://gitlab.suse.de/qa-maintenance/bot-ng/-/jobs/1044528#L2663" class="external">https://gitlab.suse.de/qa-maintenance/bot-ng/-/jobs/1044528#L2663</a> shows "INFO: Ignoring job '9078831' in development group 'Maintenance: Leap 15.4 Incidents'" so there is at least <em>some</em> ignoring going on</p>
openQA Tests - action #113306 (Resolved): [qe-core] test fails in update_installhttps://progress.opensuse.org/issues/1133062022-07-06T09:20:19Zmgrifalconi
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>openQA test in scenario sle-15-SP4-Server-DVD-Incidents-Install-ppc64le-qam-incidentinstall@ppc64le fails in<br>
<a href="https://openqa.suse.de/tests/9077467/modules/update_install/steps/67" class="external">update_install</a></p>
<a name="Test-suite-description"></a>
<h2 >Test suite description<a href="#Test-suite-description" class="wiki-anchor">¶</a></h2>
<p>Testsuite maintained at <a href="https://gitlab.suse.de/qa-maintenance/qam-openqa-yml" class="external">https://gitlab.suse.de/qa-maintenance/qam-openqa-yml</a>. Incident Installation TEST<br>
MAX_JOB_TIME=9000 due to long texlive update</p>
<a name="Reproducible"></a>
<h2 >Reproducible<a href="#Reproducible" class="wiki-anchor">¶</a></h2>
<p>Fails since (at least) Build <a href="https://openqa.suse.de/tests/9077467" class="external">:23822:ldb</a> (current job)</p>
<a name="Expected-result"></a>
<h2 >Expected result<a href="#Expected-result" class="wiki-anchor">¶</a></h2>
<p>Last good: <a href="https://openqa.suse.de/tests/9077377" class="external">:24915:PackageKit</a> (or more recent)</p>
<a name="Further-details"></a>
<h2 >Further details<a href="#Further-details" class="wiki-anchor">¶</a></h2>
<p>Always latest result in this scenario: <a href="https://openqa.suse.de/tests/latest?arch=ppc64le&distri=sle&flavor=Server-DVD-Incidents-Install&machine=ppc64le&test=qam-incidentinstall&version=15-SP4" class="external">latest</a></p>
openQA Tests - action #112277 (Resolved): [qe-core] test fails in update_install - rust test shou...https://progress.opensuse.org/issues/1122772022-06-10T12:32:40Zmgrifalconi
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>According to Marcus Meissner investigation, the test installs "cargo" and "rls" even though it should not. More info at <a href="https://suse.slack.com/archives/C02D16TCP99/p1654854438113049" class="external">https://suse.slack.com/archives/C02D16TCP99/p1654854438113049</a></p>
<p>openQA test in scenario sle-15-SP3-Server-DVD-Incidents-Install-x86_64-qam-incidentinstall@64bit fails in<br>
<a href="https://openqa.suse.de/tests/8913409/modules/update_install/steps/57" class="external">update_install</a></p>
<p>This is currently blocking the rust update approval.</p>
<a name="Test-suite-description"></a>
<h2 >Test suite description<a href="#Test-suite-description" class="wiki-anchor">¶</a></h2>
<p>Testsuite maintained at <a href="https://gitlab.suse.de/qa-maintenance/qam-openqa-yml" class="external">https://gitlab.suse.de/qa-maintenance/qam-openqa-yml</a>. Incident Installation TEST<br>
MAX_JOB_TIME=9000 due to long texlive update</p>
<a name="Reproducible"></a>
<h2 >Reproducible<a href="#Reproducible" class="wiki-anchor">¶</a></h2>
<p>Fails since (at least) Build <a href="https://openqa.suse.de/tests/8913409" class="external">:24558:rust</a> (current job)</p>
<a name="Expected-result"></a>
<h2 >Expected result<a href="#Expected-result" class="wiki-anchor">¶</a></h2>
<p>Last good: <a href="https://openqa.suse.de/tests/8913408" class="external">:24147:netty-tcnative</a> (or more recent)</p>
<a name="Further-details"></a>
<h2 >Further details<a href="#Further-details" class="wiki-anchor">¶</a></h2>
<p>Always latest result in this scenario: <a href="https://openqa.suse.de/tests/latest?arch=x86_64&distri=sle&flavor=Server-DVD-Incidents-Install&machine=64bit&test=qam-incidentinstall&version=15-SP3" class="external">latest</a></p>
<a name="Suggestions"></a>
<h2 >Suggestions<a href="#Suggestions" class="wiki-anchor">¶</a></h2>
<ol>
<li>Package rls is added as a conflict with rust, so that it isn't installed: <a href="https://openqa.suse.de/tests/8913409#step/update_install/35" class="external">https://openqa.suse.de/tests/8913409#step/update_install/35</a></li>
</ol>
QA - action #107671 (Resolved): No aggregate maintenance runs scheduled today on osd size:Mhttps://progress.opensuse.org/issues/1076712022-02-28T07:13:59Zmgrifalconi
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>Seems a different issue than <a class="issue tracker-4 status-3 priority-6 priority-high2 closed child" title="action: No aggregate maintenance runs scheduled today on osd - dashboard.qem.suse.de down size:S (Resolved)" href="https://progress.opensuse.org/issues/106179">#106179</a> since the dashboard is accessible this time.</p>
<p>Link to list aggregate runs of the day:</p>
<p><a href="https://openqa.suse.de/tests/overview?arch=&flavor=&machine=&test=&modules=&module_re=&groupid=366&groupid=308&groupid=232&groupid=165&groupid=280&groupid=218&groupid=108&groupid=54&groupid=405&groupid=412&groupid=411&groupid=369&groupid=352&groupid=353&groupid=357&groupid=355&groupid=354&groupid=358&groupid=370&groupid=348&groupid=349&groupid=351&groupid=356&groupid=375&groupid=376&groupid=397&groupid=414&build=20220228-1#" class="external">https://openqa.suse.de/tests/overview?arch=&flavor=&machine=&test=&modules=&module_re=&groupid=366&groupid=308&groupid=232&groupid=165&groupid=280&groupid=218&groupid=108&groupid=54&groupid=405&groupid=412&groupid=411&groupid=369&groupid=352&groupid=353&groupid=357&groupid=355&groupid=354&groupid=358&groupid=370&groupid=348&groupid=349&groupid=351&groupid=356&groupid=375&groupid=376&groupid=397&groupid=414&build=20220228-1#</a><br>
(This was showing an empty list at that point)</p>
<p>Impact: update approval blocked</p>
<a name="Suggestions"></a>
<h2 >Suggestions<a href="#Suggestions" class="wiki-anchor">¶</a></h2>
<ul>
<li>caused by downtime of <a href="http://download.suse.de" class="external">http://download.suse.de</a></li>
<li>read suggestions from <a class="issue tracker-4 status-3 priority-4 priority-default closed" title="action: openQABot pipeline failed: "ERROR:root:Something bad happended during reading MR data from SMELT/... (Resolved)" href="https://progress.opensuse.org/issues/105603">#105603</a></li>
<li>Some gitlab CI steps are failing but we allow them to fail to let other steps continue, e.g. in <a href="https://gitlab.suse.de/qa-maintenance/bot-ng/-/jobs/886067" class="external">https://gitlab.suse.de/qa-maintenance/bot-ng/-/jobs/886067</a> "sync smelt" fails but we allow it to fail so that "sync incidents" can continue but we also don't receive an alert about it and there is not sufficient retrying. We could split the steps into separate pipelines, make each step fatal and add configurable number of retries and interval between retries customized for each step in <a href="https://gitlab.suse.de/qa-maintenance/bot-ng/-/blob/master/.gitlab-ci.yml" class="external">https://gitlab.suse.de/qa-maintenance/bot-ng/-/blob/master/.gitlab-ci.yml</a>, e.g. for sync smelt long enough , retrying to cover the weekly SUSE IT maintenance window, less for other critical steps</li>
<li>For retrying we do not even need to change qem-bot, we could use just a wrapper in the gitlab CI job itself, e.g. <a href="https://github.com/okurz/leaky_bucket_error_count" class="external">https://github.com/okurz/leaky_bucket_error_count</a></li>
<li>Also look into gitlab CI options to either abort a previous pipeline if a new one is triggered or not start new ones as long as old ones are still running</li>
</ul>
QA - action #106179 (Resolved): No aggregate maintenance runs scheduled today on osd - dashboard....https://progress.opensuse.org/issues/1061792022-02-08T08:17:53Zmgrifalconi
<p>No aggregate runs scheduled today - dashboard.qem.suse.de down</p>
<p>Link to list aggregate runs of the day: <a href="https://openqa.suse.de/tests/overview?result=none&result=passed&result=softfailed&result=failed&result=incomplete&result=skipped&result=obsoleted&result=parallel_failed&result=parallel_restarted&result=user_cancelled&result=user_restarted&result=timeout_exceeded&state=scheduled&state=assigned&state=setup&state=running&state=uploading&state=done&state=cancelled&arch=&flavor=&machine=&test=&modules=&module_re=&groupid=366&groupid=308&groupid=232&groupid=165&groupid=280&groupid=218&groupid=108&groupid=54&groupid=405&groupid=412&groupid=411&groupid=369&groupid=352&groupid=353&groupid=357&groupid=355&groupid=354&groupid=358&groupid=370&groupid=348&groupid=349&groupid=351&groupid=356&groupid=375&groupid=376&groupid=397&groupid=414&build=20220208-1#" class="external">https://openqa.suse.de/tests/overview?result=none&result=passed&result=softfailed&result=failed&result=incomplete&result=skipped&result=obsoleted&result=parallel_failed&result=parallel_restarted&result=user_cancelled&result=user_restarted&result=timeout_exceeded&state=scheduled&state=assigned&state=setup&state=running&state=uploading&state=done&state=cancelled&arch=&flavor=&machine=&test=&modules=&module_re=&groupid=366&groupid=308&groupid=232&groupid=165&groupid=280&groupid=218&groupid=108&groupid=54&groupid=405&groupid=412&groupid=411&groupid=369&groupid=352&groupid=353&groupid=357&groupid=355&groupid=354&groupid=358&groupid=370&groupid=348&groupid=349&groupid=351&groupid=356&groupid=375&groupid=376&groupid=397&groupid=414&build=20220208-1#</a></p>
<p>This is blocking all update test/approval</p>
<a name="Suggestions"></a>
<h2 >Suggestions<a href="#Suggestions" class="wiki-anchor">¶</a></h2>
<p>Create an epic with feature requests based on this</p>