openSUSE Project Management Tool: Issueshttps://progress.opensuse.org/https://progress.opensuse.org/themes/openSUSE/favicon/favicon.ico?15829177842024-02-14T06:35:11ZopenSUSE Project Management Tool
Redmine openQA Tests - action #155440 (Resolved): [security][15-SP6] test fails in suseconnect due to mis...https://progress.opensuse.org/issues/1554402024-02-14T06:35:11Ztjyrinki_susetjyrinki+redmine@suse.de
<p>openQA test in scenario sle-15-SP6-Online-s390x-fips_ker_mode_tests_crypt_tool@s390x-kvm fails in<br>
<a href="https://openqa.suse.de/tests/13495465/modules/suseconnect/steps/2" class="external">suseconnect</a></p>
<p>It's missing SCC_REGCODE_LIVE setting, not sure from where to get the right one though as they are arch specific.</p>
openQA Tests - action #155137 (Workable): [security] oscap_bash_pci_dss_4 fails due to unexpect...https://progress.opensuse.org/issues/1551372024-02-08T06:09:44Ztjyrinki_susetjyrinki+redmine@suse.de
<p>aarch64 only:</p>
<p>15-SP4:<br>
<a href="https://openqa.suse.de/tests/13465079/modules/oscap_xccdf_eval#2/steps/25" class="external">https://openqa.suse.de/tests/13465079/modules/oscap_xccdf_eval#2/steps/25</a></p>
<p>15-SP5:<br>
<a href="https://openqa.suse.de/tests/13465157#step/oscap_xccdf_eval#2/25" class="external">https://openqa.suse.de/tests/13465157#step/oscap_xccdf_eval#2/25</a></p>
<p>15-SP6:<br>
<a href="https://openqa.suse.de/tests/13458991#step/oscap_xccdf_eval#2/25" class="external">https://openqa.suse.de/tests/13458991#step/oscap_xccdf_eval#2/25</a></p>
<p>x86_64 and s390x pass:<br>
SP4&SP5:<br>
<a href="https://openqa.suse.de/tests/13461333" class="external">https://openqa.suse.de/tests/13461333</a><br>
<a href="https://openqa.suse.de/tests/13461236" class="external">https://openqa.suse.de/tests/13461236</a><br>
<a href="https://openqa.suse.de/tests/13461772" class="external">https://openqa.suse.de/tests/13461772</a><br>
<a href="https://openqa.suse.de/tests/13461649" class="external">https://openqa.suse.de/tests/13461649</a><br>
SP6:<br>
<a href="https://openqa.suse.de/tests/13457751" class="external">https://openqa.suse.de/tests/13457751</a></p>
openQA Tests - action #155050 (Workable): [security] all oscap tests fail on x86_64 and aarch64 i...https://progress.opensuse.org/issues/1550502024-02-07T07:45:38Ztjyrinki_susetjyrinki+redmine@suse.de
<p>See for example:</p>
<p>openQA test in scenario sle-15-SP4-Server-DVD-Updates-aarch64-oscap_ansible_anssi_bp28_high@aarch64-virtio fails in<br>
<a href="https://openqa.suse.de/tests/13447534/modules/boot_to_desktop/steps/8" class="external">boot_to_desktop</a></p>
<p>This is after the yesterday's changes to switch to GUI environment.</p>
<p>However all of s390x passes: <a href="https://openqa.suse.de/tests/13446821" class="external">https://openqa.suse.de/tests/13446821</a></p>
<p>On x86_64 and aarch64 it boots the desktop HDD but somehow the DESKTOP variable is overridden from somewhere still to be textmode, which probably causes the problem.</p>
<p>On s390x it's the same but it happens not to break on boot, but it also doesn't seem to actually use the desktop as DESKTOP=textmode there too.</p>
openQA Tests - action #154696 (Resolved): [security] Most oscap tests fail on 15-SP6 in eval or r...https://progress.opensuse.org/issues/1546962024-02-01T08:59:37Ztjyrinki_susetjyrinki+redmine@suse.de
<p>Most oscap tests are failing on 15-SP6. See <a href="https://openqa.suse.de/tests/overview?arch=&flavor=&machine=&test=&modules=oscap_xccdf_eval&module_re=&group_glob=&not_group_glob=&comment=&distri=sle&version=15-SP6&build=50.2&groupid=268#" class="external">https://openqa.suse.de/tests/overview?arch=&flavor=&machine=&test=&modules=oscap_xccdf_eval&module_re=&group_glob=¬_group_glob=&comment=&distri=sle&version=15-SP6&build=50.2&groupid=268#</a> or latest build.</p>
<p>As an example <a href="https://openqa.suse.de/tests/13393791/modules/oscap_xccdf_eval#2/steps/25" class="external">https://openqa.suse.de/tests/13393791/modules/oscap_xccdf_eval#2/steps/25</a><br>
"fail" count in file stdout is 7, expected 3. Failed rules:<br>
xccdf_org.ssgproject.content_rule_aide_build_database, CCE-85787-0<br>
xccdf_org.ssgproject.content_rule_aide_periodic_cron_checking, CCE-85671-6<br>
xccdf_org.ssgproject.content_rule_aide_scan_notification, CCE-91214-7<br>
xccdf_org.ssgproject.content_rule_accounts_passwords_pam_tally2, CCE-85554-4<br>
xccdf_org.ssgproject.content_rule_audit_rules_privileged_commands_kmod, CCE-85591-6<br>
xccdf_org.ssgproject.content_rule_audit_rules_time_stime, CCE-85815-9<br>
xccdf_org.ssgproject.content_rule_grub2_l1tf_argument, </p>
ALP - action #151558 (Resolved): [security] firstrun on sle-micro >= 6.0 should be ALPifiedhttps://progress.opensuse.org/issues/1515582023-11-28T06:58:19Ztjyrinki_susetjyrinki+redmine@suse.de
<p>openQA test in scenario sle-micro-6.0-Default-encrypted-x86_64-container_selinux@uefi fails in<br>
<a href="https://openqa.suse.de/tests/12874494/modules/firstrun/steps/5" class="external">firstrun</a></p>
<p>Always latest result in this scenario: <a href="https://openqa.suse.de/tests/latest?arch=x86_64&distri=sle-micro&flavor=Default-encrypted&machine=uefi&test=container_selinux&version=6.0" class="external">latest</a></p>
<p>Firstrun is failing with distri sle-micro and version 6.0 - that is, ALP - in the encrypted flavor (that does not use ignition but uses firstrun wizard). This is likely due to it picking up the SLE Micro 5.x expections while it should now work as intended on ALP. There may or may not be difference between ALP Dolomite and ALP Marble in this regard, but likely it should be largely the same.</p>
<p>selinux_setup was updated to use is_alp || is_sle_micro('>=6.0'), jeos/firstrun should maybe using is_sle || is_sle_micro('<6.0') in some places and the former in other places, not sure what would be optimal.</p>
openQA Project - action #127238 (New): It's not possible to remove own force_result comments, and...https://progress.opensuse.org/issues/1272382023-04-05T06:42:05Ztjyrinki_susetjyrinki+redmine@suse.de
<p>Here is an example:</p>
<p><a href="https://openqa.suse.de/tests/10848924#comments" class="external">https://openqa.suse.de/tests/10848924#comments</a></p>
<p>The soft-fail forcing was done erronously, and would be wanted to be removed. However, when trying to remove the comment Forbidden message is given to the person who made the comment (the person is also Administrator).</p>
<p>This ticket looks like opposite of ticket <a class="issue tracker-4 status-1 priority-3 priority-lowest" title="action: It's possible to delete force_result comments (New)" href="https://progress.opensuse.org/issues/107239">#107239</a>, so I'm not sure what to do. However, in the recent months it feels there has been more related changes - now these un-deletable comments are also carried over automatically to new runs, and those carried over comments are also undeletable. If this kind of comment would happen in maintenance tests and couldn't be corrected (without availability of someone who has more direct rights than even Administrator has), it could mean even accidentally giving green light to an erronous update if the correction would be delayed.</p>
openQA Project - action #123724 (Resolved): auto_review not working despite ticket in openQA auto...https://progress.opensuse.org/issues/1237242023-01-27T07:20:44Ztjyrinki_susetjyrinki+redmine@suse.de
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>Ticket #122215 seems not functional still, the affected tests do not get the softfail forcing still automatically but need to be set manually daily.</p>
<p>The ticket has:<br>
<code>auto_review:"Test result is NOT same as baseline":force_result:softfailed</code><br>
... at the end of the subject, and the ticket was moved to openQA auto review project.</p>
<p>The tests can be found at the 15-SP4 runs at <a href="https://openqa.suse.de/group_overview/429" class="external">https://openqa.suse.de/group_overview/429</a></p>
<a name="Acceptance-criteria"></a>
<h2 >Acceptance criteria<a href="#Acceptance-criteria" class="wiki-anchor">¶</a></h2>
<ul>
<li><strong>AC1</strong>: Result is changed to softfailed as specified in the title of the ticket</li>
</ul>
<a name="Suggestions"></a>
<h2 >Suggestions<a href="#Suggestions" class="wiki-anchor">¶</a></h2>
<ul>
<li>Verify that the regex valid/working e.g. <code>auto_review:"Test result is NOT same as baseline":force_result:softfailed</code>, maybe by putting the title in a unit test and seeing if that fails</li>
<li>Check that the ticket is filed correctly, i.e. using the force result tracker - this <strong>seems</strong> to be correct</li>
<li>Check the code in <a href="https://github.com/os-autoinst/scripts" class="external">https://github.com/os-autoinst/scripts</a></li>
</ul>
openQA Project - action #119467 (Resolved): "Internal server error" on opening any job group fron...https://progress.opensuse.org/issues/1194672022-10-27T06:00:25Ztjyrinki_susetjyrinki+redmine@suse.de
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>Build specific pages etc work, but job group front page like <a href="https://openqa.suse.de/group_overview/268" class="external">https://openqa.suse.de/group_overview/268</a> does not.</p>
<a name="Acceptance-criteria"></a>
<h2 >Acceptance criteria<a href="#Acceptance-criteria" class="wiki-anchor">¶</a></h2>
<ul>
<li><strong>AC1:</strong> No obvious related regression on OSD</li>
<li><strong>AC2:</strong> Test coverage for the branding button code exists for both the job comments as well as job group comments</li>
</ul>
openQA Tests - action #94441 (Resolved): [qe-core] Re-schedule pidgin testshttps://progress.opensuse.org/issues/944412021-06-22T05:37:00Ztjyrinki_susetjyrinki+redmine@suse.de
<p>These were disabled due to <a href="https://progress.opensuse.org/issues/94177" class="external">https://progress.opensuse.org/issues/94177</a> and blocking many updates.</p>
<p>However, it turns out pidgin is still part of even SLE 15 WE, so it's a good idea to consider fixing the test too.</p>
<a name="Acceptance-Criteria"></a>
<h2 >Acceptance Criteria<a href="#Acceptance-Criteria" class="wiki-anchor">¶</a></h2>
<p>AC1: Schedule pidgin tests - this time via YAML - on functional + maintenance releases.<br>
AC2: Before enabling, fix any problems that may still remain - likely useful to switch to irc.libera.chat if not yet done.</p>
openQA Tests - action #91187 (Resolved): [qe-core] Add existing tomcat test module to Product QE ...https://progress.opensuse.org/issues/911872021-04-15T06:17:58Ztjyrinki_susetjyrinki+redmine@suse.de
<p>After comparing test coverage of SLE 15 SP2 and SP3 in opneQA I identified several tests that are exclusive to either pre-release or post-release testing and could be easily used to increase our coverage just by using what we already have.</p>
<p>It is possible that are some errors and some of the tests from the list are already being used. It is also just my opinion, therefor review from other colleagues is more then welcome.</p>
<p>These are the X11 tests that run only after release and could be used also before the release (QEM -> product QE)</p>
<p>tests/x11/piglit/piglit.pm<br>
tests/x11/tomcat.pm</p>
openQA Tests - action #91184 (Rejected): [qe-core] Add existing network QEM tests to Product QEhttps://progress.opensuse.org/issues/911842021-04-15T06:17:20Ztjyrinki_susetjyrinki+redmine@suse.de
<p>After comparing test coverage of SLE 15 SP2 and SP3 in opneQA I identified several tests that are exclusive to either pre-release or post-release testing and could be easily used to increase our coverage just by using what we already have.</p>
<p>It is possible that are some errors and some of the tests from the list are already being used. It is also just my opinion, therefor review from other colleagues is more then welcome.</p>
<p>These are the tests that run only after release and could be used also before the release (QEM -> product QE)</p>
<p>tests/network/autofs_client.pm & tests/network/autofs_server.pm<br>
tests/network/salt_master.pm<br>
tests/network/salt_minion.pm<br>
tests/network/samba/samba_adcli.pm</p>
openQA Tests - action #91181 (Resolved): [qe-core] Add existing console QEM tests to Product QEhttps://progress.opensuse.org/issues/911812021-04-15T06:16:21Ztjyrinki_susetjyrinki+redmine@suse.de
<p>After comparing test coverage of SLE 15 SP2 and SP3 in opneQA I identified several tests that are exclusive to either pre-release or post-release testing and could be easily used to increase our coverage just by using what we already have.</p>
<p>It is possible that are some errors and some of the tests from the list are already being used. It is also just my opinion, therefor review from other colleagues is more then welcome.</p>
<p>These are the console tests that run only after release and could be used also before the release (QEM -> product QE)</p>
<p>tests/console/bind.pm<br>
tests/console/dracut_enhanced.pm<br>
tests/console/gd.pm<br>
tests/console/iotop.pm<br>
tests/console/journalctlLevels.pm<br>
tests/console/krb5.pm<br>
tests/console/libgcrypt.pm<br>
tests/console/libssh.pm<br>
tests/console/openssl_nodejs.pm<br>
tests/console/perf.pm<br>
tests/console/rsync_client.pm<br>
tests/console/rsync_server.pm<br>
tests/console/systemd_rpm_macros.pm<br>
tests/console/valgrind.pm</p>
openQA Infrastructure - action #90275 (Resolved): Replacement openQA OSD aarch64 hardware (was: D...https://progress.opensuse.org/issues/902752021-03-18T10:29:54Ztjyrinki_susetjyrinki+redmine@suse.de
<a name="Motivation"></a>
<h2 >Motivation<a href="#Motivation" class="wiki-anchor">¶</a></h2>
<p>We have had a machine from orthos, but we need to renew the lease every now and then or lose it. It would be easier to have a more permanent solution for eg snapshot validations.</p>
<p>openQA tests aarch64 on KVM, this ticket would be about improving the baremetal aarch64 testing. We were not able to test RC1 on baremetal (non-rpi) aarch64 due to lack of access to hardware (eg problems on thunderx10)</p>
<a name="Acceptance-criteria"></a>
<h2 >Acceptance criteria<a href="#Acceptance-criteria" class="wiki-anchor">¶</a></h2>
<ul>
<li><strong>AC1</strong>: <strong>DONE</strong> Make new arm machines available</li>
<li><strong>AC2</strong>: openqaworker-arm-4 is salted and processing jobs</li>
<li><strong>AC3</strong>: openqaworker-arm-5 is salted and processing jobs</li>
</ul>
openQA Tests - action #89479 (Resolved): [qe-core] test fails in sssd - Fix sssd testing with ope...https://progress.opensuse.org/issues/894792021-03-04T10:40:19Ztjyrinki_susetjyrinki+redmine@suse.de
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>openQA test in scenario sle-15-SP3-Online-x86_64-extra_tests_textmode_phub@64bit fails in<br>
<a href="https://openqa.suse.de/tests/5646951/modules/sssd/steps/22" class="external">sssd</a></p>
<p>In the past, sssd used to be tested using python-pam module, however it's unmantained and it belonged in the package hub repo, see <a href="https://bugzilla.suse.com/show_bug.cgi?id=1180120">https://bugzilla.suse.com/show_bug.cgi?id=1180120</a> for more information</p>
<p>It is still affecting other teams like <a href="https://progress.opensuse.org/issues/88736" class="external">public cloud</a>, but on 15SP2 it is still being run for maintenance test repo and passing at <a href="https://openqa.suse.de/tests/5636854">https://openqa.suse.de/tests/5636854</a></p>
<p>In the end, this might be a complete rewrite of the sssd tests, rendering tests/sysauth/sssd.pm useless. A bit of background, those tests are a bit extensive but they were written by a developer that is long gone from the company, investing time in fixing or looking for an alternative to python-pam and rewrite the test suite is more expensive than switching to directly testing the service against the system (by means of directly using pam/passwd/getent)</p>
<a name="Reproducible"></a>
<h2 >Reproducible<a href="#Reproducible" class="wiki-anchor">¶</a></h2>
<p>Fails since (at least) Build <a href="https://openqa.suse.de/tests/4559227" class="external">14.2</a></p>
<a name="Further-details"></a>
<h2 >Further details<a href="#Further-details" class="wiki-anchor">¶</a></h2>
<p>Always latest result in this scenario: <a href="https://openqa.suse.de/tests/latest?arch=x86_64&distri=sle&flavor=Online&machine=64bit&test=extra_tests_textmode_phub&version=15-SP3" class="external">latest</a></p>
<a name="Suggestions"></a>
<h2 >Suggestions<a href="#Suggestions" class="wiki-anchor">¶</a></h2>
<p>Follow the approach used by <a href="https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/12075/files#diff-4e63d910050359ba446d37c691c72c4a4ee4fc8ae8c2b98d00996804e2bdb55dR91" class="external">this</a> pr, that sets up the Directory Service (389-ds in this case, but there's also the information for openldap).</p>
<a name="Acceptance-criteria"></a>
<h2 >Acceptance criteria<a href="#Acceptance-criteria" class="wiki-anchor">¶</a></h2>
<p>AC1: sssd is being tested without other dependencies than pam for all codestreams (openSUSE and SLE)<br>
AC2: Move sssd tests from extra_tests_textmode_phub to extra_tests_textmode<br>
AC3: re-enable sssd testing on public cloud (Or team is informed that they can be re-enabled)</p>
<a name="Followup-during-review"></a>
<h3 >Followup during review<a href="#Followup-during-review" class="wiki-anchor">¶</a></h3>
<p>See how <a href="https://progress.opensuse.org/issues/89320">https://progress.opensuse.org/issues/89320</a> was implemented and evaluate whether a subticket is needed to unify the testing strategy or they are tackling two different approaches</p>
openQA Project - action #56789 (New): New needles from git repository not working with openqa-clo...https://progress.opensuse.org/issues/567892019-09-11T08:14:09Ztjyrinki_susetjyrinki+redmine@suse.de
<p>Use case:</p>
<p>To be able to test tests in production (openqa.opensuse.org or openqa.suse.de) before merging the needles and tests themselves, since results have proven to vary on those loaded machines compared to local openQA instance or VM.</p>
<p>It is currently working if there are no new needles involved, but this issue describes the problem with custom needles which would be theoretically supported by openqa-clone-custom-git-refspec but not working in practice.</p>
<p>Problem description:</p>
<p>First of all, documentation [1] says "Path to needles subdirectory to use, defaults to "needles" within PRODUCTDIR. Can be a git repository URL, comparable to CASEDIR", and CASEDIR states "for example <a href="mailto:git@github.com">git@github.com</a>:os-autoinst/os-autoinst-distri-opensuse.git#feature/test".</p>
<p>[1] <a href="https://github.com/os-autoinst/os-autoinst/blob/master/doc/backend_vars.asciidoc" class="external">https://github.com/os-autoinst/os-autoinst/blob/master/doc/backend_vars.asciidoc</a></p>
<p>However,</p>
<pre><code>openqa-clone-custom-git-refspec https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/8332 https://openqa.opensuse.org/tests/xxxxxxxx --apikey=XXX --apisecret=XXX NEEDLES_DIR=git@github.com:tjyrinki/os-autoinst-needles-opensuse.git#poo-12345
</code></pre>
<p>results in:</p>
<pre><code>Could not create directory '/var/lib/empty/.ssh'.
Host key verification failed.
fatal: Could not read from remote repository.
</code></pre>
<p>Using https instead (NEEDLES_DIR=<a href="https://github.com/tjyrinki/os-autoinst-needles-opensuse.git" class="external">https://github.com/tjyrinki/os-autoinst-needles-opensuse.git</a>) gets one further, but openQA complains about wrong needles location:</p>
<pre><code>init needles from /var/lib/openqa/pool/15/os-autoinst-needles-opensuse
Needle /var/lib/openqa/pool/15/os-autoinst-needles-opensuse/ssh-login-ok-20160813.json is not under project directory /var/lib/openqa/cache/xxx at /usr/lib/os-autoinst/needle.pm line 63.
</code></pre>
<p>Oliver Kurz suggested playing around with the PRODUCTDIR to try to workaround the forbidding use of (correctly checked out) needles dir which I did but it didn't get any better with trying to set that to eg pool/... directories, which also changes on every run.</p>