openSUSE Project Management Tool: Issueshttps://progress.opensuse.org/https://progress.opensuse.org/themes/openSUSE/favicon/favicon.ico?15829177842023-11-28T06:58:19ZopenSUSE Project Management Tool
Redmine ALP - action #151558 (Resolved): [security] firstrun on sle-micro >= 6.0 should be ALPifiedhttps://progress.opensuse.org/issues/1515582023-11-28T06:58:19Ztjyrinki_susetjyrinki+redmine@suse.de
<p>openQA test in scenario sle-micro-6.0-Default-encrypted-x86_64-container_selinux@uefi fails in<br>
<a href="https://openqa.suse.de/tests/12874494/modules/firstrun/steps/5" class="external">firstrun</a></p>
<p>Always latest result in this scenario: <a href="https://openqa.suse.de/tests/latest?arch=x86_64&distri=sle-micro&flavor=Default-encrypted&machine=uefi&test=container_selinux&version=6.0" class="external">latest</a></p>
<p>Firstrun is failing with distri sle-micro and version 6.0 - that is, ALP - in the encrypted flavor (that does not use ignition but uses firstrun wizard). This is likely due to it picking up the SLE Micro 5.x expections while it should now work as intended on ALP. There may or may not be difference between ALP Dolomite and ALP Marble in this regard, but likely it should be largely the same.</p>
<p>selinux_setup was updated to use is_alp || is_sle_micro('>=6.0'), jeos/firstrun should maybe using is_sle || is_sle_micro('<6.0') in some places and the former in other places, not sure what would be optimal.</p>
openQA Tests - action #134012 (New): [qe-core] Repository QA:/Head/SLE-15-SP6 missinghttps://progress.opensuse.org/issues/1340122023-08-09T05:43:01Ztjyrinki_susetjyrinki+redmine@suse.de
<p>Hello QE Core. Can we request enabling the repository mentioned in the subject, or is there some other path to go for?</p>
<p>See openQA test in scenario sle-15-SP6-Online-x86_64-fips_tests_crypt_openvpn_server@64bit fails in<br>
<a href="https://openqa.suse.de/tests/11769917/modules/openvpn_server/steps/86" class="external">openvpn_server</a> - always latest result in this scenario: <a href="https://openqa.suse.de/tests/latest?arch=x86_64&distri=sle&flavor=Online&machine=64bit&test=fips_tests_crypt_openvpn_server&version=15-SP6" class="external">latest</a></p>
<p>"[zypp-core] Exception.cc(log):186 - [qa-head|<a href="http://dist.suse.de/ibs/QA:/Head/SLE-15-SP6" class="external">http://dist.suse.de/ibs/QA:/Head/SLE-15-SP6</a>] Repository type can't be determined."</p>
openQA Project - action #127238 (New): It's not possible to remove own force_result comments, and...https://progress.opensuse.org/issues/1272382023-04-05T06:42:05Ztjyrinki_susetjyrinki+redmine@suse.de
<p>Here is an example:</p>
<p><a href="https://openqa.suse.de/tests/10848924#comments" class="external">https://openqa.suse.de/tests/10848924#comments</a></p>
<p>The soft-fail forcing was done erronously, and would be wanted to be removed. However, when trying to remove the comment Forbidden message is given to the person who made the comment (the person is also Administrator).</p>
<p>This ticket looks like opposite of ticket <a class="issue tracker-4 status-1 priority-3 priority-lowest" title="action: It's possible to delete force_result comments (New)" href="https://progress.opensuse.org/issues/107239">#107239</a>, so I'm not sure what to do. However, in the recent months it feels there has been more related changes - now these un-deletable comments are also carried over automatically to new runs, and those carried over comments are also undeletable. If this kind of comment would happen in maintenance tests and couldn't be corrected (without availability of someone who has more direct rights than even Administrator has), it could mean even accidentally giving green light to an erronous update if the correction would be delayed.</p>
openQA Project - action #123724 (Resolved): auto_review not working despite ticket in openQA auto...https://progress.opensuse.org/issues/1237242023-01-27T07:20:44Ztjyrinki_susetjyrinki+redmine@suse.de
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>Ticket #122215 seems not functional still, the affected tests do not get the softfail forcing still automatically but need to be set manually daily.</p>
<p>The ticket has:<br>
<code>auto_review:"Test result is NOT same as baseline":force_result:softfailed</code><br>
... at the end of the subject, and the ticket was moved to openQA auto review project.</p>
<p>The tests can be found at the 15-SP4 runs at <a href="https://openqa.suse.de/group_overview/429" class="external">https://openqa.suse.de/group_overview/429</a></p>
<a name="Acceptance-criteria"></a>
<h2 >Acceptance criteria<a href="#Acceptance-criteria" class="wiki-anchor">¶</a></h2>
<ul>
<li><strong>AC1</strong>: Result is changed to softfailed as specified in the title of the ticket</li>
</ul>
<a name="Suggestions"></a>
<h2 >Suggestions<a href="#Suggestions" class="wiki-anchor">¶</a></h2>
<ul>
<li>Verify that the regex valid/working e.g. <code>auto_review:"Test result is NOT same as baseline":force_result:softfailed</code>, maybe by putting the title in a unit test and seeing if that fails</li>
<li>Check that the ticket is filed correctly, i.e. using the force result tracker - this <strong>seems</strong> to be correct</li>
<li>Check the code in <a href="https://github.com/os-autoinst/scripts" class="external">https://github.com/os-autoinst/scripts</a></li>
</ul>
openQA Project - action #119467 (Resolved): "Internal server error" on opening any job group fron...https://progress.opensuse.org/issues/1194672022-10-27T06:00:25Ztjyrinki_susetjyrinki+redmine@suse.de
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>Build specific pages etc work, but job group front page like <a href="https://openqa.suse.de/group_overview/268" class="external">https://openqa.suse.de/group_overview/268</a> does not.</p>
<a name="Acceptance-criteria"></a>
<h2 >Acceptance criteria<a href="#Acceptance-criteria" class="wiki-anchor">¶</a></h2>
<ul>
<li><strong>AC1:</strong> No obvious related regression on OSD</li>
<li><strong>AC2:</strong> Test coverage for the branding button code exists for both the job comments as well as job group comments</li>
</ul>
openQA Tests - action #107956 (New): [qe-core][sporadic][qem] test sometimes fails in ntp_client ...https://progress.opensuse.org/issues/1079562022-03-08T07:22:17Ztjyrinki_susetjyrinki+redmine@suse.de
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>openQA test in scenario sle-15-SP3-Server-DVD-Updates-x86_64-mau-extratests-security-fips@64bit fails in<br>
<a href="https://openqa.suse.de/tests/8286663/modules/ntp_client/steps/22" class="external">ntp_client</a></p>
<p>This is part of the FIPS test suite but ntp sounds generid enough for QE Core.</p>
<a name="Test-suite-description"></a>
<h2 >Test suite description<a href="#Test-suite-description" class="wiki-anchor">¶</a></h2>
<p>Testsuite maintained at <a href="https://gitlab.suse.de/qa-maintenance/qam-openqa-yml" class="external">https://gitlab.suse.de/qa-maintenance/qam-openqa-yml</a>.</p>
<a name="Reproducible"></a>
<h2 >Reproducible<a href="#Reproducible" class="wiki-anchor">¶</a></h2>
<p>Fails since (at least) Build <a href="https://openqa.suse.de/tests/8286663" class="external">20220308-1</a> (current job)</p>
<a name="Expected-result"></a>
<h2 >Expected result<a href="#Expected-result" class="wiki-anchor">¶</a></h2>
<p>Last good: <a href="https://openqa.suse.de/tests/8280627" class="external">20220307-1</a> (or more recent)</p>
<a name="Further-details"></a>
<h2 >Further details<a href="#Further-details" class="wiki-anchor">¶</a></h2>
<p>Always latest result in this scenario: <a href="https://openqa.suse.de/tests/latest?arch=x86_64&distri=sle&flavor=Server-DVD-Updates&machine=64bit&test=mau-extratests-security-fips&version=15-SP3" class="external">latest</a></p>
openQA Tests - coordination #107770 (New): [qe-core][epic] Create additional Java testinghttps://progress.opensuse.org/issues/1077702022-03-02T07:02:20Ztjyrinki_susetjyrinki+redmine@suse.de
<p>(draft)</p>
<p>Ideas:</p>
<ul>
<li>maven (in addition to current ant)</li>
<li>Compiling a stable but complicated application - this is to detect if an update would modify compilation experience</li>
<li>Other big java software besides tomcat that could be run</li>
</ul>
openQA Tests - coordination #107065 (New): [qe-core][epic] Zypper test planhttps://progress.opensuse.org/issues/1070652022-02-18T08:40:47Ztjyrinki_susetjyrinki+redmine@suse.de
<p>(draft)</p>
<p>Zypper is core to the updating and migrating SLE so it's important to cover in automated testing the variety of how it is used. Repositories are closely intertwiner with zypper, and especially rmt (repository mirroring tool) is of importance. PackageKit is a frontend that uses zypper via libzypp.</p>
<p>On a high level, zypper is used most of all to:</p>
<ul>
<li>Install the packages to the installed system</li>
<li>Install new packages</li>
<li>Upgrade packages</li>
<li>Upgrade (migrate) the whole system</li>
</ul>
<p>Currently QE Core runs the following:</p>
<ul>
<li>Modules zypper_clear_repos zypper_ar zypper_ref zypper_lr zypper_in zypper_log zypper_up zypper_lifecycle zypper_extend orphaned_packages_check (in extra_tests_textmode and mau-extratests-zypper)</li>
<li>Test suite toolchain_zypper installs a build environment but otherwise is mostly about building, not about zypper any more than many other modules</li>
<li>Especially zypper_extend is quite extensive regression tests suite, I think main functions are quite well covered by it and the other modules</li>
<li>Upgrading between Leap major versions in O3</li>
<li>rmt_feature module via qam_rmt</li>
<li>updates_packagekit_gpk</li>
</ul>
<p>Areas uncovered that QE Core could cover in automated tests:</p>
<ul>
<li>Catch new zypper warning messages that might be important - <a href="https://progress.opensuse.org/issues/106916">https://progress.opensuse.org/issues/106916</a></li>
<li>Execute rmt_feature also in Development version</li>
<li>For cleanup purposes, investigate "console/rmt.pm" and either enable using of it for all releases or get rid of it if it's obsoleted by the console/rmt/* - if removing, also remove RMT_TEST and its use (is_rmt) from main_common (nothing seems to specify RMT_TEST but double-check).</li>
<li>Executing the tests but with using the new environment variables that enable different zypper features. At least ZYPP_SINGLE_RPMTRANS=1, possibly also ZYPP_MEDIANETWORK=1 which is another new feature.</li>
<li>Checking Ctrl-C aborting of zypper midway and resuming</li>
<li>gnome-software - but this is low priority because of SUSE having next to no desktop business compared to Red Hat or Ubuntu</li>
</ul>
<p>Other squads cover:</p>
<ul>
<li>QE Yast tests zypper_in/zypper_lr/zypper_lifecycle/zypper_ref/zypper_log/zypper_clear_repos during install time and has a module yast2_rmt</li>
<li>QE SAP tests zypper in online migration / zypper patch</li>
<li>QE Migration tests migrations between SLE major versions (currently does not cover Leap, so it's left to Core for time being), and executes the same zypper tests as Yast and SAP together, and for rmt modules rmt_feature, rmt_export, rmt_chinese, rmt_host_migration, rmt_import</li>
<li>JeOS tests the same zypper tests as others, including zypper_extend</li>
<li>In openSUSE there's also updates_packagekit_kde</li>
</ul>
openQA Tests - action #94832 (New): [qe-core] Development collaboration service automated deploym...https://progress.opensuse.org/issues/948322021-06-29T06:44:38Ztjyrinki_susetjyrinki+redmine@suse.de
<p>A test case for a site with scaling ready development collaboration, including automated deployment and monitoring.</p>
<p>Ideas/examples about implementation details:</p>
<ul>
<li>Rancher/K3S for the Kubernetes solution</li>
<li>Ansible for automated deployment (actual user wouldn't use openQA for deployment)</li>
<li>Monitoring with Grafana</li>
<li>Gitlab for the collaboration service</li>
</ul>
<a name="Acceptance-Criteria"></a>
<h2 >Acceptance Criteria<a href="#Acceptance-Criteria" class="wiki-anchor">¶</a></h2>
<ul>
<li>A default installation target container as the automation target</li>
<li>One-command (like ansible-playbook deploy.yml) deployment once the target is available for the test to use</li>
<li>A development collaboration platform running</li>
<li>Ready to scale outside the initial container</li>
<li>A monitoring solution deployment for the cluster/service status (no need to configure actual alerts, but that the capability is there using a popular software)</li>
</ul>
openQA Tests - action #94829 (New): [qe-core] Web/app serving end-to-end scenariohttps://progress.opensuse.org/issues/948292021-06-29T06:15:22Ztjyrinki_susetjyrinki+redmine@suse.de
<p>A test case for a theoretical app server running inside a Kubernetes cluster, serving clients.</p>
<p>Ideas/examples about possible implementation details:</p>
<ul>
<li>Rancher K3S for the server <a href="https://rancher.com/docs/rancher/v2.5/en/installation/resources/k8s-tutorials/ha-with-external-db/" class="external">https://rancher.com/docs/rancher/v2.5/en/installation/resources/k8s-tutorials/ha-with-external-db/</a></li>
<li>NodeJS 14 LTS serving either http directly or from behind nginx proxy
<ul>
<li>From SLE repositories where available, or <a href="https://nodejs.org/en/download/package-manager/#opensuse-and-sle" class="external">https://nodejs.org/en/download/package-manager/#opensuse-and-sle</a></li>
</ul></li>
<li>For client, one could use something like minimal containers running multiple (tens of?) httperf in a local network. If SUSE containers not minimal enough, maybe for example docker alpine:3.13.5. One can balance between the number of containers and number of threads within.</li>
</ul>
<p>Try not to overlap eg with what QE Performance is testing, see eg "database/http traffic" tests and others.</p>
<a name="Acceptance-Criteria"></a>
<h2 >Acceptance Criteria<a href="#Acceptance-Criteria" class="wiki-anchor">¶</a></h2>
<ul>
<li>Runs a simple https server (self-signed certificate) inside a single node Kubernetes</li>
<li>Each client connecting to it uses https (ignoring self-signed certificate)</li>
<li>The clients altogether doing at least hundreds of requests per second.</li>
<li>Runs for minimum 1 hour</li>
<li>Runs on at minimum 15-SP2, 15-SP3, Tumbleweed, Leap 15.3 and SLE12 if possible without extra effort</li>
</ul>
openQA Tests - action #94090 (New): [qe-core][qac][study] End-to-end combination test scenarioshttps://progress.opensuse.org/issues/940902021-06-16T08:36:44Ztjyrinki_susetjyrinki+redmine@suse.de
<p>Researching end-to-end scenarios combining multiple SUSE technologies and popular real life uses across functional areas into test cases.</p>
<p>The focus is on testing the combination, not any specific component.</p>
<p>Focus should be in splitting items into smaller subtickets that could be in the backlog eventually. Feel free to create new tickets, adjust current ones, the plans are not set in stone.</p>
openQA Tests - action #93210 (New): [migration][qe-core] sssd openldap/389-ds basic testing, modi...https://progress.opensuse.org/issues/932102021-05-28T08:22:13Ztjyrinki_susetjyrinki+redmine@suse.de
<p>In ticket 89479 the goals for QE Core were fulfilled to modernize directory service testing. However, QE Migration would also like to test sssd with openldap (older SLE) and 389-ds (newer SLE) and testing how the functionality remains after upgrade to newer SLE.</p>
<p>There is a draft design for such tests at <a href="https://confluence.suse.com/display/qasleapac1/Draft+service+check+design+and+implementation" class="external">https://confluence.suse.com/display/qasleapac1/Draft+service+check+design+and+implementation</a> - once there it is fully agreed that we want to go ahead with such architecture, this test, which is probably one of the more important migration tests, should be modified towards that. Whether done by people from QE Core or QE Migration is likely depending on a resource issue, now that developer of 89479 is moving to yet another squad. Community contribution would be of course always welcome as well, although for that the design plan should be incorporated to this ticket. tl;dr; divide the test to 1. install_service, 2. configure_service, 3. enable_service, 4. start_service, 5. check_service, 6. check_function, of which 1-6 are done before distro upgrade and 5-6 after. Maybe it could be possibly to do just division to two steps, and execute either both or just the latter? Ticket will be updated once the draft solidifies.</p>
<a name="Further-Information"></a>
<h2 >Further Information<a href="#Further-Information" class="wiki-anchor">¶</a></h2>
<p>See discussion at <a href="https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/12528" class="external">https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/12528</a> and <a href="https://progress.opensuse.org/issues/89479" class="external">https://progress.opensuse.org/issues/89479</a></p>
<a name="Acceptance-Criteria"></a>
<h2 >Acceptance Criteria<a href="#Acceptance-Criteria" class="wiki-anchor">¶</a></h2>
<p>AC1: Modify test so that it continues to execute as is, but is split so that distro upgrade (migration) tests can be also run using the same module.</p>
openQA Infrastructure - action #90275 (Resolved): Replacement openQA OSD aarch64 hardware (was: D...https://progress.opensuse.org/issues/902752021-03-18T10:29:54Ztjyrinki_susetjyrinki+redmine@suse.de
<a name="Motivation"></a>
<h2 >Motivation<a href="#Motivation" class="wiki-anchor">¶</a></h2>
<p>We have had a machine from orthos, but we need to renew the lease every now and then or lose it. It would be easier to have a more permanent solution for eg snapshot validations.</p>
<p>openQA tests aarch64 on KVM, this ticket would be about improving the baremetal aarch64 testing. We were not able to test RC1 on baremetal (non-rpi) aarch64 due to lack of access to hardware (eg problems on thunderx10)</p>
<a name="Acceptance-criteria"></a>
<h2 >Acceptance criteria<a href="#Acceptance-criteria" class="wiki-anchor">¶</a></h2>
<ul>
<li><strong>AC1</strong>: <strong>DONE</strong> Make new arm machines available</li>
<li><strong>AC2</strong>: openqaworker-arm-4 is salted and processing jobs</li>
<li><strong>AC3</strong>: openqaworker-arm-5 is salted and processing jobs</li>
</ul>
openQA Tests - action #57584 (Resolved): [qam] test fails in evolution_setup_servers - dovecot n...https://progress.opensuse.org/issues/575842019-10-01T11:52:32Ztjyrinki_susetjyrinki+redmine@suse.de
<p>After <a href="https://progress.opensuse.org/issues/56273" class="external">https://progress.opensuse.org/issues/56273</a> got fixed, the evolution_setup_servers is now failing, a rerun done first at <a href="https://openqa.suse.de/tests/3397700" class="external">https://openqa.suse.de/tests/3397700</a> and a newer one at <a href="https://openqa.suse.de/tests/3424806" class="external">https://openqa.suse.de/tests/3424806</a></p>
<p>"No provider of 'dovecot' found." at <a href="https://openqa.suse.de/tests/3397700#step/evolution_prepare_servers/18" class="external">https://openqa.suse.de/tests/3397700#step/evolution_prepare_servers/18</a></p>
<p>qam-regression-message is currently disabled, <a href="https://progress.opensuse.org/issues/57260" class="external">https://progress.opensuse.org/issues/57260</a> is about re-enabling it once the issues are all fixed.</p>
openQA Project - action #56789 (New): New needles from git repository not working with openqa-clo...https://progress.opensuse.org/issues/567892019-09-11T08:14:09Ztjyrinki_susetjyrinki+redmine@suse.de
<p>Use case:</p>
<p>To be able to test tests in production (openqa.opensuse.org or openqa.suse.de) before merging the needles and tests themselves, since results have proven to vary on those loaded machines compared to local openQA instance or VM.</p>
<p>It is currently working if there are no new needles involved, but this issue describes the problem with custom needles which would be theoretically supported by openqa-clone-custom-git-refspec but not working in practice.</p>
<p>Problem description:</p>
<p>First of all, documentation [1] says "Path to needles subdirectory to use, defaults to "needles" within PRODUCTDIR. Can be a git repository URL, comparable to CASEDIR", and CASEDIR states "for example <a href="mailto:git@github.com">git@github.com</a>:os-autoinst/os-autoinst-distri-opensuse.git#feature/test".</p>
<p>[1] <a href="https://github.com/os-autoinst/os-autoinst/blob/master/doc/backend_vars.asciidoc" class="external">https://github.com/os-autoinst/os-autoinst/blob/master/doc/backend_vars.asciidoc</a></p>
<p>However,</p>
<pre><code>openqa-clone-custom-git-refspec https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/8332 https://openqa.opensuse.org/tests/xxxxxxxx --apikey=XXX --apisecret=XXX NEEDLES_DIR=git@github.com:tjyrinki/os-autoinst-needles-opensuse.git#poo-12345
</code></pre>
<p>results in:</p>
<pre><code>Could not create directory '/var/lib/empty/.ssh'.
Host key verification failed.
fatal: Could not read from remote repository.
</code></pre>
<p>Using https instead (NEEDLES_DIR=<a href="https://github.com/tjyrinki/os-autoinst-needles-opensuse.git" class="external">https://github.com/tjyrinki/os-autoinst-needles-opensuse.git</a>) gets one further, but openQA complains about wrong needles location:</p>
<pre><code>init needles from /var/lib/openqa/pool/15/os-autoinst-needles-opensuse
Needle /var/lib/openqa/pool/15/os-autoinst-needles-opensuse/ssh-login-ok-20160813.json is not under project directory /var/lib/openqa/cache/xxx at /usr/lib/os-autoinst/needle.pm line 63.
</code></pre>
<p>Oliver Kurz suggested playing around with the PRODUCTDIR to try to workaround the forbidding use of (correctly checked out) needles dir which I did but it didn't get any better with trying to set that to eg pool/... directories, which also changes on every run.</p>