openSUSE Project Management Tool: Issueshttps://progress.opensuse.org/https://progress.opensuse.org/themes/openSUSE/favicon/favicon.ico?15829177842023-02-28T02:53:45ZopenSUSE Project Management Tool
Redmine openQA Project - action #125102 (New): Support multi machine when triggering job use 'isos post'https://progress.opensuse.org/issues/1251022023-02-28T02:53:45ZXiaojing_liuxliu1@suse.com
<a name="User-story"></a>
<h2 >User story<a href="#User-story" class="wiki-anchor">¶</a></h2>
<p>As a QE, sometimes I need to trigger jobs on different machines. But when I use <code>isos post</code>, the parameter <code>MACHINE</code> only can accept one value. Could you help to support more than one values of <code>MACHINE</code>? just like the parameter <code>TEST=foo,boo</code></p>
<p>For example, we define three cases in job group:</p>
<pre><code> test_case0:
machine: hana01
test_case1:
machine: [hana02,hana04]
settings:
PARALLEL_with: test_case0@hana01
test_case2:
machine: [hana03,hana05]
settings:
PARALLEL_with: test_case0@hana01
</code></pre>
<p>We would like to trigger the three jobs on one time: test_case0@hana01, test_case1@hana02, test_case2@hana03, <strong>OR</strong> test_case0@hana01, test_case1@hana04, test_case2@hana05.</p>
<p>Thanks.</p>
<a name="Acceptance-criteria"></a>
<h2 >Acceptance criteria<a href="#Acceptance-criteria" class="wiki-anchor">¶</a></h2>
<ul>
<li>We could specify many MACHINEs when doing <code>isos post</code>, e.g. <code>MACHINE=hana01,hana02</code>, and only the jobs test1@hana01, test2@hana02 will be triggered.</li>
</ul>
openQA Tests - action #111899 (Resolved): [sle][security][backlog][CC]automation: Integrate test ...https://progress.opensuse.org/issues/1118992022-06-01T07:45:58ZXiaojing_liuxliu1@suse.com
<a name="Story"></a>
<h2 >Story<a href="#Story" class="wiki-anchor">¶</a></h2>
<p>According to <a href="https://confluence.suse.com/pages/viewpage.action?spaceKey=CC&title=Evaluator+Test+Plan+SLES+15+SP2#EvaluatorTestPlanSLES15SP2-4.8Accessiblenetworkinterface" class="external">https://confluence.suse.com/pages/viewpage.action?spaceKey=CC&title=Evaluator+Test+Plan+SLES+15+SP2#EvaluatorTestPlanSLES15SP2-4.8Accessiblenetworkinterface</a><br>
integrate this test case into openQA</p>
openQA Tests - action #111809 (Resolved): [sle][security][backlog][CC]automation: Integrate test ...https://progress.opensuse.org/issues/1118092022-05-31T03:47:20ZXiaojing_liuxliu1@suse.com
<a name="Story"></a>
<h2 >Story<a href="#Story" class="wiki-anchor">¶</a></h2>
<p>According to <a href="https://confluence.suse.com/pages/viewpage.action?spaceKey=CC&title=Evaluator+Test+Plan+SLES+15+SP2#EvaluatorTestPlanSLES15SP2-4.7Checkforprocessorvulnerabilitymitigations" class="external">https://confluence.suse.com/pages/viewpage.action?spaceKey=CC&title=Evaluator+Test+Plan+SLES+15+SP2#EvaluatorTestPlanSLES15SP2-4.7Checkforprocessorvulnerabilitymitigations</a><br>
integrate this test case into openQA</p>
openQA Tests - action #111671 (Resolved): [sle][security][backlog][CC]automation: Integrate test ...https://progress.opensuse.org/issues/1116712022-05-27T07:14:51ZXiaojing_liuxliu1@suse.com
<a name="Story"></a>
<h2 >Story<a href="#Story" class="wiki-anchor">¶</a></h2>
<p>According to <a href="https://confluence.suse.com/pages/viewpage.action?spaceKey=CC&title=Evaluator+Test+Plan+SLES+15+SP2#EvaluatorTestPlanSLES15SP2-4.6Checkforundocumentedsecurity-relevantprograms" class="external">https://confluence.suse.com/pages/viewpage.action?spaceKey=CC&title=Evaluator+Test+Plan+SLES+15+SP2#EvaluatorTestPlanSLES15SP2-4.6Checkforundocumentedsecurity-relevantprograms</a><br>
integrate this test case into openQA</p>
openQA Tests - action #111518 (Resolved): [sle][security][backlog][CC]automation: Integrate test ...https://progress.opensuse.org/issues/1115182022-05-24T10:11:34ZXiaojing_liuxliu1@suse.com
<a name="Story"></a>
<h2 >Story<a href="#Story" class="wiki-anchor">¶</a></h2>
<p>According to <a href="https://confluence.suse.com/pages/viewpage.action?spaceKey=CC&title=Evaluator+Test+Plan+SLES+15+SP2#EvaluatorTestPlanSLES15SP2-4.5Permissionsettingsofrelevantconfigurationfiles" class="external">https://confluence.suse.com/pages/viewpage.action?spaceKey=CC&title=Evaluator+Test+Plan+SLES+15+SP2#EvaluatorTestPlanSLES15SP2-4.5Permissionsettingsofrelevantconfigurationfiles</a><br>
integrate this test case into openQA</p>
openQA Tests - action #108548 (Blocked): [sle][security][backlog]automation: Integrate 'secure-bo...https://progress.opensuse.org/issues/1085482022-03-18T03:11:45ZXiaojing_liuxliu1@suse.com
<a name="Story"></a>
<h2 >Story<a href="#Story" class="wiki-anchor">¶</a></h2>
<p>We got a feature <a href="https://jira.suse.com/browse/SLE-18851" class="external">https://jira.suse.com/browse/SLE-18851</a></p>
<a name="Task"></a>
<h2 >Task<a href="#Task" class="wiki-anchor">¶</a></h2>
<ol>
<li>Start a OS on Power with secure-boot=on</li>
<li>Check if the secure boot is enabled after OS started.</li>
<li>load signed kernel and modules successfully</li>
<li>load signed with wrong key kernel and modules fail</li>
</ol>
openQA Tests - action #108170 (Blocked): [sle][security][ppc64le] Move secure-boot test cases int...https://progress.opensuse.org/issues/1081702022-03-11T06:41:32ZXiaojing_liuxliu1@suse.com
<p>All powerkvm workers in OSD install qemu-5.2.0, this version doesn't support secure-boot=on parameter.<br>
So we setup an openQA to test this feature: <a href="https://jira.suse.com/browse/SLE-18851" class="external">https://jira.suse.com/browse/SLE-18851</a><br>
And test cases are: <a href="http://10.161.8.44/tests/9#step/bootloader/10" class="external">http://10.161.8.44/tests/9#step/bootloader/10</a> <a href="http://10.161.8.44/tests/18#" class="external">http://10.161.8.44/tests/18#</a></p>
<p>After powerKVM workers in OSD support this, we need to move the cases into OSD.</p>
openQA Infrastructure - action #94399 (Workable): No alert when arm workers are offline, alert if...https://progress.opensuse.org/issues/943992021-06-22T03:30:25ZXiaojing_liuxliu1@suse.com
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>On 2021-06-22, all arm workers (arm-1, arm-2, arm-3) couldn't be connected by using <code>ssh</code> or <code>ping</code>.<br>
But <a href="https://stats.openqa-monitor.qa.suse.de/d/4KkGdvvZk/osd-status-overview?orgId=1" class="external">https://stats.openqa-monitor.qa.suse.de/d/4KkGdvvZk/osd-status-overview?orgId=1</a> showed that all of them were <code>Online</code>.</p>
<a name="Acceptance-criteria"></a>
<h2 >Acceptance criteria<a href="#Acceptance-criteria" class="wiki-anchor">¶</a></h2>
<ul>
<li><del><strong>AC1:</strong> We can receive the alerting e-mail when arm workers down</del></li>
<li><strong>AC2:</strong> <a href="https://stats.openqa-monitor.qa.suse.de/d/4KkGdvvZk/osd-status-overview?orgId=1" class="external">https://stats.openqa-monitor.qa.suse.de/d/4KkGdvvZk/osd-status-overview?orgId=1</a> should show the correct state</li>
<li><strong>AC3:</strong> We receive alert notices for errors in telegraf on osd</li>
</ul>
<a name="Suggestions"></a>
<h2 >Suggestions<a href="#Suggestions" class="wiki-anchor">¶</a></h2>
<ol>
<li>We should look into feeding something into influxdb when the telegraf service especially on OSD shows errors or log error monitoring</li>
<li>Than one could add a dashboard/graph with an alert within Grafana using the data from <code>1.</code>.</li>
</ol>
openQA Project - action #81374 (New): openqa-dump-templates dumps all result (product, machine ....https://progress.opensuse.org/issues/813742020-12-28T10:15:34ZXiaojing_liuxliu1@suse.com
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>openqa-dump-templates --help shows that</p>
<pre><code> --test NAME
dump only testsuite with specified name
</code></pre>
<p>but when the user specifies <code>--test</code>, other information (such as product, machines) also been dumped.</p>
<p>Users reported this in <a href="https://chat.suse.de/channel/testing?msg=k4aSuyB7zgW4tWTsa" class="external">https://chat.suse.de/channel/testing?msg=k4aSuyB7zgW4tWTsa</a></p>
<a name="Steps-to-reproduce"></a>
<h2 >Steps to reproduce<a href="#Steps-to-reproduce" class="wiki-anchor">¶</a></h2>
<ul>
<li>just run the command <code>openqa-dump-templates --test create_hdd_gnome --json</code></li>
</ul>
<a name="Suggestion"></a>
<h2 >Suggestion<a href="#Suggestion" class="wiki-anchor">¶</a></h2>
<p>the help information is the same as the script's operation</p>
openQA Project - action #81022 (Workable): auto_review:"cache failure: Failed to download.*Asset ...https://progress.opensuse.org/issues/810222020-12-14T07:41:28ZXiaojing_liuxliu1@suse.com
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>There are many jobs that run on openqaworker-arm-2 incomplete. The reason is:</p>
<pre><code>[info] [#387] Downloading "SLES-12-SP4-aarch64-minimal_installed_for_LTP.qcow2" from "http://openqa.suse.de/tests/5169987/asset/hdd/SLES-12-SP4-aarch64-minimal_installed_for_LTP.qcow2"
[info] [#387] Download of "/var/lib/openqa/cache/openqa.suse.de/SLES-12-SP4-aarch64-minimal_installed_for_LTP.qcow2" failed: 521 Connect timeout
[info] [#387] Purging "/var/lib/openqa/cache/openqa.suse.de/SLES-12-SP4-aarch64-minimal_installed_for_LTP.qcow2" because of too many download errors
[2020-12-14T07:25:44.0888 UTC] [error] [pid:44645] Failed to download SLES-12-SP4-aarch64-minimal_installed_for_LTP.qcow2 to /var/lib/openqa/cache/openqa.suse.de/SLES-12-SP4-aarch64-minimal_installed_for_LTP.qcow2. Asset was pruned immediately after download (poo#71827), please retrigger
</code></pre>
<p>See more details in: <br>
job:<br>
<a href="https://openqa.suse.de/tests/5170023" class="external">https://openqa.suse.de/tests/5170023</a><br>
worker:<br>
<a href="https://openqa.suse.de/admin/workers/1078" class="external">https://openqa.suse.de/admin/workers/1078</a></p>
<a name="Steps-to-reproduce"></a>
<h2 >Steps to reproduce<a href="#Steps-to-reproduce" class="wiki-anchor">¶</a></h2>
<p>Find jobs referencing this ticket with the help of<br>
<a href="https://raw.githubusercontent.com/os-autoinst/scripts/master/openqa-query-for-job-label" class="external">https://raw.githubusercontent.com/os-autoinst/scripts/master/openqa-query-for-job-label</a> ,<br>
call <code>openqa-query-for-job-label poo#81022</code></p>
<a name="Expected-result"></a>
<h2 >Expected result<a href="#Expected-result" class="wiki-anchor">¶</a></h2>
<p>obviously the asset that we rely upon should not be pruned immediately :)</p>
<a name="Workaround"></a>
<h2 >Workaround<a href="#Workaround" class="wiki-anchor">¶</a></h2>
<p>Retrigger. auto-review should automatically retrigger anyway.</p>
openQA Tests - action #77956 (New): [openQABot] openQA supports showing parent group name when ge...https://progress.opensuse.org/issues/779562020-11-16T06:53:53ZXiaojing_liuxliu1@suse.com
<a name="Motivation"></a>
<h2 >Motivation<a href="#Motivation" class="wiki-anchor">¶</a></h2>
<p>OpenQA supports showing parent group name when getting a job's details, and the pr has been deployed on o3, will be deployed on OSD this week.<br>
If a job belongs to a parent job group, the parent job group name and id will been shown.<br>
such as:</p>
<pre><code>openqa-cli api --o3 jobs/1470855 --pretty
{
"job" : {
... ....
"group" : "Development Tumbleweed",
"group_id" : 38,
"has_parents" : 0,
"id" : 1470855,
"name" : "opensuse-Tumbleweed-DVD-x86_64-Build20201113-create_hdd_textmode_ext4@uefi",
"parent_group" : "Development",
"parent_group_id" : 6,
},
</code></pre>
<a name="Acceptance-criteria"></a>
<h2 >Acceptance criteria<a href="#Acceptance-criteria" class="wiki-anchor">¶</a></h2>
<ul>
<li><strong>AC1:</strong> All jobs below the parent job group "Development" are ignored regardless if "Development" is in the name of the child job group itself</li>
</ul>
<a name="Suggestion"></a>
<h2 >Suggestion<a href="#Suggestion" class="wiki-anchor">¶</a></h2>
<p>maybe we could use the parent group name to check if a job belongs to <code>Development</code> in <a href="https://gitlab.suse.de/qa-maintenance/openQABot/-/blob/master/openqabot/utils.py#L127" class="external">https://gitlab.suse.de/qa-maintenance/openQABot/-/blob/master/openqabot/utils.py#L127</a></p>
openQA Project - action #73525 (New): Job incompletes with auto_review:"backend died: unexpected ...https://progress.opensuse.org/issues/735252020-10-19T08:10:55ZXiaojing_liuxliu1@suse.com
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>job <a href="https://openqa.suse.de/tests/4847590" class="external">https://openqa.suse.de/tests/4847590</a> is incomplete, the logs show:</p>
<pre><code>[2020-10-19T02:18:59.635 CEST] [debug] <<< backend::svirt::start_serial_grab(name="openQA-SUT-1")
[2020-10-19T02:18:59.635 CEST] [debug] <<< backend::baseclass::start_ssh_serial(username="root", password="SECRET", hostname="s390p8.suse.de")
[2020-10-19T02:18:59.635 CEST] [debug] <<< backend::baseclass::new_ssh_connection(username="root", password="SECRET", hostname="s390p8.suse.de")
�[37m[2020-10-19T02:18:59.740 CEST] [debug] SSH connection to root@s390p8.suse.de established
�[0m�[37m[2020-10-19T02:18:59.790 CEST] [debug] svirt: grabbing serial console
�[0mConnected to domain openQA-SUT-1
Escape character is ^]
�[37m[2020-10-19T02:19:00.058 CEST] [debug] Backend process died, backend errors are reported below in the following lines:
unexpected end of data at /usr/lib/os-autoinst/consoles/VNC.pm line 932.
</code></pre>
<p>See more details in <a href="https://openqa.suse.de/tests/4847590/file/autoinst-log.txt" class="external">https://openqa.suse.de/tests/4847590/file/autoinst-log.txt</a></p>
openQA Project - action #73273 (Workable): job incompletes with auto_review:"setup failure: Cache...https://progress.opensuse.org/issues/732732020-10-13T07:57:03ZXiaojing_liuxliu1@suse.com
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>job <a href="https://openqa.suse.de/tests/4812517" class="external">https://openqa.suse.de/tests/4812517</a> is incomplete, the log shows:</p>
<pre><code>[2020-10-13T04:14:16.0500 CEST] [debug] [pid:36910] Updating status so job 4812517 is not considered dead.
[2020-10-13T04:14:16.0501 CEST] [debug] [pid:36910] REST-API call: POST http://openqa.suse.de/api/v1/jobs/4812517/status
[2020-10-13T04:14:21.0568 CEST] [debug] [pid:36910] Updating status so job 4812517 is not considered dead.
[2020-10-13T04:14:21.0569 CEST] [debug] [pid:36910] REST-API call: POST http://openqa.suse.de/api/v1/jobs/4812517/status
[2020-10-13T04:14:21.0630 CEST] [error] [pid:36910] Unable to setup job 4812517: Cache service status error from API: Minion job #43206 failed: DBD::SQLite::db do failed: file is not a database at /usr/lib/perl5/vendor_perl/5.26.1/Mojo/SQLite.pm line 85.
</code></pre>
<p>please see more details in <a href="https://openqa.suse.de/tests/4812517/file/worker-log.txt" class="external">https://openqa.suse.de/tests/4812517/file/worker-log.txt</a></p>
<a name="Acceptance-criteria"></a>
<h2 >Acceptance criteria<a href="#Acceptance-criteria" class="wiki-anchor">¶</a></h2>
<ul>
<li><strong>AC1:</strong> The issue is automatically handled or the test maintainer is informed what he or she did wrong</li>
</ul>
<a name="Suggestions"></a>
<h2 >Suggestions<a href="#Suggestions" class="wiki-anchor">¶</a></h2>
<ul>
<li>Try to handle this problem with an automatic recovery within openQA if this is not a problem introduced by test maintainers (unlikely)</li>
</ul>
<a name="Workaround"></a>
<h2 >Workaround<a href="#Workaround" class="wiki-anchor">¶</a></h2>
<p>Delete cache database on the affected worker and retry</p>
openQA Project - action #71227 (Workable): [ux] job incompletes with auto_review:"backend died: '...https://progress.opensuse.org/issues/712272020-09-11T07:47:21ZXiaojing_liuxliu1@suse.com
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p><a href="https://openqa.suse.de/tests/4673747">https://openqa.suse.de/tests/4673747</a> in incomplete, reason is: "Backend process died: 'current_console' is not set at /usr/lib/os-autoinst/backend/baseclass.pm line 932."<br>
<a href="https://openqa.suse.de/tests/4673747/file/autoinst-log.txt">https://openqa.suse.de/tests/4673747/file/autoinst-log.txt</a> shows:</p>
<pre><code>�[37m[2020-09-11T02:59:50.763 CEST] [debug] Backend process died, backend errors are reported below in the following lines:
'current_console' is not set at /usr/lib/os-autoinst/backend/baseclass.pm line 932.
backend::baseclass::wait_serial(backend::svirt=HASH(0x55a087239aa8), HASH(0x55a08804a420)) called at /usr/lib/os-autoinst/backend/baseclass.pm line 89
backend::baseclass::handle_command(backend::svirt=HASH(0x55a087239aa8), HASH(0x55a088c2c420)) called at /usr/lib/os-autoinst/backend/baseclass.pm line 625
backend::baseclass::check_socket(backend::svirt=HASH(0x55a087239aa8), IO::Handle=GLOB(0x55a087bb56f8), 0) called at /usr/lib/os-autoinst/backend/svirt.pm line 337
backend::svirt::check_socket(backend::svirt=HASH(0x55a087239aa8), IO::Handle=GLOB(0x55a087bb56f8), 0) called at /usr/lib/os-autoinst/backend/baseclass.pm line 276
eval {...} called at /usr/lib/os-autoinst/backend/baseclass.pm line 191
backend::baseclass::run_capture_loop(backend::svirt=HASH(0x55a087239aa8)) called at /usr/lib/os-autoinst/backend/baseclass.pm line 146
backend::baseclass::run(backend::svirt=HASH(0x55a087239aa8), 13, 16) called at /usr/lib/os-autoinst/backend/driver.pm line 86
backend::driver::__ANON__(Mojo::IOLoop::ReadWriteProcess=HASH(0x55a087733d48)) called at /usr/lib/perl5/vendor_perl/5.26.1/Mojo/IOLoop/ReadWriteProcess.pm line 326
eval {...} called at /usr/lib/perl5/vendor_perl/5.26.1/Mojo/IOLoop/ReadWriteProcess.pm line 326
Mojo::IOLoop::ReadWriteProcess::_fork(Mojo::IOLoop::ReadWriteProcess=HASH(0x55a087733d48), CODE(0x55a088ef7228)) called at /usr/lib/perl5/vendor_perl/5.26.1/Mojo/IOLoop/ReadWriteProcess.pm line 477
Mojo::IOLoop::ReadWriteProcess::start(Mojo::IOLoop::ReadWriteProcess=HASH(0x55a087733d48)) called at /usr/lib/os-autoinst/backend/driver.pm line 88
backend::driver::start(backend::driver=HASH(0x55a0884b1268)) called at /usr/lib/os-autoinst/backend/driver.pm line 52
backend::driver::new("backend::driver", "svirt") called at /usr/bin/isotovideo line 222
main::init_backend() called at /usr/bin/isotovideo line 273
</code></pre>
<p>Checking the git log of test distri between "first bad" and "last good" (when tests were green again) shows:</p>
<pre><code>$ git log1 --no-merges de9fd2b5dcf5d9bc31807e29cd720ac80a1c7f9f..bb6dc4672fc0177c4fb20d7dfd08ddb824e0d0a4
f9f7351ab Remove /tmp as subvolume in profile validation
ee4098d8f Improve pre- and post-feature test
74cd182e8 Rename sap.slice to SAP.slice
f2d6865d7 Wait snapper to settle down to assert root-console
</code></pre>
<p>with some changes to "lib/virt_feature_test_base.pm" which might be related.</p>
<a name="Problem"></a>
<h2 >Problem<a href="#Problem" class="wiki-anchor">¶</a></h2>
<p>This can happen if the console is not properly initialized by test code.<br>
From the error message it is unclear if test code is the culprit or some temporary, flaky error in the infrastructure. This could be cleaner.</p>
<a name="Acceptance-criteria"></a>
<h2 >Acceptance criteria<a href="#Acceptance-criteria" class="wiki-anchor">¶</a></h2>
<ul>
<li><strong>AC1:</strong> It is obvious to the test reviewer if this is a (temporary) infrastructure issue or a problem in test code that needs to be fixed</li>
</ul>
<a name="Suggestions"></a>
<h2 >Suggestions<a href="#Suggestions" class="wiki-anchor">¶</a></h2>
<ul>
<li>Crosscheck how "current_console" is used within os-autoinst</li>
<li>Improve error handling with explicit error messages on misuse</li>
<li>Optional: Apply something similar for "reset_consoles". For this also see #60437 and <a href="https://github.com/os-autoinst/os-autoinst/pull/1288">https://github.com/os-autoinst/os-autoinst/pull/1288</a></li>
</ul>
openQA Project - action #59085 (New): Make the YAML editor in Job group more user friendlyhttps://progress.opensuse.org/issues/590852019-11-05T10:11:51ZXiaojing_liuxliu1@suse.com
<p>Users need to write YAML file when editing a job group. It is not friendly. There are some requests about syntax and structure. should we provide a way to make the YAML editor more friendly?</p>