https://progress.opensuse.org/https://progress.opensuse.org/themes/openSUSE/favicon/favicon.ico?15829177842018-03-29T07:48:11ZopenSUSE Project Management ToolopenQA Tests - action #34012: [kernel] too generic test failure in "execute_test_run" for stress tests, was previously something more specific like "acceptance_fs_stress"https://progress.opensuse.org/issues/34012?journal_id=1069542018-03-29T07:48:11Zokurzokurz@suse.com
<ul><li><strong>Assignee</strong> set to <i>yosun</i></li></ul><p><a class="user active user-mention" href="https://progress.opensuse.org/users/15720">@yosun</a> do you have an idea how we can improve the test feedback again? In before we had a more helpful test module name "acceptance_fs_stress" failing, now it is "execute_test_run".</p>
openQA Tests - action #34012: [kernel] too generic test failure in "execute_test_run" for stress tests, was previously something more specific like "acceptance_fs_stress"https://progress.opensuse.org/issues/34012?journal_id=1117572018-04-11T07:34:16Zyosunyosun@suse.com
<ul></ul><p>In this fail case, some time you can't get useful log in snapshot, but by looking into console you may found some Call trace happened.<br>
<a href="https://openqa.suse.de/tests/1561334/file/serial0.txt" class="external">https://openqa.suse.de/tests/1561334/file/serial0.txt</a></p>
<p>Most kernel test fails need to be check via console and tarbal(if has), when you see a fail.<br>
BTW, those three stress test, most likely a kernel acceptance test, if you think it fails most likely in kernel way and better to debug in kernel-qa team. Then feel free to transfer them into kernel job group for better classification.</p>
openQA Tests - action #34012: [kernel] too generic test failure in "execute_test_run" for stress tests, was previously something more specific like "acceptance_fs_stress"https://progress.opensuse.org/issues/34012?journal_id=1120122018-04-11T20:28:15Zokurzokurz@suse.com
<ul></ul><p>yosun wrote:</p>
<blockquote>
<p>In this fail case, some time you can't get useful log in snapshot, but by looking into console you may found some Call trace happened.<br>
<a href="https://openqa.suse.de/tests/1561334/file/serial0.txt" class="external">https://openqa.suse.de/tests/1561334/file/serial0.txt</a></p>
</blockquote>
<p>Well, sorry that does not help because I am mainly asking because the label carry over, the test overview page as well as openqa-review mainly rely on the name of the first test module failing in a scenario.</p>
<blockquote>
<p>Most kernel test fails need to be check via console and tarbal(if has), when you see a fail.</p>
</blockquote>
<p>Well, actually there <em>is</em> a better way. I implemented a simple y2log parser for installation and yast failures already and slindomansilla has adopted it in a cool way for the systemd-testsuite in <a href="https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/4795/files" class="external">https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/4795/files</a> so I am thinking that a similar way could be applied for the kernel test failures as well.</p>
<blockquote>
<p>BTW, those three stress test, most likely a kernel acceptance test, if you think it fails most likely in kernel way and better to debug in kernel-qa team. Then feel free to transfer them into kernel job group for better classification.</p>
</blockquote>
<p>That's actually a good idea. I will discuss with marita as PM and sebchlad as kernel&network PO.</p>
openQA Tests - action #34012: [kernel] too generic test failure in "execute_test_run" for stress tests, was previously something more specific like "acceptance_fs_stress"https://progress.opensuse.org/issues/34012?journal_id=1202972018-05-18T02:51:33Zyosunyosun@suse.com
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>In Progress</i></li></ul><p>okurz wrote:</p>
<blockquote>
<p>yosun wrote:</p>
<blockquote>
<p>In this fail case, some time you can't get useful log in snapshot, but by looking into console you may found some Call trace happened.<br>
<a href="https://openqa.suse.de/tests/1561334/file/serial0.txt" class="external">https://openqa.suse.de/tests/1561334/file/serial0.txt</a></p>
</blockquote>
<p>Well, sorry that does not help because I am mainly asking because the label carry over, the test overview page as well as openqa-review mainly rely on the name of the first test module failing in a scenario.</p>
<blockquote>
<p>Most kernel test fails need to be check via console and tarbal(if has), when you see a fail.</p>
</blockquote>
<p>Well, actually there <em>is</em> a better way. I implemented a simple y2log parser for installation and yast failures already and slindomansilla has adopted it in a cool way for the systemd-testsuite in <a href="https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/4795/files" class="external">https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/4795/files</a> so I am thinking that a similar way could be applied for the kernel test failures as well.</p>
</blockquote>
<p>Nice suggestion, it will reduce the workforce to analysis log. In that case, this ticket need to split into many ticket for particular tests in different set of script. Kernel fails are little bit complicated than userspace's, you need to check test log to find minor issues, to check /var/log/messages or journal log for other issue, and to check kdump files for crash bugs. I suggest to create subtask to follow.<br>
For kernel tests part split to:</p>
<ul>
<li>ctcs2 based testcase</li>
<li>xfstests</li>
<li>ltp</li>
<li>network
...</li>
</ul>
<blockquote>
<blockquote>
<p>BTW, those three stress test, most likely a kernel acceptance test, if you think it fails most likely in kernel way and better to debug in kernel-qa team. Then feel free to transfer them into kernel job group for better classification.</p>
</blockquote>
<p>That's actually a good idea. I will discuss with marita as PM and sebchlad as kernel&network PO.</p>
</blockquote>
<p>Any update in this part?</p>
openQA Tests - action #34012: [kernel] too generic test failure in "execute_test_run" for stress tests, was previously something more specific like "acceptance_fs_stress"https://progress.opensuse.org/issues/34012?journal_id=1229142018-06-01T08:45:53Zyosunyosun@suse.com
<ul><li><strong>Assignee</strong> deleted (<del><i>yosun</i></del>)</li></ul><p>no response, remove assignee.</p>
openQA Tests - action #34012: [kernel] too generic test failure in "execute_test_run" for stress tests, was previously something more specific like "acceptance_fs_stress"https://progress.opensuse.org/issues/34012?journal_id=1229862018-06-03T09:29:46Zokurzokurz@suse.com
<ul><li><strong>Assignee</strong> set to <i>okurz</i></li></ul> openQA Tests - action #34012: [kernel] too generic test failure in "execute_test_run" for stress tests, was previously something more specific like "acceptance_fs_stress"https://progress.opensuse.org/issues/34012?journal_id=1277532018-06-14T15:36:58Zokurzokurz@suse.com
<ul><li><strong>Due date</strong> set to <i>2018-06-19</i></li></ul><p>I just need to talk to sebchlad and mawerner about this but I keep forgetting :(</p>
openQA Tests - action #34012: [kernel] too generic test failure in "execute_test_run" for stress tests, was previously something more specific like "acceptance_fs_stress"https://progress.opensuse.org/issues/34012?journal_id=1289352018-06-15T18:52:54Zokurzokurz@suse.com
<ul><li><strong>Target version</strong> changed from <i>Milestone 17</i> to <i>Milestone 17</i></li></ul> openQA Tests - action #34012: [kernel] too generic test failure in "execute_test_run" for stress tests, was previously something more specific like "acceptance_fs_stress"https://progress.opensuse.org/issues/34012?journal_id=1304382018-06-18T13:16:43Zokurzokurz@suse.com
<ul><li><strong>Due date</strong> changed from <i>2018-06-19</i> to <i>2018-07-17</i></li><li><strong>Status</strong> changed from <i>In Progress</i> to <i>Feedback</i></li><li><strong>Assignee</strong> changed from <i>okurz</i> to <i>sebchlad</i></li></ul><p><a class="user active user-mention" href="https://progress.opensuse.org/users/21718">@sebchlad</a> as discussed. IIUC QSK is fine to take over the "…_stress" test suites and schedule according scenarios within SLE12 and SLE15 tests as well as improve them over time and hopefully also schedule them for relevant openSUSE tests. So I propose that you remove the according scenarios from the SLE15 and SLE12 schedule and add the according ones in the Kernel job group. OK? If you expect QSF to do something feel free to reassign to me, otherwise remove the "[functional]" tag and add "[kernel]" after moving the test scenarios.</p>
openQA Tests - action #34012: [kernel] too generic test failure in "execute_test_run" for stress tests, was previously something more specific like "acceptance_fs_stress"https://progress.opensuse.org/issues/34012?journal_id=1325272018-06-26T12:40:39Zokurzokurz@suse.com
<ul><li><strong>Related to</strong> <i><a class="issue tracker-4 status-3 priority-4 priority-default closed" href="/issues/37782">action #37782</a>: [kernel][functional][u][medium] test fails in execute_test_run because it cannot handle broken pipes</i> added</li></ul> openQA Tests - action #34012: [kernel] too generic test failure in "execute_test_run" for stress tests, was previously something more specific like "acceptance_fs_stress"https://progress.opensuse.org/issues/34012?journal_id=1345102018-07-04T11:40:58Zriafarov
<ul><li><strong>Estimated time</strong> set to <i>5.00 h</i></li></ul> openQA Tests - action #34012: [kernel] too generic test failure in "execute_test_run" for stress tests, was previously something more specific like "acceptance_fs_stress"https://progress.opensuse.org/issues/34012?journal_id=1357492018-07-17T08:34:56Zmgriessmeiermgriessmeier@suse.com
<ul><li><strong>Subject</strong> changed from <i>[functional][u] too generic test failure in "execute_test_run" for stress tests, was previously something more specific like "acceptance_fs_stress"</i> to <i>[kernel] too generic test failure in "execute_test_run" for stress tests, was previously something more specific like "acceptance_fs_stress"</i></li><li><strong>Due date</strong> deleted (<del><i>2018-07-17</i></del>)</li><li><strong>Target version</strong> deleted (<del><i>Milestone 17</i></del>)</li></ul> openQA Tests - action #34012: [kernel] too generic test failure in "execute_test_run" for stress tests, was previously something more specific like "acceptance_fs_stress"https://progress.opensuse.org/issues/34012?journal_id=1375282018-07-23T10:17:40Zsebchladsebastian.chlad@suse.com
<ul><li><strong>Status</strong> changed from <i>Feedback</i> to <i>Workable</i></li><li><strong>Assignee</strong> deleted (<del><i>sebchlad</i></del>)</li><li><strong>Target version</strong> set to <i>445</i></li></ul> openQA Tests - action #34012: [kernel] too generic test failure in "execute_test_run" for stress tests, was previously something more specific like "acceptance_fs_stress"https://progress.opensuse.org/issues/34012?journal_id=1379692018-07-25T06:07:35Zyosunyosun@suse.com
<ul></ul><p>qa_test_* and xfstests use different testscript.<br>
qa_test_* use test script in tests/qa_automation <br>
xfstests test script in tests/xfstests</p>
<p>Then I remove this related issue.</p>
openQA Tests - action #34012: [kernel] too generic test failure in "execute_test_run" for stress tests, was previously something more specific like "acceptance_fs_stress"https://progress.opensuse.org/issues/34012?journal_id=1379842018-07-25T06:11:24Zokurzokurz@suse.com
<ul></ul><p>ok, sorry</p>
openQA Tests - action #34012: [kernel] too generic test failure in "execute_test_run" for stress tests, was previously something more specific like "acceptance_fs_stress"https://progress.opensuse.org/issues/34012?journal_id=1380022018-07-25T06:30:09Zyosunyosun@suse.com
<ul></ul><p>no problem, btw, I just more sched_stress, fs_stress, process_stress into kernel job group as told in previous comments.</p>
openQA Tests - action #34012: [kernel] too generic test failure in "execute_test_run" for stress tests, was previously something more specific like "acceptance_fs_stress"https://progress.opensuse.org/issues/34012?journal_id=1380082018-07-25T06:34:50Zokurzokurz@suse.com
<ul></ul><p>great. Can you do the same for the <a href="https://openqa.suse.de/admin/job_templates/110" class="external">SLE15</a> codestream please?</p>
openQA Tests - action #34012: [kernel] too generic test failure in "execute_test_run" for stress tests, was previously something more specific like "acceptance_fs_stress"https://progress.opensuse.org/issues/34012?journal_id=1382182018-07-25T09:41:50Zyosunyosun@suse.com
<ul></ul><p>The sle15 code base used by QAM now. So I just remove them from functional job group in sle15, to add them or no could decide by QAM.</p>
openQA Tests - action #34012: [kernel] too generic test failure in "execute_test_run" for stress tests, was previously something more specific like "acceptance_fs_stress"https://progress.opensuse.org/issues/34012?journal_id=1382272018-07-25T09:44:39Zyosunyosun@suse.com
<ul></ul><p>I guess you mean SLE15SP1, I also remove from there, and will add in kernel job group when needed.</p>
openQA Tests - action #34012: [kernel] too generic test failure in "execute_test_run" for stress tests, was previously something more specific like "acceptance_fs_stress"https://progress.opensuse.org/issues/34012?journal_id=1382602018-07-25T11:51:09Zokurzokurz@suse.com
<ul></ul><p>I stated SLE15 codestream. Of course I mean whatever current service pack is in development. Please make sure to only remove it when you also add it. As we use the same job group for the whole SLE15 code stream I suggest to just add it to the kernel tests already now.</p>
openQA Tests - action #34012: [kernel] too generic test failure in "execute_test_run" for stress tests, was previously something more specific like "acceptance_fs_stress"https://progress.opensuse.org/issues/34012?journal_id=1383652018-07-26T01:09:11Zyosunyosun@suse.com
<ul></ul><p>OK, done</p>
openQA Tests - action #34012: [kernel] too generic test failure in "execute_test_run" for stress tests, was previously something more specific like "acceptance_fs_stress"https://progress.opensuse.org/issues/34012?journal_id=1383862018-07-26T06:11:04Zokurzokurz@suse.com
<ul></ul><p><a class="user active user-mention" href="https://progress.opensuse.org/users/15720">@yosun</a> the team QSF thanks you :)</p>
openQA Tests - action #34012: [kernel] too generic test failure in "execute_test_run" for stress tests, was previously something more specific like "acceptance_fs_stress"https://progress.opensuse.org/issues/34012?journal_id=1856272019-02-06T10:34:46Zsebchladsebastian.chlad@suse.com
<ul><li><strong>Target version</strong> changed from <i>445</i> to <i>future</i></li></ul> openQA Tests - action #34012: [kernel] too generic test failure in "execute_test_run" for stress tests, was previously something more specific like "acceptance_fs_stress"https://progress.opensuse.org/issues/34012?journal_id=2527762019-10-25T11:06:13Zjlausuchjalausuch@suse.com
<ul><li><strong>Status</strong> changed from <i>Workable</i> to <i>Resolved</i></li></ul><p>sched_stress, fs_stress, process_stress are not run in kernel job group any more. </p>
openQA Tests - action #34012: [kernel] too generic test failure in "execute_test_run" for stress tests, was previously something more specific like "acceptance_fs_stress"https://progress.opensuse.org/issues/34012?journal_id=2529712019-10-25T20:52:07Zokurzokurz@suse.com
<ul><li><strong>Status</strong> changed from <i>Resolved</i> to <i>Workable</i></li></ul><p>but they are! E.g. <a href="https://openqa.suse.de/tests/3527726" class="external">https://openqa.suse.de/tests/3527726</a> in <a href="https://openqa.suse.de/group_overview/116" class="external">Kernel</a> for SLE15SP2. Also the ticket is not about "tests should not be run in kernel group" but about a too generic test failure in "execute_test_run". Please see the initial ticket description for the actual issue. Btw, <a href="https://openqa.suse.de/tests/3530449" class="external">mau-qa_acceptance_fs_stress</a> seems to handle this better already with <code>QA_TESTSET=acceptance_fs_stress</code> instead of <code>QA_TESTSUITE=fs_stress</code>. Maybe that's the easy fix.</p>
openQA Tests - action #34012: [kernel] too generic test failure in "execute_test_run" for stress tests, was previously something more specific like "acceptance_fs_stress"https://progress.opensuse.org/issues/34012?journal_id=2530072019-10-26T11:18:36Zjlausuchjalausuch@suse.com
<ul><li><strong>Status</strong> changed from <i>Workable</i> to <i>Resolved</i></li></ul><p>okurz wrote:</p>
<blockquote>
<p>but they are! E.g. <a href="https://openqa.suse.de/tests/3527726" class="external">https://openqa.suse.de/tests/3527726</a> in <a href="https://openqa.suse.de/group_overview/116" class="external">Kernel</a> for SLE15SP2. </p>
</blockquote>
<p>Our bad. We decided to not run stress tests in kernel group and removed it for SLE12-SP5 but forgot about SLE15 group. Now they are removed from that group as well.</p>
<blockquote>
<p>Also the ticket is not about "tests should not be run in kernel group" but about a too generic test failure in "execute_test_run". Please see the initial ticket description for the actual issue. Btw, <a href="https://openqa.suse.de/tests/3530449" class="external">mau-qa_acceptance_fs_stress</a> seems to handle this better already with <code>QA_TESTSET=acceptance_fs_stress</code> instead of <code>QA_TESTSUITE=fs_stress</code>. Maybe that's the easy fix.</p>
</blockquote>
<p>I know this ticket is not about that. Anyway, I have removed <code>QA_TESTSUITE=fs_stress</code> and added <code>QA_TESTSET=acceptance_fs_stress</code> in the test settings in case someone wants to enable that test again for whatever purpose.</p>
openQA Tests - action #34012: [kernel] too generic test failure in "execute_test_run" for stress tests, was previously something more specific like "acceptance_fs_stress"https://progress.opensuse.org/issues/34012?journal_id=2530132019-10-26T12:15:52Zokurzokurz@suse.com
<ul></ul><p>Yes, thanks. I think this should solve it for good! Without testing we wouldn't know if the "testset" approach works but let's take the chance ;)</p>