https://progress.opensuse.org/https://progress.opensuse.org/themes/openSUSE/favicon/favicon.ico?15829177842018-04-11T08:53:57ZopenSUSE Project Management ToolopenQA Project - action #34507: Document requirements for external test suite results to be rendered by openQAhttps://progress.opensuse.org/issues/34507?journal_id=1118292018-04-11T08:53:57Zszarate
<ul><li><strong>Subject</strong> changed from <i>Document requirements for external test results to be rendered by openQA</i> to <i>Document requirements for external test suite results to be rendered by openQA</i></li></ul><p>I had a conversation with Anton yesterday regarding this ticket.</p>
<p>One thing to note, is that this is only meant to allow projects that want to execute their own test suites within openQA, and have openQA to also display these results.</p>
<p>One idea is to use LTP as use case, and there is also the <a href="https://openqa.opensuse.org/tests/639337" class="external">Open Build Service tests</a> that are also doing something similar.</p>
openQA Project - action #34507: Document requirements for external test suite results to be rendered by openQAhttps://progress.opensuse.org/issues/34507?journal_id=1124142018-04-13T07:45:24Zszarate
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>In Progress</i></li></ul><p>I'm working on this since yesterday</p>
openQA Project - action #34507: Document requirements for external test suite results to be rendered by openQAhttps://progress.opensuse.org/issues/34507?journal_id=1124922018-04-13T11:36:36ZEDiGiacintoedigiacinto@suse.com
<ul><li><strong>Assignee</strong> set to <i>szarate</i></li></ul> openQA Project - action #34507: Document requirements for external test suite results to be rendered by openQAhttps://progress.opensuse.org/issues/34507?journal_id=1141452018-04-20T08:01:42Zszarate
<ul><li><strong>File</strong> <a href="/attachments/6043">2018-04-20-094734_1020x143_scrot.png</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/6043/2018-04-20-094734_1020x143_scrot.png">2018-04-20-094734_1020x143_scrot.png</a> added</li></ul><p>Current requirements to be able to use the external results parser resolve to:</p>
<ul>
<li>Using a compatible format that is registered in <a href="https://github.com/os-autoinst/openQA/blob/master/lib/OpenQA/Schema/Result/Jobs.pm#L1258" class="external">OpenQA::Schema::Result::Jobs::parse_extra_tests</a></li>
<li><p>The test results from the external Harness can be uploaded via testapi::parse_extra_log($output_file) within an openQA tests.</p>
<p>script_run('prove --verbose --formatter=TAP::Formatter::JUnit t/28-logging.t > junit-logging.xml');<br>
parse_extra_log('XUnit', 'junit-logging.xml');</p></li>
</ul>
<p>Current examples of the formats are in the t/data directory:</p>
<ul>
<li><a href="https://github.com/os-autoinst/openQA/blob/master/t/data/ltp_test_result_format.json" class="external">LTP</a></li>
<li><a href="https://github.com/os-autoinst/openQA/blob/master/t/data/new_ltp_result_array.json" class="external">LTP New format</a></li>
<li><a href="https://github.com/os-autoinst/openQA/blob/master/t/data/slenkins_control-junit-results-fail.xml" class="external">JUnit</a></li>
<li><a href="https://github.com/os-autoinst/openQA/blob/master/t/data/xunit_format_example.xml" class="external">XUnit</a></li>
</ul>
<p>Once the files have been uploaded, currently openQA will display them as shown in the atached image, which is simply by adding an extra row with the text results.</p>
<ul>
<li>It should be noted that currently only the slenkins junit tests are properly parsed when specifiying JUnit, JUnit parser is specific for slenkins, if the user wishes to use JUnit comming from another source that spills JUnit, then XUnit should be used. </li>
<li>A TAP parser is being developed <a href="https://github.com/foursixnine/openQA/commit/8e8d5c6547459861d4bc32d124cf1be18f637374" class="external">here</a> as a poc</li>
</ul>
openQA Project - action #34507: Document requirements for external test suite results to be rendered by openQAhttps://progress.opensuse.org/issues/34507?journal_id=1151082018-04-24T12:10:42Zsebchladsebastian.chlad@suse.com
<ul></ul><p><a class="user active user-mention" href="https://progress.opensuse.org/users/23010">@szarate</a>: what's the status on this one? Documenting requirements is still in progress? Can I help here as the Product Owner of the QA-Kernel&Network scrum team? I'm worry if collecting requirements take now 12 days, how long will the implementation take?</p>
<p>I'm serious. Don't we overengineer here? </p>
openQA Project - action #34507: Document requirements for external test suite results to be rendered by openQAhttps://progress.opensuse.org/issues/34507?journal_id=1153692018-04-25T07:04:40Zszarate
<ul><li><strong>Related to</strong> <i><a class="issue tracker-4 status-3 priority-4 priority-default closed" href="/issues/35374">action #35374</a>: Rendering of external harness output in test results as rows</i> added</li></ul> openQA Project - action #34507: Document requirements for external test suite results to be rendered by openQAhttps://progress.opensuse.org/issues/34507?journal_id=1153752018-04-25T07:04:45Zszarate
<ul><li><strong>Related to</strong> deleted (<i><a class="issue tracker-4 status-3 priority-4 priority-default closed" href="/issues/35374">action #35374</a>: Rendering of external harness output in test results as rows</i>)</li></ul> openQA Project - action #34507: Document requirements for external test suite results to be rendered by openQAhttps://progress.opensuse.org/issues/34507?journal_id=1153812018-04-25T07:04:58Zszarate
<ul><li><strong>Related to</strong> <i><a class="issue tracker-4 status-3 priority-4 priority-default closed" href="/issues/35374">action #35374</a>: Rendering of external harness output in test results as rows</i> added</li></ul> openQA Project - action #34507: Document requirements for external test suite results to be rendered by openQAhttps://progress.opensuse.org/issues/34507?journal_id=1175022018-05-03T07:42:33Zszarate
<ul></ul><p><a class="user active user-mention" href="https://progress.opensuse.org/users/21718">@sebchlad</a> what took long there was actually developing a new parser. Anywho, we already spoke about this the same day :)</p>
openQA Project - action #34507: Document requirements for external test suite results to be rendered by openQAhttps://progress.opensuse.org/issues/34507?journal_id=1175042018-05-03T07:43:08Zszarate
<ul></ul><p>PR is open, only tests for new parser are failing</p>
openQA Project - action #34507: Document requirements for external test suite results to be rendered by openQAhttps://progress.opensuse.org/issues/34507?journal_id=1213422018-05-23T09:04:26Zszarate
<ul><li><strong>Status</strong> changed from <i>In Progress</i> to <i>Resolved</i></li><li><strong>Target version</strong> changed from <i>Current Sprint</i> to <i>Done</i></li></ul><p>poo#34204 still needs work but as for this ticket, we are kind of finished.</p>