action #34507

Document requirements for external test suite results to be rendered by openQA

Added by szarate almost 2 years ago. Updated almost 2 years ago.

Status:ResolvedStart date:09/04/2018
Priority:NormalDue date:
Assignee:szarate% Done:

0%

Category:Feature requests
Target version:Done
Difficulty:
Duration:

Description

Currently there's some documentation on this pr that has some information on how to use the parser.

The openQA documentation needs to have the required information on how to use external test results and allow to parse/display them on the webUI.

AC1: Projects can look up what's required from their test results so that openQA displays the information from external test results.

2018-04-20-094734_1020x143_scrot.png - Current view of the external parser (7.85 KB) szarate, 20/04/2018 07:49 am

6043

Related issues

Related to openQA Project - action #35374: Rendering of external harness output in test results as rows Resolved 23/04/2018

History

#2 Updated by szarate almost 2 years ago

  • Subject changed from Document requirements for external test results to be rendered by openQA to Document requirements for external test suite results to be rendered by openQA

I had a conversation with Anton yesterday regarding this ticket.

One thing to note, is that this is only meant to allow projects that want to execute their own test suites within openQA, and have openQA to also display these results.

One idea is to use LTP as use case, and there is also the Open Build Service tests that are also doing something similar.

#3 Updated by szarate almost 2 years ago

  • Status changed from New to In Progress

I'm working on this since yesterday

#4 Updated by EDiGiacinto almost 2 years ago

  • Assignee set to szarate

#5 Updated by szarate almost 2 years ago

Current requirements to be able to use the external results parser resolve to:

  • Using a compatible format that is registered in OpenQA::Schema::Result::Jobs::parse_extra_tests
  • The test results from the external Harness can be uploaded via testapi::parse_extra_log($output_file) within an openQA tests.

    script_run('prove --verbose --formatter=TAP::Formatter::JUnit t/28-logging.t > junit-logging.xml');
    parse_extra_log('XUnit', 'junit-logging.xml');

Current examples of the formats are in the t/data directory:

Once the files have been uploaded, currently openQA will display them as shown in the atached image, which is simply by adding an extra row with the text results.

  • It should be noted that currently only the slenkins junit tests are properly parsed when specifiying JUnit, JUnit parser is specific for slenkins, if the user wishes to use JUnit comming from another source that spills JUnit, then XUnit should be used.
  • A TAP parser is being developed here as a poc

#6 Updated by sebchlad almost 2 years ago

@szarate: what's the status on this one? Documenting requirements is still in progress? Can I help here as the Product Owner of the QA-Kernel&Network scrum team? I'm worry if collecting requirements take now 12 days, how long will the implementation take?

I'm serious. Don't we overengineer here?

#7 Updated by szarate almost 2 years ago

  • Related to action #35374: Rendering of external harness output in test results as rows added

#8 Updated by szarate almost 2 years ago

  • Related to deleted (action #35374: Rendering of external harness output in test results as rows)

#9 Updated by szarate almost 2 years ago

  • Related to action #35374: Rendering of external harness output in test results as rows added

#10 Updated by szarate almost 2 years ago

@sebchlad what took long there was actually developing a new parser. Anywho, we already spoke about this the same day :)

#11 Updated by szarate almost 2 years ago

PR is open, only tests for new parser are failing

#12 Updated by szarate almost 2 years ago

  • Status changed from In Progress to Resolved
  • Target version changed from Current Sprint to Done

poo#34204 still needs work but as for this ticket, we are kind of finished.

Also available in: Atom PDF