Project

General

Profile

Actions
Copy link

action #17462

closed

Testlib function record_info doesn't mark overall and category results as 'fail'

Added by thehejik about 7 years ago. Updated about 7 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
okurz
Category:
Feature requests
Target version:
-
Start date:
2017-03-02
Due date:
% Done:

0%

Estimated time:

Description

How to reproduce:

  • Create a new file for test and put following:

    use base "opensusebasetest";
    use strict;
    use testapi;
    sub run() {
    record_info('It failed!', 'Enter some text here', result => 'fail');
    }
    sub test_flags() {
    return {important => 1};
    }
    1;

  • call the file within main.pm and run the test.

You will see that corresponding box detail is correctly red/failed but overall and category result remained 'ok'.

record_info_failure

Original PR lives at https://github.com/os-autoinst/os-autoinst/pull/716/files

Actions
Copy link
 #1

Updated by okurz about 7 years ago

  • Project changed from openQA Tests to openQA Project
  • Category set to 132
  • Assignee set to okurz
Actions
Copy link
 #2

Updated by okurz about 7 years ago

not currently working on this

Actions
Copy link
 #3

Updated by thehejik about 7 years ago

Possibly related also to https://openqa.suse.de/tests/834809 - test failed but overall result is PASSED :(

maybe record_resultfile function?

Actions
Copy link
 #4

Updated by okurz about 7 years ago

  • Status changed from New to Feedback

I checked the implementation and the documentation of 'record_info' as I defined it in testapi.pm. It behaves exactly like described there "meant for informational purposes". 'record_info' records one step like a "check_screen" or "script_run" which can also fail but does not fail the test. I see the following ways:

  • add optional argument to record_info to softfail/fail the module and therefore influence the job result
  • use additional explicit record_soft_fail or a fail call in the test where needed

@thehejik what do you think?

Actions
Copy link
 #5

Updated by okurz about 7 years ago

  • Status changed from Feedback to Resolved

assuming you agree with the second approach for now

Actions
Copy link

Also available in: Atom PDF