QA » openQA Project

This is what the backend generates:

   {
      "needles" : [],
      "result" : "unk"
   },
   {
      "text" : "userspace_postfix-4.txt",
      "title" : "wait_serial",
      "needles" : [],
      "result" : "ok"
   },
   {
      "result" : "unk",
      "needles" : []
   },
   {
      "result" : "fail",
      "text" : "userspace_postfix-6.txt",
      "needles" : [],
      "title" : "wait_serial"
   },
   {
      "needles" : [],
      "result" : "unk"
   },
   {
      "result" : "fail",
      "needles" : [],
      "title" : "wait_serial",
      "text" : "userspace_postfix-8.txt"
   },

No idea what these unk steps are

Assigning to current sprint (201712.1)

Just to update this ticket with my findings:
There are three types of results:

1. ok
2. fail
3. unk

The "unk" is used when the developer wants to show the result in the webinterface, without being necessarily a failure or a success, like for example to show progress of some operation.

When creating the miniatures, the object is initialized as being unk.

The problem in this case is when is determining if the terminal is serial or not in here: https://github.com/os-autoinst/os-autoinst/blame/master/basetest.pm#L413

If i remove the unless from there, everything (on that test) seems to be working fine:
http://tragicbox.suse.cz/tests/296#step/userspace_postfix/1

This needs more in depth analysis, as i'm afraid, this might break a lot of stuff. :-(

the problem is that there is nothing happening on that screen you keep recording. So recording a result here is just wrong. the return value is just unused - and shouldn't be there.

So based on your findings, I would try:

--- a/basetest.pm
+++ b/basetest.pm
@@ -409,8 +409,8 @@ sub record_serialresult {

     $string //= '';

-    my $result = $self->record_testresult('unk');
     unless (testapi::is_serial_terminal) {
+        my $result = $self->record_testresult('unk');
         # the screenshot is not the fail, it's just for documentation
         $self->_result_add_screenshot($result);
     }
@@ -418,7 +418,7 @@ sub record_serialresult {
     $output .= "# Result:\n";
     $output .= "$string\n";
     $self->record_resultfile('wait_serial', $output, result => $res);
-    return $result;
+    return;
 }

 sub record_soft_failure_result {
@@ -429,7 +429,7 @@ sub record_soft_failure_result {
     $self->_result_add_screenshot($result);
     my $output = "# Soft Failure:\n$reason\n";
     $self->record_resultfile('Soft Failed', $output, result => $result);
-    return $result;
+    return;
 }

 sub register_extra_test_results {

coolo wrote:

the problem is that there is nothing happening on that screen you keep recording. So recording a result here is just wrong. the return value is just unused - and shouldn't be there.

Thanks for the clarification.

coolo wrote:

So based on your findings, I would try:

Thanks! I guess that's it. My tests passed with the changes you wrote: http://tragicbox.suse.cz/tests/329#step/userspace_postfix/1