Project

General

Profile

action #70648

Show serial tests in a more feasible way

Added by dheidler 5 months ago. Updated 4 months ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
Feature requests
Target version:
Start date:
2020-08-28
Due date:
% Done:

0%

Estimated time:
Difficulty:
easy

Description

Currently tests using the serial terminal look like this on the openQA webUI: https://openqa.opensuse.org/tests/1373824#step/timezone/3
This is not very helpful as an icon needs to be selected and then only a single command output is visable.
It might be possible to view the complete serial logfile of the test but then you loose track of which test module starts where as well as the context of record_info output.

The visualisation of external testsuites (like https://openqa.opensuse.org/tests/1373824#step/pam_sh/4) would provide a better way of displaying this output.
Optionally the command and the output of that command (that are currently part of separate results) each could be combined into one result element for both.
A little challange might be to view existing record_infos (which should be easily distinguishable from serial commands) and tests that have a single screenshot within a serial test (https://openqa.opensuse.org/tests/1373824#step/journalctl/15)

History

#1 Updated by mkittler 5 months ago

  • Status changed from New to In Progress
  • Difficulty set to easy

Not having special rendering for serial results would actually simplify the code. Here's a PR: https://github.com/os-autoinst/openQA/pull/3340

#2 Updated by mkittler 5 months ago

  • Status changed from In Progress to Resolved

PR merged

#3 Updated by okurz 5 months ago

  • Status changed from Resolved to Feedback
  • Priority changed from Normal to High
  • Target version set to Ready

Feedback from users in [#opensuse-factory](irc://chat.freenode.net/opensuse-factory):

[01/09/2020 08:22:46] <fvogt> The openQA test page looks broken: https://openqa.opensuse.org/tests/1378056
[01/09/2020 08:23:04] <fvogt> Is that intentional?

I guess this relates to the new way how to display results. I see as a problem that now module results can become very long to display when in before the details where much more condensed and only where needed the user would click to show details. It is getting awkward especially when one is looking for the failed test module step. Personally I even fully prefer the old style. I would like to rediscuss this.

EDIT: Also there is:

[01/09/2020 11:22:59] <defolos> openQA looks different
[01/09/2020 11:23:23] <fvogt> Yes, everything is harder to see now and I've got to scroll for iles
[01/09/2020 11:23:26] <defolos> my scrolwheel is yelling at me
[01/09/2020 11:23:50] <fvogt> My touchpad lost half of its thickness already
[01/09/2020 11:24:22] <defolos> my scrollwheel will need the winter tires change way earlier this year

EDIT: Discussed with mkittler: We should try to keep the code to preload the text content but limit the horizontal display of the text. E.g. keep the size to 60x45 pixel thumbnail boxes (without the chevron but show text preview). We can either revert the existing PR for now until we have incorporated new ideas or keep the state as-is and improve based on that.

#4 Updated by favogt 5 months ago

Some suggestions how the display could be improved:

  • If the test (soft)failed, collapse green test modules (except for a linked #anchor), those aren't the main point of interest anyway
  • Treat successive terminal output/input more like an actual terminal and use an inline line break instead of separate boxes
  • If the output matched, only show the output (with leading and trailing newlines removed) and not the "# wait_serial expected..." and "result:" lines
  • If the space is too narrow for the "Report bug" buttons, hide them until the relevant step is clicked or hovered

#5 Updated by mkittler 5 months ago

PR (first step): https://github.com/os-autoinst/openQA/pull/3350


Treat successive terminal output/input more like an actual terminal and use an inline line break instead of separate boxes

That's a good idea. So the "wavpack" module in the screenshot of my initial change (https://github.com/os-autoinst/openQA/pull/3340) would have only one expandable box. I guess the individual serial outputs could be separated via horizontal lines.

If the output matched, only show the output (with leading and trailing newlines removed) and not the "# wait_serial expected..." and "result:" lines

But isn't the regex sometimes used to figure where the check comes from (test code) and what the purpose of the check was?

#6 Updated by jkohoutek 5 months ago

favogt wrote:

Some suggestions how the display could be improved:

Revealing each result by click is really annoying.

  • if there is need of the reveal, do it for whole test.
  • output should be more readable, if it's like side diff:

green for match

# wait_serial    | # Results:
> qr/1mAID-\d+-/ |  \textcolor{green}{> 1mAID-0-}

red for not match

# wait_serial    | # Results:
> qr/1mAID-\d+-/ |   \textcolor{red}{> 1mAID-1-}

#7 Updated by MDoucha 5 months ago

favogt wrote:

  • Treat successive terminal output/input more like an actual terminal and use an inline line break instead of separate boxes

I'd keep the boxes as quick links into the single long terminal view. When you click the box, it'll open the terminal view and jump to the location of the corresponding output.

mkittler wrote:

If the output matched, only show the output (with leading and trailing newlines removed) and not the "# wait_serial expected..." and "result:" lines

But isn't the regex sometimes used to figure where the check comes from (test code) and what the purpose of the check was?

I'd much prefer to get rid of the # wait_serial expected: blahblahblah as much as possible. The following output format would be much easier to work with. Bold lines represent extra info from OpenQA and would be highlighted with different colors:

boot_ltp.pm:67: script_run('ps axf');
PID TTY STAT TIME COMMAND
2 ? S 0:00 [kthreadd]
3 ? S 0:00 _ [kworker/0:0]
4 ? S< 0:00 _ [kworker/0:0H]
... (snip)
1407 ? Ss 0:00 /usr/sbin/cron -n
1413 ? Ss 0:00 /usr/lib/systemd/systemd --user
1414 ? S 0:00 _ (sd-pam)
Command exit code: 0


boot_ltp.pm:69: script_run('aa-enabled; aa-status');
If 'aa-enabled' is not a typo you can use command-not-found to lookup the package that contains it, like this:
cnf aa-enabled
apparmor module is loaded.
0 profiles are loaded.
0 profiles are in enforce mode.
0 profiles are in complain mode.
0 processes have profiles defined.
0 processes are in enforce mode.
0 processes are in complain mode.
0 processes are unconfined but have a profile defined.
Command exit code: 2


boot_ltp.pm:131: script_run('grep -e Huge -e PageTables /proc/meminfo');
Serial terminal error
wait_serial expected: "grep -e Huge -e PageTables /proc/meminfo; echo zlQih-\$?-"
Result: grep -e Huge -e PageTables /proc/meminfo; eccccccc

#8 Updated by mkittler 4 months ago

  • Status changed from Feedback to In Progress

#9 Updated by mkittler 4 months ago

  • Status changed from In Progress to Feedback

#10 Updated by okurz 4 months ago

mkittler the new new design seems to be ok for most. Could you please add the still open points as ideas to #65271 and close this ticket then?

#11 Updated by mkittler 4 months ago

  • Status changed from Feedback to Resolved

updated #65271

Also available in: Atom PDF