action #178885
opencoordination #154768: [saga][epic][ux] State-of-art user experience for openQA
coordination #157345: [epic] Improved test reviewer user experience
Console test overview
0%
Description
Hello,
I want to discuss the test run overview, especially for text-mode tests.
Right now, we have two ways to check these tests in openQA:
1) Clicking through the small squares to figure out the commands and results. This is confusing, especially for new people.
2) Reading the serial_terminal.txt
file, which is just a huge, unformatted text dump, making it hard to find what's important.
Compared to other automation tools, this is really hard to read.
I understand openQA started with GUI automation, and that's still important. But, I think we should have two overview tabs: one for GUI tests, and one for CLI tests.
I suggest a view with clearly separated commands, where the input and output are easily recognizable. Each test module could be foldable, allowing users to collapse sections as needed. While this view might be less compact than the current one, it would significantly enhance readability.
I also think that the backend internals (e.g. ; echo Gi8yV-$?-
or cat > /tmp/scriptOSm16.sh << 'EOT_OSm16'; echo OSm16-$?-
) should be hidden.
I've attached screenshots to show what I mean.
Happy hacking,
Pavel Dostál
Files
Updated by okurz about 1 month ago
- Category set to Feature requests
- Target version set to future
I agree with your assessment. I think the overall design with the "little boxes" can be improved with a better approach for both VNC output as well as serial terminal based output.
Updated by pdostal about 1 month ago
- Description updated (diff)
- Category deleted (
Feature requests) - Target version deleted (
future)
Updated by okurz about 1 month ago
- Category set to Feature requests
- Target version set to future
Updated by pdostal about 1 month ago
- Description updated (diff)
- Category deleted (
Feature requests) - Target version deleted (
future)
Updated by pdostal about 1 month ago
- Category set to Feature requests
- Target version set to future
Updated by ybonatakis about 1 month ago · Edited
- Description updated (diff)
- Category deleted (
Feature requests) - Target version deleted (
future) - Parent task set to #157345
I think this belongs as a subtask to the #157345
Updated by tinita about 1 month ago
- Category set to Feature requests
- Target version set to future
Updated by MDoucha about 1 month ago · Edited
I've been thinking about how to make test results cleaner and easier to navigate for a while and here's a very simple suggestion: Collect the command, output and exit code into a single box.
https://github.com/os-autoinst/os-autoinst-distri-opensuse/commit/8fd43d237262f442fbdd51a1823e26d4feae3d31
Verification run: https://openqa.suse.de/tests/overview?distri=sle&version=15-SP6&build=mdoucha_cmd_run
The above jobs run an artificial scenario:
echo foo
in serial terminalsleep 3
in serial terminal with 1 second timeoutecho foo
in VNCsleep 3
in VNC with 1 second timeoutfalse
in serial terminal with assert (job dies)
As a bonus, the functions return console output in list context so they can also replace script_output()
.
For quick demonstration, I've put the cmd_run()
functions in osado lib/utils.pm but this code could be easily moved to script_run()
in os-autoinst. There is also space for further improvements, for example rendering the results the same way as wait_serial()
boxes and allowing verbose wait_serial()
through job settings.
Updated by livdywan about 1 month ago
Another example of redudant output: https://openqa.suse.de/tests/17054787#step/azure_aitl/340
Most of the "result" boxes are really 1) revealing implementation details or 2) adding internallly formatted versions of the original commands and files used.