action #52517
closed
Sorting out tests: Visualize parallel job dependencies in test overview (was: Enhance the view for the multimachine tests)
Added by sebchlad over 5 years ago.
Updated over 5 years ago.
Category:
Feature requests
Description
The current openQA webUI seems sorting out elements (tests) alphabetically.
For instance: https://openqa.suse.de/tests/overview?distri=sle&version=15-SP1&build=228.2&groupid=130
In the meantime, Marius implemented the feature of displaying dependencies within a test run.
https://openqa.suse.de/tests/2923320#dependencies
I'm introducing more tests for cluster testing, so there are multiple VMs (cluster nodes) which are displayed as separate 'tests' while in fact those are VMs constituting multi-node test run.
The current webUI does not provide a nice way to display such tests in a way which would be easy for people to recognize it.
I wonder if there is a quick way to take advantage of Marius' work and be able to render the page of a job group for a given build in way to group certain tests together.
Of course this could be achieved by proper naming, but this still seems like a workaround and not a proper solution.
I wonder if people see similar need and if there are any ideas on how to go about this use case.
Files
I wonder if there is a quick way to take advantage of Marius' work
One could definitely move OpenQA::WebAPI::Controller::Test::dependencies() and related functions to its own module and re-use that data for other pages, too.
be able to render the page of a job group for a given build in way to group certain tests together
I assume you're talking about the test result overview page as linked and not the job group page. I'm just asking because the test result overview page is not necessarily showing results of a certain job group.
This would be a table with nested rows for chained dependencies and 'grouped' rows for parallel dependencies.
- Not sure whether the DataTables-API for nested rows would be the right fit here. Maybe the level of nesting could simply be visualized by using indentation.
- Parallel dependencies could be highlighted using a background color like in the graph.
Dependencies in openQA as displayed in the graph and used by the scheduler are tracked on job-level. However, one row in the test result overviewr represents multiple jobs (one for each architecture). That is a problem and I'm not sure how to solve it.
Hi Marius, seems like there is no more need to clarify this: "I assume you're talking about the test result overview page", tight?
Just for the sake of being super-clear let me add: in the attached files you can see intermixed tests.
There are in fact 2 "test scenarios" which are multi-machine tests.
(1) hpc_slurm_adv_supportserver
- hpc_slurm_master_adv_1
- hpc_slurm_master_backup
- hpc_slurmslave_adv_01
- hpc_slurmslave_adv_02
(2) hpc_slurm_supportserver
- hpc_slurm_master
- hpc_slurm_slave_01
- hpc_slurm_slave_02
- hpc_slurm_slave_03
Would be great to be able to group multimachine tests together on the the test result overview page.
I will be having way more multi-machine jobs for the same components, so alphabetical sorting will be getting more and more in the way.
mkittler wrote:
Dependencies in openQA as displayed in the graph and used by the scheduler are tracked on job-level. However, one row in the test result overviewr represents multiple jobs (one for each architecture). That is a problem and I'm not sure how to solve it.
I have absolutely no experience in that part of the openqa. Will check that however.
- File Screenshot_20190605_233019.png Screenshot_20190605_233019.png added
- Subject changed from Sorrting out tests: enhance the view for the multimachine tests to Sorting out tests: Visualize parallel job dependencies in test overview (was: Enhance the view for the multimachine tests)
- Category set to Feature requests
Adjusting subject as I understood this should be mainly about the test overview and also mainly "parallel jobs" and less the linear dependencies of what jobs run after which.
What I am thinking is that the circles for multiple parallel jobs could be collapsed into a single blob, e.g. with
<span class="fa-clone"></span>
from https://fontawesome.com/icons/clone?style=solid
which can look like 
@sebchlad already identified the right location in code which could be adjusted, e.g.
https://github.com/os-autoinst/openQA/blob/master/lib/OpenQA/WebAPI/Controller/Test.pm#L567
where we could go over the result hash and combine all parallel test results into a stack.
- Clicking on the icon could in the most simple case just open the result from the first of the parallel job, e.g. jump immediately to the "Dependencies" tab.
- How to handle labels on either job? -> I suggest to combine them all, e.g. for two parallel jobs with one label each show the stack icon and two flash / bug icons next to it.
- The retry button can link to the first job as AFAIK the parallel jobs are started with it anyway already – or should be ;)
- Optional: On hovering over the icon show all stacked test job icons with popover (clickable, to be able to click into all icons, failed modules, bug icon, etc.)
That icon looks nice for a stack and I agree with merging the labels and the restart. But I guess if you click on it, it should open a popup with the individual results to go there.
I wonder how to name the cluster though. The parent would be 'supportserver' in most - so it's not a good choice.
coolo wrote:
if you click on it, it should open a popup with the individual results to go there.
That's what I meant with the popover, like a mini test overview iframe ;)
I wonder how to name the cluster though. The parent would be 'supportserver' in most - so it's not a good choice.
Yeah, maybe start to look for a name from the end of a list ;)
I think I would make that explicit and rather show a custom string from a test variable in any of the stack's job with fallback to all names with ellipsis.
Open point: what result and color to show when they differ within the stack. I am thinking of unicorn-glitter-rainbow ;) or with a precedence: incomplete beats fail beats cancel beats softfail beats passed
I'd think the user expectation on the result color is indeed 'worst'. Supporting a custom cluster name sounds sensible, as fallback the alphabeticly primary child or something alike followed by a ...
As far as the popover is concerned: Don't overdo graphing there. A list of test results will do.
That way to visualize it would make sense. One could still re-use the algo from the graph to find clusters. The custom cluster name could simply be assigned as a setting on one of the jobs. That would at least be easy to implement within the current cluster algo and the dependency graph could benefit from it as well.
Having a cluster name for creating subgroups might actually be sensible as well - so cluster parallel jobs automatically (with autopicking a name if not present) and other jobs if they have the same cluster name. If parallel jobs have multiple cluster names, we'll explode and call shutdown
This cluster name might actually be an excellent choice for a job group template parameter - and so the circle closes :)
- Status changed from New to Rejected
- Assignee set to okurz
Incorporated what I think are our findings into the duplicatee #15850
Also available in: Atom
PDF
