action #60458
closed
Improve consistency of job states/results (was: /tests/overview shows Passed: 0, Failed: 0 in summary but nothing else for a build that consists of single incomplete job only)
Added by okurz about 5 years ago.
Updated almost 5 years ago.
Category:
Regressions/Crashes
which consists of a single, incomplete job
The problem here is that these jobs don't have the exact result "incomplete" but instead one of the other results considered incomplete:
use constant INCOMPLETE_RESULTS => (INCOMPLETE, SKIPPED, OBSOLETED, PARALLEL_FAILED,
PARALLEL_RESTARTED, USER_CANCELLED, USER_RESTARTED, TIMEOUT_EXCEEDED
);
I suppose the overview statistics and the filter should take all of these results into account when looking for incomplete jobs.
- Status changed from New to In Progress
- Target version set to Current Sprint
PR: https://github.com/os-autoinst/openQA/pull/2564
[...] when filtering for only incomplete jobs
Note that the PR doesn't change the filter. It still distinguishes between the different incomplete results, e.g. one can filter for timeouts specifically.
- Subject changed from /tests/overview shows Passed: 0, Failed: 0 in summary but nothing else for a build that consists of single incomplete job only to Improve consistency of job states/results (was: /tests/overview shows Passed: 0, Failed: 0 in summary but nothing else for a build that consists of single incomplete job only)
We already have "meta" results like "incomplete results" defined as constant list. It makes sense to improve the concept and use it consistently so accounting/filtering in test results overview, favicons and other CSS formatting behaves consistently and redundant code is avoided. The names of these "meta" results should also be improved (to avoid confusion with actual results).
WIP branch for improving the UI as well: https://github.com/Martchus/openQA/tree/meta-results-ui
However, I've ran into a few problems/questions:
- "skipped" in the UI isn't exactly the meta-result "aborted". It accounts jobs with "aborted" results and jobs without result in state "cancelled". To resolve this I would change "skipped" to "aborted" in the UI ignoring that it is not exactly the
ABORTED meta-result. Note that "skipped" for job modules should stay as-is. This is just about overall job results.
- Then there's also "failed" which actually maps to "not ok results". But the meta-result name is not really nice and I wouldn't show it in the UI. So I prefer to keep "failed" here.
- "unfinished" would become "pending". The meaning in this context is any of "scheduled", "assigned", "setup", "running" or "uploading". So in this context there actually are further results to be expected and therefore "pending" would actually make sense.
Generally it is a bit messy that the UI mixes results, meta-results, states and meta-states here but this somehow makes sense. We sometimes just don't have a result so the state matters, some results should be finder distinguished and for others the meta-result is sufficient, …. So I wouldn't overthink/oversimplify it for the sake of consistency with our internal names.
- Copied to action #63802: Further improve consistency of job states/results added
- Status changed from In Progress to Resolved
Discussed this in the weekly QA Tools team meeting. We agreed that the mentioned suggestions are valid further improvements but not necessarily something that we want or can do know. But this ticket originally started out as a concrete bug that was fixed in the meantime. Hence moving the left suggestions out into a specific ticket #63802 . Whenever we work on states/results again we should keep this in mind or pick it up again at any time.
Also available in: Atom
PDF