Project

General

Profile

coordination #65402

coordination #34357: [epic] Improve openQA performance

[epic] Revamp test details page to improve loading times and prevent timeouts

Added by mkittler over 1 year ago. Updated 12 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Feature requests
Target version:
-
Start date:
2017-08-25
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Difficulty:

Description

problem

The test details page loads very slow for big results and easily runs into timeouts.

suggestions

  • By default we should only load content synchronously that comes from the database, everything loading from filesystem should be done asynchronously or "on demand", either later by requests from the initial page loaded or by user actions, e.g. when clicking on tabs or buttons.
  • Refreshing the entire page (currently done for scheduled/assigned/setup/uploaded jobs) should be avoided.
  • The query for results should not only be done asynchronously but also be splitted to prevent timeouts (similarly like modules are currently loaded on-by-one while a test is running).
  • Running jobs should not be treated completely different from other jobs (like it is done now).
    • The only thing which can be avoided when a job is done is the check for status changes.

notes

  • Of course loading from "fast" (expensive) storage like SSD or NVMe can help but shouldn't be relied upon.
  • So far the table for "External results" is lot "live updated" while the job is running and only refreshed once the job is done. For reducing the immediate effort I would not change this for now.
  • So far we are polling for the job state. This could be improved by using web sockets but maybe that would be over-engineering.
  • Avoiding refreshing the entire page means that additional HTML elements (e.g. for the live view) need to be displayed via JavaScript.
    • It would be possible to render these HTML fragments on the server and request and insert them as needed in the client.
    • It would be possible to render HTML elements for all future job states from the beginning and hide them until a certain state is reached.
  • All points mentioned here and in the sub tickets should be considered together as everything is related. Maybe improvements can nevertheless be done in steps (which raises the question where to start).

Also see https://progress.opensuse.org/issues/25814#note-15 and https://progress.opensuse.org/issues/32611#note-8

running_to_uploading.mkv (59.7 KB) running_to_uploading.mkv video recording of state transition in browser from running to uploading after https://github.com/os-autoinst/openQA/pull/2932 okurz, 2020-04-21 18:48
uploading_to_passed.mkv (51.8 KB) uploading_to_passed.mkv video recording of state transition in browser from uploading to passed after https://github.com/os-autoinst/openQA/pull/2932 okurz, 2020-04-21 18:49
Screenshot_20200511_133037.png (27.4 KB) Screenshot_20200511_133037.png okurz, 2020-05-11 11:39
9775

Subtasks

action #25814: load job page, e.g. test details, only on demandResolvedmkittler

action #32611: job details in browser windows do not automatically jump from "assigned" to "running" when they start - take 2Resolvedmkittler

action #66622: Live view - 'Autoscroll log' check box has no effect anymoreResolvedmkittler

History

#1 Updated by mkittler over 1 year ago

  • Parent task set to #34357

#2 Updated by mkittler over 1 year ago

  • Due date set to 2020-04-07

due to changes in a related task

#3 Updated by okurz over 1 year ago

  • Subject changed from Revamp test details page to improve loading times and prevent timeouts to [epic] Revamp test details page to improve loading times and prevent timeouts
  • Status changed from New to Blocked
  • Assignee set to okurz

it's a nice collection of ideas but no defined end criteria so I think it makes sense to declare this as an epic, also because there is a subtask. So I will also assign to myself and track as blocked.

#4 Updated by mkittler over 1 year ago

  • Assignee changed from okurz to mkittler

The end criteria I had in mind is that all subtasks are resolved.

Note that #32611 is also a subtask. (Assigning the relation doesn't work.) With the new Redmine version I was now able to add the subtask.

#5 Updated by okurz over 1 year ago

I recorded videos how it currently looks like after https://github.com/os-autoinst/openQA/pull/2932 . There is still a small step visible between "running" and "uploading" when the screenshots are rendered again even though previously they were all already visible. And in "uploading" to "passed" the same happens. However it is not very likely to be noticed by users and also doesn't feel harmful or annoying. We can keep the current state as is.

#6 Updated by okurz over 1 year ago

9775

mkittler I observed that tests nicely went from running to passed but the info box color isn't updated, see attached
Screenshot_20200511_133037.png

#7 Updated by mkittler over 1 year ago

  • Status changed from Blocked to In Progress

I've already fixed a few regressions and created another PR to fix two more bugs: https://progress.opensuse.org/issues/65402

I guess when this PR has been merged the issue can be considered resolved.


okurz I've just noticed the problem as well. The new PR should fix it.

#8 Updated by cdywan over 1 year ago

  • Target version set to Current Sprint

#9 Updated by mkittler over 1 year ago

  • Status changed from In Progress to Resolved
  • Target version deleted (Current Sprint)

The last PR has been deployed on o3 and seems to work.

#10 Updated by szarate 12 months ago

  • Tracker changed from action to coordination
  • Difficulty deleted (hard)

Also available in: Atom PDF