Project

General

Profile

Actions
Copy link

coordination #65402

closed

coordination #34357: [epic] Improve openQA performance

[epic] Revamp test details page to improve loading times and prevent timeouts

Added by mkittler over 4 years ago. Updated about 4 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
mkittler
Category:
Feature requests
Target version:
-
Start date:
2017-08-25
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)

Description

problem

The test details page loads very slow for big results and easily runs into timeouts.

suggestions

  • By default we should only load content synchronously that comes from the database, everything loading from filesystem should be done asynchronously or "on demand", either later by requests from the initial page loaded or by user actions, e.g. when clicking on tabs or buttons.
  • Refreshing the entire page (currently done for scheduled/assigned/setup/uploaded jobs) should be avoided.
  • The query for results should not only be done asynchronously but also be splitted to prevent timeouts (similarly like modules are currently loaded on-by-one while a test is running).
  • Running jobs should not be treated completely different from other jobs (like it is done now).
    • The only thing which can be avoided when a job is done is the check for status changes.

notes

  • Of course loading from "fast" (expensive) storage like SSD or NVMe can help but shouldn't be relied upon.
  • So far the table for "External results" is lot "live updated" while the job is running and only refreshed once the job is done. For reducing the immediate effort I would not change this for now.
  • So far we are polling for the job state. This could be improved by using web sockets but maybe that would be over-engineering.
  • Avoiding refreshing the entire page means that additional HTML elements (e.g. for the live view) need to be displayed via JavaScript.
    • It would be possible to render these HTML fragments on the server and request and insert them as needed in the client.
    • It would be possible to render HTML elements for all future job states from the beginning and hide them until a certain state is reached.
  • All points mentioned here and in the sub tickets should be considered together as everything is related. Maybe improvements can nevertheless be done in steps (which raises the question where to start).

Also see https://progress.opensuse.org/issues/25814#note-15 and https://progress.opensuse.org/issues/32611#note-8

Files

Download all files
running_to_uploading.mkv (59.7 KB) running_to_uploading.mkv video recording of state transition in browser from running to uploading after https://github.com/os-autoinst/openQA/pull/2932 okurz, 2020-04-21 18:48
uploading_to_passed.mkv (51.8 KB) uploading_to_passed.mkv video recording of state transition in browser from uploading to passed after https://github.com/os-autoinst/openQA/pull/2932 okurz, 2020-04-21 18:49
Screenshot_20200511_133037.png (27.4 KB) Screenshot_20200511_133037.png okurz, 2020-05-11 11:39

Subtasks 3 (0 open — 3 closed)

action #25814: load job page, e.g. test details, only on demandResolvedmkittler2020-04-07

Actions
action #32611: job details in browser windows do not automatically jump from "assigned" to "running" when they start - take 2Resolvedmkittler2017-08-25

Actions
action #66622: Live view - 'Autoscroll log' check box has no effect anymoreResolvedmkittler2020-05-07

Actions
Actions
Copy link
 #1

Updated by mkittler over 4 years ago

  • Parent task set to #34357
Actions
Copy link
 #2

Updated by mkittler over 4 years ago

  • Due date set to 2020-04-07

due to changes in a related task

Actions
Copy link
 #3

Updated by okurz over 4 years ago

  • Subject changed from Revamp test details page to improve loading times and prevent timeouts to [epic] Revamp test details page to improve loading times and prevent timeouts
  • Status changed from New to Blocked
  • Assignee set to okurz

it's a nice collection of ideas but no defined end criteria so I think it makes sense to declare this as an epic, also because there is a subtask. So I will also assign to myself and track as blocked.

Actions
Copy link
 #4

Updated by mkittler over 4 years ago

  • Assignee changed from okurz to mkittler

The end criteria I had in mind is that all subtasks are resolved.

Note that #32611 is also a subtask. (Assigning the relation doesn't work.) With the new Redmine version I was now able to add the subtask.

Actions
Copy link Download all files
 #5

Updated by okurz over 4 years ago

I recorded videos how it currently looks like after https://github.com/os-autoinst/openQA/pull/2932 . There is still a small step visible between "running" and "uploading" when the screenshots are rendered again even though previously they were all already visible. And in "uploading" to "passed" the same happens. However it is not very likely to be noticed by users and also doesn't feel harmful or annoying. We can keep the current state as is.

Actions
Copy link
 #6

Updated by okurz over 4 years ago

@mkittler I observed that tests nicely went from running to passed but the info box color isn't updated, see attached
Screenshot_20200511_133037.png

Actions
Copy link
 #7

Updated by mkittler over 4 years ago

  • Status changed from Blocked to In Progress

I've already fixed a few regressions and created another PR to fix two more bugs: https://progress.opensuse.org/issues/65402

I guess when this PR has been merged the issue can be considered resolved.

@okurz I've just noticed the problem as well. The new PR should fix it.

Actions
Copy link
 #8

Updated by livdywan over 4 years ago

  • Target version set to Current Sprint
Actions
Copy link
 #9

Updated by mkittler over 4 years ago

  • Status changed from In Progress to Resolved
  • Target version deleted (Current Sprint)

The last PR has been deployed on o3 and seems to work.

Actions
Copy link
 #10

Updated by szarate about 4 years ago

  • Tracker changed from action to coordination
  • Difficulty deleted (hard)
Actions
Copy link

Also available in: Atom PDF