https://progress.opensuse.org/https://progress.opensuse.org/themes/openSUSE/favicon/favicon.ico?15829177842018-08-06T06:13:17ZopenSUSE Project Management ToolopenQA Project - action #35389: Improve performance of /admin/assets https://progress.opensuse.org/issues/35389?journal_id=1402882018-08-06T06:13:17Zmitiao
<ul><li><strong>Assignee</strong> set to <i>mitiao</i></li></ul><p>this page is getting bigger and browser always lose response during loading.<br>
investigate a better solution is needed. </p>
openQA Project - action #35389: Improve performance of /admin/assets https://progress.opensuse.org/issues/35389?journal_id=1430992018-08-16T09:31:45Zszarate
<ul><li><strong>Assignee</strong> deleted (<del><i>mitiao</i></del>)</li><li><strong>Priority</strong> changed from <i>Normal</i> to <i>High</i></li></ul> openQA Project - action #35389: Improve performance of /admin/assets https://progress.opensuse.org/issues/35389?journal_id=1431142018-08-16T09:59:26Zmkittlermarius.kittler@suse.com
<ul><li><strong>Assignee</strong> set to <i>mkittler</i></li><li><strong>Priority</strong> changed from <i>High</i> to <i>Normal</i></li></ul><blockquote>
<p>I don't know what's best</p>
</blockquote>
<p>I would not make it a server-side data table because those are also not so nice to use (loading time after <em>every</em> action like selecting a page, filtering, ...). Better just load the whole data as JSON via a separate route. The initial loading time would be longer but the table more usable. (Compared to what we have now the initial loading time would still decrease and the browser would stay responsible.)</p>
openQA Project - action #35389: Improve performance of /admin/assets https://progress.opensuse.org/issues/35389?journal_id=1432042018-08-16T14:06:52Zmkittlermarius.kittler@suse.com
<ul></ul><p>According to my tests with a database dump from OSD using an extra JSON route would only improve the responsiveness of the page but the initial loading would still take some time.</p>
<p>The bottleneck is - according to nytprof - the database query and also the sorting/prioritizing of the results which is currently done in Perl code. Without sorting it takes only half of the time.</p>
<p>I can try to optimize that. Especially the sorting should be done by the database. However, this will be difficult because the criteria for the order is spread over multiple tables. But I'll give it a try.</p>
<p>Note that fast sorting would also be a prerequisite for making the table fully server-side because in this case the entire sorting must be done by the server for <em>each</em> user interaction with the table.</p>
openQA Project - action #35389: Improve performance of /admin/assets https://progress.opensuse.org/issues/35389?journal_id=1442722018-08-21T12:27:53Zmkittlermarius.kittler@suse.com
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>In Progress</i></li></ul><p>PR: <a href="https://github.com/os-autoinst/openQA/pull/1750" class="external">https://github.com/os-autoinst/openQA/pull/1750</a></p>
<p>The computation of the data is now faster and there are more tests. However, the table is still rendered as HTML elements and there's no caching. So the next step would be serving the data as JSON and implement caching. The cache would be refreshed when <code>limit_assets</code> runs or when the user passes some parameter to force the refresh.</p>
<p>Serving JSON via an additional route will be a bit tricky, though. This is because the 'Assets by group' list relies on the same data but can not easily be made up from some JSON data served by another route like it is possible with the DataTable.</p>
openQA Project - action #35389: Improve performance of /admin/assets https://progress.opensuse.org/issues/35389?journal_id=1542112018-10-07T08:02:44Zcoolocoolo@suse.com
<ul><li><strong>Target version</strong> changed from <i>Ready</i> to <i>Current Sprint</i></li><li><strong>Difficulty</strong> changed from <i>medium</i> to <i>hard</i></li></ul> openQA Project - action #35389: Improve performance of /admin/assets https://progress.opensuse.org/issues/35389?journal_id=1548712018-10-09T14:06:42Zmkittlermarius.kittler@suse.com
<ul></ul><p>PR: <a href="https://github.com/os-autoinst/openQA/pull/1821" class="external">https://github.com/os-autoinst/openQA/pull/1821</a></p>
<p>The static/cache approach works basically. Locally it takes now 10 seconds to load (including client side rendering). The code and performance could still be improved, but at least the table is usable again.</p>
<p>A button to re-trigger the minion job is still missing. Right now it is only possible to use the query parameter <code>force_refresh=1</code> but I suppose running the asset cleanup/status algorithm only via minion would be the better option.</p>
<p>It is still possible to delete assets within the table. However, the change is not immediately visible anymore due to the caching. Not sure how 'bad' this is (the cleanup is automated anyways).</p>
openQA Project - action #35389: Improve performance of /admin/assets https://progress.opensuse.org/issues/35389?journal_id=1574212018-10-16T07:58:51Zmkittlermarius.kittler@suse.com
<ul><li><strong>Status</strong> changed from <i>In Progress</i> to <i>Feedback</i></li></ul><p>Let's see how fast is loads in production. If it is at least usable again I would close the issue. Making it server-side would be a further enhancement.</p>
openQA Project - action #35389: Improve performance of /admin/assets https://progress.opensuse.org/issues/35389?journal_id=1584232018-10-18T08:14:53Zmkittlermarius.kittler@suse.com
<ul><li><strong>Status</strong> changed from <i>Feedback</i> to <i>Resolved</i></li></ul><p>It takes now < 25 seconds to load in production. Since the page isn't accessed that often I suppose that's ok (and it is already a huge improvement to how slow it was before).</p>
<p>So I'm closing the ticket. When further improvements are required (see comments on PR), just reopen or create a new ticket.</p>
openQA Project - action #35389: Improve performance of /admin/assets https://progress.opensuse.org/issues/35389?journal_id=1689832018-11-27T09:36:10Zcoolocoolo@suse.com
<ul><li><strong>Target version</strong> changed from <i>Current Sprint</i> to <i>Done</i></li></ul>