action #180851
opencoordination #154768: [saga][epic][ux] State-of-art user experience for openQA
coordination #157345: [epic] Improved test reviewer user experience
Better sorting jobs in parent_group_overview by_groups
0%
Description
Problem Description¶
The parent_group_overview provides limit_builds getting the latest n jobs id which uses to render both the by_builds and by_groups interfaces. Problem is that in this situation
the data do not include the latest job ids per group, ending up to display groups without its latest builds. A by_group can show a job 2 months old from first group but not a 1 week old from group B
Reproduce¶
See screenshots
https://github.com/os-autoinst/openQA/issues/6309
https://github.com/os-autoinst/openQA/pull/6343#issuecomment-2776338075
Further Analysis - current functionality¶
The limit_builds and the process of the resultset is coming from compute_build_results.
The query always fetches the first 400 job ID by default and looks like this
SELECT me.VERSION, me.BUILD, MAX(id) AS lasted_job
FROM jobs me
WHERE (group_id IN ('123', '121', '122'))
GROUP BY VERSION, BUILD
ORDER BY lasted_job DESC
LIMIT 400;
compute_build_results then will loop for each build until it reaches the limit_builds. This means that it will never check some builds if the first item of the list have jobs==limits
on group_builds_functionality_view.html.ep
Already tried:
https://github.com/os-autoinst/openQA/pull/6343
https://github.com/os-autoinst/openQA/pull/6367
Both suffer from insufficient and tragic performance issues
Challenges:
- One API request for two templates. by_group uses the ResultSet of by_build. Maybe the logic should be separated
- not easy to solve the problem with sql. it can get complicated
- performance. The rendering is exponential slow as the limits grow. Maybe we can improve but for sure not make it worse
- It impacts also dashboard_build_results.html.ep
Feature in request and expectations:
by_builds interface doesnt change
by_groups interface presents latest job(s) in some limit, as would say chronological sorted. I mean that if the job date of a groupB is earlier that another job in groupA it should be shown in the dashboard.
Not break dashboard_build_results.html.ep
Updated by ybonatakis 5 days ago
- Related to action #179296: [beginner] View: Parent group with limit confusing size:S added
Updated by ybonatakis 5 days ago
livdywan wrote in #note-2:
I assume improved phrasing is helpful for now, and the performance impact will need more long-term planning so it shouldn't go on the backlog.
just a comment on the description and the performance reference. it is not about improving the performance. I just want to emphasise requirements and challenges of the implementation, we can approach feature in tecnical matters and not general discussion. I want to believe that this approach helps in the estimation. Maybe it is too much. I dont know how it looks like to others but this is an attempt to see tickets which can actually discuss and give clues in estimation.
Updated by rclobus 5 days ago
Hi,
I've slightly modified the SQL query mentioned above. Now you have only 2 parameters: parent_group_number (1) and limit (50):
select jobs.version, jobs.build, max(id) as lasted_job
from jobs
where group_id in (select id from job_groups where parent_id=1)
group by version, build
order by lasted_job desc limit 50;
If this data would be further processed, it should not matter whether it is (in the post-processing code) regrouped by group or build. It contains the desired number of results and contains the most recent ones. If for a specific 'version' no builds are listed, then that version has too old builds for the given limit.
I don't know whether the current code already does something similar, but by sending the limit at this stage, there would be no severe performance penalty (assuming proper indexes etc.) due to data that is thrown away in the post-processing code.