QA » openQA Project

okurz wrote:

Your user story makes sense but I wonder why you wish for this to be present in dashboard.qam.suse.de? Links like for example https://openqa.suse.de/tests/overview?result=failed&modules=patch_and_reboot&modules_result=failed&distri=sle to show all jobs where a module "patch_and_reboot" fails in SLE tests for all versions, architectures, flavors already works.

Important part for me during a review is to have good overview. List like I described would show me the possible pattern of failures, while editing link would mean I have to start spotting it first and then confirming it by such links.

hurhaj wrote:

Important part for me during a review is to have good overview. List like I described would show me the possible pattern of failures, while editing link would mean I have to start spotting it first and then confirming it by such links.

I see. Well I am convinced that dashboard.qam.suse.de will never become a good tool to get statistics about openQA tests in the past. The focus is to show the present state and only that. All data within the dashboard is considered transient. How about https://maintenance-statistics.dyn.cloud.suse.de/dashboard/57-osd as a starting point showing "Which openQA test modules fail most often" as well as "which testsuites fail the most on OSD".

okurz wrote:

I see. Well I am convinced that dashboard.qam.suse.de will never become a good tool to get statistics about openQA tests in the past. The focus is to show the present state and only that. All data within the dashboard is considered transient. How about https://maintenance-statistics.dyn.cloud.suse.de/dashboard/57-osd as a starting point showing "Which openQA test modules fail most often" as well as "which testsuites fail the most on OSD".

But for the purposes of the review I am interested in present only. There is zero need for statistical data. With that, maybe...
** AC2: ** List shows only modules that have failed in the latest runs.

On the contrary, during review I am only interested in the tests that "are failing right now". If retry or next run passes, module should disappear from the list. To better illustrate, we currently have "Blocked" tab that groups by incident (maint. update) and shows related groups. What I want is to go one level deeper - to test modules in those groups, and list it by those.

And yes, I am aware that single incidents will be a problem again, as they don't really have next run. So they will need some special treatment again. My guess would be based on whether maint update is still active.

We discussed in jitsi and found some ideas that we could do. https://openqa.suse.de/tests/overview?result=failed&distri=sle&build=20171108-1 is a good starting point but we would need to compute the build number. This could be done in qem-dashboard to have a clickable link. As alternative our maintenance aggregate tests are commonly defined by having the word "Updates" in the flavor name but we have different flavors. So would be nice to have a wildcard filter like https://openqa.suse.de/tests/overview?distri=sle&flavor=*Updates&result=failed which would resolve to all matching flavors, e.g. AZURE-BYOS-Updates and SAP-DVD-Updates and Server-DVD-Updates.

Additionally the count of module appearance would help. E.g. currently on https://openqa.suse.de/tests/overview?distri=sle&build=20220125-1&result=failed we see 7 times "register_system", 2 times "Sysctl". So maybe a list at the bottom listing the numbers in a list, by descending value.

Hi @okurz. After revisiting this poo one year later, I have to admit I no longer see much use for this. Unless you believe it's still relevant from your point of view, I suggest rejecting.