Project

General

Profile

Actions

action #38024

closed

[functional][y] Add ability to search for a module occurrences in all the tests of job group

Added by oorlov over 5 years ago. Updated about 5 years ago.

Status:
Resolved
Priority:
Low
Assignee:
Category:
Feature requests
Target version:
SUSE QA - Milestone 22
Start date:
2018-06-29
Due date:
2019-01-29
% Done:

0%

Estimated time:
8.00 h

Description

As an openQA operator,
I want to search for a module occurrences in all job groups (or at least in one job group),
So that I'll be able to know all the tests that contain the certain module.

We break quite some jobs when introducing changes to the test modules as we don't know all the cases where it is used.

I've discussed the feature with @mkittler and he has an idea on how to implement this.

Acceptance criteria

  1. There is documented way to search for a jobs which use given test module.

Related issues 1 (0 open1 closed)

Related to openQA Project - action #34486: database of "test cases" or how to search for tests we have in openQAResolvedlivdywan2018-04-08

Actions
Actions #1

Updated by mkittler over 5 years ago

Likely easy to implement since we have this already for failed modules.

This WIP branch already allows to do this via the client: https://github.com/Martchus/openQA/tree/filter_by_module_name

eg. script/client jobs build=20180521 modules=welcome

Actions #2

Updated by oorlov over 5 years ago

  • Project changed from openQA Project to openQA Tests
  • Subject changed from Add ability to search for a module occurrences in all the tests of job group to [functional][y] Add ability to search for a module occurrences in all the tests of job group
  • Due date set to 2018-12-04
  • Category changed from Feature requests to Infrastructure
Actions #3

Updated by okurz over 5 years ago

Do you really think this is not a feature for openQA? Are you thinking about an external tool? Is this the same as #34486 ?

Actions #4

Updated by okurz over 5 years ago

  • Related to action #34486: database of "test cases" or how to search for tests we have in openQA added
Actions #5

Updated by okurz over 5 years ago

  • Target version set to Milestone 21
Actions #6

Updated by riafarov over 5 years ago

  • Description updated (diff)
  • Status changed from New to Workable
Actions #7

Updated by riafarov over 5 years ago

  • Project changed from openQA Tests to openQA Project
  • Category changed from Infrastructure to 124
Actions #8

Updated by oorlov over 5 years ago

@okurz said:

Do you really think this is not a feature for openQA? Are you thinking about an external tool? Is this the same as #34486 ?

Currently we have an ability to search for failed modules using Filter section on “Test result overview” Page.

This ticket is a request for a search by modules regardless of their status. So, this requires just another query to DB (which seems to be already done by @mkittler) and small changes in UI (like adding new text field or editing existing one).

#34486 seems more complicated and general, but I guess we can count this ticket as a part of the #34486.

Actions #9

Updated by okurz over 5 years ago

oorlov wrote:

#34486 seems more complicated and general

Yes, probably also because of the comments from different people to make it more complicated ;)

Actions #10

Updated by okurz over 5 years ago

https://github.com/okurz/scripts/blob/master/openqa-db_query_last_use_of_module can do it already as a "hacky external script", could be helpful as reference.

Actions #11

Updated by riafarov over 5 years ago

  • Estimated time set to 8.00 h
Actions #12

Updated by okurz over 5 years ago

  • Priority changed from Normal to Low
Actions #13

Updated by riafarov over 5 years ago

  • Due date changed from 2018-12-04 to 2018-12-18

We feel motivated to work on it.

Actions #14

Updated by riafarov over 5 years ago

  • Due date changed from 2018-12-18 to 2019-01-01
Actions #15

Updated by riafarov over 5 years ago

  • Due date changed from 2019-01-01 to 2019-01-15
Actions #16

Updated by riafarov over 5 years ago

  • Due date changed from 2019-01-15 to 2019-01-29
Actions #17

Updated by okurz about 5 years ago

  • Target version changed from Milestone 21 to Milestone 22

adjusting target version to sprint

Actions #18

Updated by oorlov about 5 years ago

  • Assignee set to oorlov
Actions #19

Updated by oorlov about 5 years ago

  • Status changed from Workable to In Progress
Actions #20

Updated by oorlov about 5 years ago

  • Status changed from In Progress to Feedback
Actions #21

Updated by okurz about 5 years ago

Your PR is nice but it only works as a filter starting from an existing test overview, right? So how would I be able to search "in all job groups" as your user story describes?

Actions #23

Updated by okurz about 5 years ago

Well, but wasn't the original problem that we break tests in other products, e.g. maintenance tests? The other job groups of the same products are less likely to show the test modules failing.

Actions #24

Updated by okurz about 5 years ago

The feature can now be observed in production on e.g. https://openqa.opensuse.org/tests/overview?arch=&modules=keymap_or_locale&modules_result=failed&distri=kubic&distri=opensuse&version=Tumbleweed&build=20190121&groupid=1#

@oorlov @mkittler what I find confusing now from UX point of view is that we have check boxes for "Result" and a multi-selection field for "Module Result". Looks a bit inconsistent. Do you have ideas how to improve that?

Actions #25

Updated by oorlov about 5 years ago

  • Status changed from Feedback to Resolved

Even if the labels will require the change, I guess we can resolve the ticket, as it fulfills the initial requirement.

Actions #26

Updated by okurz about 5 years ago

Hey, have you seen my comment #38024#note-23 or missed it? We can also talk about it personally if you prefer.

Actions #27

Updated by oorlov about 5 years ago

Oliver, sorry for late reply. Didn't see the comment above.

Probably you just misunderstood on how the search by modules is implemented.

It searches through any amount of job groups. You just need to stay on the page where all the needed groups are shown.

For example, you may search in only "Functional: Server" job group for Sle12 build 0120: https://openqa.suse.de/tests/overview?distri=sle&version=12-SP5&build=0120&groupid=139

Or you can even search among all the Sle15 and Sle12 groups. Just go to the page where all the jobs of the groups are listed: https://openqa.suse.de/tests/overview?distri=sle&version=12-SP5&build=0120&version=15-SP1&build=159.1

So it's up to you where to call the filter.

The only thing is the database (or queries) are slow and on huge amount of data there might be Gateway Error. But it is another thing and it should be done in scope of another tasks.

Actions #28

Updated by okurz about 5 years ago

oorlov wrote:

Oliver, sorry for late reply. Didn't see the comment above.

Well, thanks for finding it. Lately some people did not see notifications and have not realized comments. You did :)

Probably you just misunderstood on how the search by modules is implemented.

Well, I doubt I misunderstood in this case, also as I have implemented the multi-job-group view :)

You have added a nice and helpful feature, I am not questioning that but I wonder how we can go on, as in #38024#note-23 :

Well, but wasn't the original problem that we break tests in other products, e.g. maintenance tests? The other job groups of the same products are less likely to show the test modules failing.

Isn't this what also happened in the past and looking in multiple job groups of the same build will not help?

It would be nice if

https://openqa.opensuse.org/tests/overview?modules=keymap_or_locale&result=failed

would help us to return all the failed jobs where this module was scheduled and it kinda does but hardly how people would expect it to work as it can show jobs that are about 10 months old as the job resolution depends on the order, do we look for "last failed" first, then for the module and then for the build or in different order?

In any case, isn't your PR https://github.com/os-autoinst/openQA/pull/1993 still related to this ticket? Maybe it would be good to have a follow-up ticket? I would be happy to follow-on with you to find the next best step.

Actions

Also available in: Atom PDF