Project

General

Profile

action #96058

QA - coordination #91646: [saga][epic] SUSE Maintenance QA workflows with fully automated testing, approval and release

coordination #91467: [epic] Surface openQA failures per squad in a single place

[spike] Filter test results on /tests or /tests/overview by regex match in modules size:M

Added by okurz 4 months ago. Updated 3 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Feature requests
Target version:
Start date:
2021-07-22
Due date:
% Done:

0%

Estimated time:
Difficulty:

Description

Motivation

As a squad member with responsibility areas defined by a field Maintainer: my-team@suse.de in a test module I would like to filter by a search term searching the test modules on /tests or /tests/overview to only see scenarios which fail in modules that list my team as maintainer

Acceptance criteria

  • AC1: /tests or /tests/overview can be configured to show only test results matching a specified regex search term in test module source code

Suggestions

  • Add a query parameter to /tests or /tests/overview or both, e.g. "module_content", that if specified calls git grep -l in the specified test distribution and returns only the jobs that run/fail in job modules that match the git grep call
  • Look how we call "git" in other cases, e.g. for the "investigation" details

Out of scope

  • Support for non-git test distributions
  • Support for multiple test distributions, just rely on "distri" being specified for now

Related issues

Copied from openQA Project - action #95878: Add filter on /tests/overview for search term in test suite descriptionNew2021-07-22

Copied to openQA Project - action #98258: No results on /tests/overview w/o buildResolved2021-07-22

Copied to openQA Project - action #98445: improve description for "Test module" UI element as followup to #96058Resolved

Copied to openQA Project - action #98460: Filter actual test results on /tests or /tests/overview by regex match in modulesResolved

History

#1 Updated by okurz 4 months ago

  • Copied from action #95878: Add filter on /tests/overview for search term in test suite description added

#2 Updated by okurz 4 months ago

  • Subject changed from [spike] Filter test results on /tests or /tests/overview by regex match in failed modules to [spike] Filter test results on /tests or /tests/overview by regex match in failed modules size:M
  • Description updated (diff)
  • Status changed from New to Workable

#3 Updated by osukup 4 months ago

  • Assignee set to osukup

I'll try

#4 Updated by osukup 4 months ago

Support for multiple test distributions, just rely on "distri" being specified for now ... in tests_overview and tests is possible to query without specific distri and get results for multiple distri set.

standard grep over testdirectory is too slow .. maybe we can use something like The Silver Searcher / Ripgrep ?

#5 Updated by cdywan 4 months ago

osukup wrote:

Support for multiple test distributions, just rely on "distri" being specified for now
in tests_overview and tests is possible to query without specific distri and get results for multiple distri set.

Not sure if that is a guess or based on reading the code. I'd say when writing tests for now you should assume distri is set. And see later if it's trivial to make more generic.

standard grep over testdirectory is too slow .. maybe we can use something like The Silver Searcher / Ripgrep ?

rg could be faster than grep because it skips a lot in a big test folder, but it would also be a new dep. And it's out of scope for this ticket anyway.

#6 Updated by osukup 4 months ago

cdywan wrote:

osukup wrote:

Support for multiple test distributions, just rely on "distri" being specified for now
in tests_overview and tests is possible to query without specific distri and get results for multiple distri set.

Not sure if that is a guess or based on reading the code. I'd say when writing tests for now you should assume distri is set. And see later if it's trivial to make more generic.

based on reading code :D , It's much harder made it working with distri than over testdir.

standard grep over testdirectory is too slow .. maybe we can use something like The Silver Searcher / Ripgrep ?

rg could be faster than grep because it skips a lot in a big test folder, but it would also be a new dep. And it's out of scope for this ticket anyway.

rg is in most cases one order faster than grep

#7 Updated by osukup 4 months ago

my Idea ->

1) add a parameter to overview
2) if is used --> grep files in tests directory
3) extract filenames without extension
4) fill these filenames to 'module' search logic search_args{module}
5) profit ...

#8 Updated by okurz 4 months ago

osukup wrote:

my Idea ->

1) add a parameter to overview
2) if is used --> grep files in tests directory
3) extract filenames without extension
4) fill these filenames to 'module' search logic search_args{module}
5) profit ...

sounds good! In the ticket description we suggested to use git grep -l which should be way faster than grep or ag of course assuming that we can identify the right test distribution. If this is too hard just iterate over all directories, e.g.

for i in $(find /var/lib/openqa/share/tests -maxdepth 1 -type d); do test -d $i/.git && git -C $i grep -l kontact ; done

or a perl equivalent of the same :)

#9 Updated by osukup 4 months ago

okurz wrote:

osukup wrote:

my Idea ->

1) add a parameter to overview
2) if is used --> grep files in tests directory
3) extract filenames without extension
4) fill these filenames to 'module' search logic search_args{module}
5) profit ...

sounds good! In the ticket description, we suggested to use git grep -l which should be way faster than grep or ag of course assuming that we can identify the right test distribution. If this is too hard just iterate over all directories, e.g.

git grep isn't much faster than standard grep ( his biggest benefit it limits files for search ), on other side ripgrep is really much faster and the command is pretty simple:

my $res = run_cmd_with_log_return_error(['rg', '-l', $regexp, $dir]);

for i in $(find /var/lib/openqa/share/tests -maxdepth 1 -type d); do test -d $i/.git && git -C $i grep -l kontact ; done

or a perl equivalent of the same :)

#10 Updated by osukup 4 months ago

osukup wrote:

okurz wrote:

osukup wrote:

my Idea ->

1) add a parameter to overview
2) if is used --> grep files in tests directory
3) extract filenames without extension
4) fill these filenames to 'module' search logic search_args{module}
5) profit ...

sounds good! In the ticket description, we suggested to use git grep -l which should be way faster than grep or ag of course assuming that we can identify the right test distribution. If this is too hard just iterate over all directories, e.g.

git grep isn't much faster than standard grep ( his biggest benefit it limits files for search ), on other side ripgrep is really much faster and the command is pretty simple:

my $res = run_cmd_with_log_return_error(['rg', '-l', $regexp, $dir]);

for i in $(find /var/lib/openqa/share/tests -maxdepth 1 -type d); do test -d $i/.git && git -C $i grep -l kontact ; done

or a perl equivalent of the same :)

on other side this is oppurtinity to look at File::Utils etc modules :D

#11 Updated by osukup 4 months ago

after going through some File:: perl modules I found there isn't module providing something like python's pathlib :(

so -> now to test okurz oneliner

#12 Updated by osukup 3 months ago

  • Status changed from Workable to In Progress

#13 Updated by openqa_review 3 months ago

  • Due date set to 2021-09-07

Setting due date based on mean cycle time of SUSE QE Tools

#14 Updated by osukup 3 months ago

after testing oneliner with IPC::Run, its impossible use it with run_cmd_with_log_return_error

any solution with git grep -l needs:

1) something traverse testcasedir and find all subdirs with .git folder (there isn't any nice module for this in Perl :( )
2) then run git grep on this dirs, merge results and they use results

on another side -> using ripgrep brings one call of run_cmd.. on testcasedir with results from all subdirs. --> pretty simple implementation on price one new dep (and only one because ripgrep is literary static binary)

#16 Updated by okurz 3 months ago

how about sh -c 'for i in $(find /var/lib/openqa/share/tests -maxdepth 1 -type d); do test -d $i/.git && git -C $i grep -l kontact ; done'?

#17 Updated by cdywan 3 months ago

We already have code searching files and file contents in lib/OpenQA/WebAPI/Controller/API/V1/Search.pm.

#18 Updated by osukup 3 months ago

cdywan wrote:

We already have code searching files and file contents in lib/OpenQA/WebAPI/Controller/API/V1/Search.pm.

wau, thx.

used in last version of PR

#19 Updated by osukup 3 months ago

  • Subject changed from [spike] Filter test results on /tests or /tests/overview by regex match in failed modules size:M to [spike] Filter test results on /tests or /tests/overview by regex match in modules size:M

#20 Updated by osukup 3 months ago

  • Status changed from In Progress to Feedback

merged :D

#21 Updated by cdywan 3 months ago

osukup wrote:

merged :D

It seems like it's deployed on o3, but I don't see any results when trying it e.g. https://openqa.opensuse.org/tests/overview?groupid=24&module_re=openqa
Am I using it wrong?

#22 Updated by osukup 3 months ago

  • Status changed from Feedback to Resolved

cdywan wrote:

osukup wrote:

merged :D

It seems like it's deployed on o3, but I don't see any results when trying it e.g. https://openqa.opensuse.org/tests/overview?groupid=24&module_re=openqa
Am I using it wrong?

looks working for me? https://openqa.opensuse.org/tests/overview?arch=&flavor=&machine=&test=&modules=&module_re=openqa&distri=openqa&version=Tumbleweed&build=%3ATW.9034&groupid=24#

but with a more general query looks like o3 hits timeout

and it seems also deployed on osd https://openqa.suse.de/tests/overview?arch=&flavor=&machine=&test=&modules=&module_re=okurz&distri=sle&version=15-SP3&build=20210907-1&groupid=366#

#24 Updated by cdywan 3 months ago

osukup wrote:

cdywan wrote:

osukup wrote:

merged :D

It seems like it's deployed on o3, but I don't see any results when trying it e.g. https://openqa.opensuse.org/tests/overview?groupid=24&module_re=openqa
Am I using it wrong?

looks working for me? https://openqa.opensuse.org/tests/overview?arch=&flavor=&machine=&test=&modules=&module_re=openqa&distri=openqa&version=Tumbleweed&build=%3ATW.9034&groupid=24#

It seems like it breaks when no build is specified. https://openqa.opensuse.org/tests/overview?groupid=24&module_re=openqa&build=%3ATW.9034 works. That's probably unrelated then, filed #98258

#25 Updated by cdywan 3 months ago

  • Copied to action #98258: No results on /tests/overview w/o build added

#26 Updated by cdywan 3 months ago

  • Due date deleted (2021-09-07)

#27 Updated by okurz 3 months ago

  • Copied to action #98445: improve description for "Test module" UI element as followup to #96058 added

#28 Updated by okurz 3 months ago

  • Copied to action #98460: Filter actual test results on /tests or /tests/overview by regex match in modules added

Also available in: Atom PDF