Project

General

Profile

action #98258

QA - coordination #91646: [saga][epic] SUSE Maintenance QA workflows with fully automated testing, approval and release

coordination #91467: [epic] Surface openQA failures per squad in a single place

No results on /tests/overview w/o build

Added by cdywan 3 months ago. Updated 2 months ago.

Status:
Resolved
Priority:
Low
Assignee:
Category:
Concrete Bugs
Target version:
Start date:
2021-07-22
Due date:
% Done:

0%

Estimated time:
Difficulty:

Description

Motivation

Acceptance criteria

  • AC1: /tests/overview returns results or surfaces an error

Suggestions

  • Improve parameter validation - why do we get no results if a parameter is not specified?
  • Make the build number optional
  • Show an error like an infobox or a JSON error in the API route

Workaround

Add build manually or follow from links where the build is already included.


Related issues

Copied from openQA Project - action #96058: [spike] Filter test results on /tests or /tests/overview by regex match in modules size:MResolved2021-07-22

History

#1 Updated by cdywan 3 months ago

  • Copied from action #96058: [spike] Filter test results on /tests or /tests/overview by regex match in modules size:M added

#2 Updated by VANASTASIADIS 3 months ago

  • Target version set to Ready

#3 Updated by okurz 3 months ago

  • Description updated (diff)
  • Category changed from Concrete Bugs to Feature requests
  • Target version changed from Ready to future

Not a regression. I assume this was always like this. Also, this can only be hit by "expert users" or when following manually crafted URLs hence adding "Workaround" section and removing from current backlog as suggested in weekly 2021-09-10. Maybe we will add it back soon after we collected a bit more feedback about the "module_re" feature and see more people wanting to use it.

#4 Updated by okurz 3 months ago

  • Target version changed from future to Ready

#5 Updated by asmorodskyi 3 months ago

okurz wrote:

Hm, I thought no one else would be able to hit that but of course asmorodskyi needed to dig up the dirt and hit it as well :D https://suse.slack.com/archives/C02CANHLANP/p1631636724242400

So e.g. something like https://openqa.suse.de/tests/overview?modules=ssh_interactive_init&modules_result=failed&distri=sle&version=15-SP1&build=20210914-1&groupid=349# works but https://openqa.suse.de/tests/overview?modules=ssh_interactive_init&modules_result=failed&distri=sle&version=15-SP1&groupid=349# does not. Now adding to backlog.

looks like you missing something in my use case . I get this problem after hitting "Show latest results of all the groups within parent group" ( not sure what correct name for this feature ). so my URL before applying any filter look like https://openqa.suse.de/tests/overview?groupid=370&groupid=348&groupid=349&groupid=351&groupid=356&groupid=360&groupid=285&groupid=276 .

And page description saying "Overall Summary of Maintenance: Public Cloud 12-SP4 Updates, Latest - HA + SAP, Maintenance: 15-SP2 Staging Images, Maintenance: Public Cloud 15-SP3 Updates, PC Tools Image, Maintenance: Public Cloud 15-SP1 Updates, Maintenance: Public Cloud 15-SP2 Updates, Maintenance: Public Cloud 12-SP5 Updates build 0021,0081,20210915,20210915-1 "

So simply adding &build won't help . you need to add multiply builds and even this will leave open a question how filter suppose to figure out which build should be applied to which group

#6 Updated by okurz 3 months ago

asmorodskyi wrote:

looks like you missing something in my use case . I get this problem after hitting "Show latest results of all the groups within parent group"
( not sure what correct name for this feature )

I would call it "multi-job-group test overview"

. so my URL before applying any filter look like https://openqa.suse.de/tests/overview?groupid=370&groupid=348&groupid=349&groupid=351&groupid=356&groupid=360&groupid=285&groupid=276 .

And page description saying "Overall Summary of Maintenance: Public Cloud 12-SP4 Updates, Latest - HA + SAP, Maintenance: 15-SP2 Staging Images, Maintenance: Public Cloud 15-SP3 Updates, PC Tools Image, Maintenance: Public Cloud 15-SP1 Updates, Maintenance: Public Cloud 15-SP2 Updates, Maintenance: Public Cloud 12-SP5 Updates build 0021,0081,20210915,20210915-1 "

So simply adding &build won't help . you need to add multiply builds and even this will leave open a question how filter suppose to figure out which build should be applied to which group

Correct. I guess I have seen in some systems applying multiple instances of the same parameter where each has to map to another parameter. Maybe this works based either on the order or there is some numbering, e.g. &group[0]=123&group[1]=234&build[0]=42.1&build[1]=42.3. This is out of scope for the current ticket. If you would like record this feature wish please add it to https://progress.opensuse.org/issues/65271 # 65271 or in another separate ticket

#7 Updated by osukup 2 months ago

Group and Groupid in search query is handled by this code:

    # add group query params to search args
    # (By 'every_param' we make sure to use multiple values for groupid and
    # group at the same time as a logical or, i.e. all specified groups are
    # returned.)
    my $schema = $c->schema;
    my @groups;
    if ($v->is_valid('groupid') || $v->is_valid('group')) {
        my @group_id_search   = map { {id   => $_} } @{$v->every_param('groupid')};
        my @group_name_search = map { {name => $_} } @{$v->every_param('group')};
        my @search_terms      = (@group_id_search, @group_name_search);
        @groups = $schema->resultset('JobGroups')->search(\@search_terms)->all;
    }

so it should find all groups for given build/distri etc

slightly different situation is if build isn't specified , code check latest build for each group, and there is probadly wrong log message for situation with multiple groups

see: $search_args{build} = [sort keys %builds]; vs $c->app->log->info('More than one group but no build specified, selecting build of first group');

but in Schema::ResultSet::Jobs prepare_complex_query effectively usesbuild` in same way as version or flavor / distri ?

this code:

        for my $key (qw(build distri version flavor arch test machine)) {
            push(@conds, {"me." . uc($key) => $args->{$key}}) if $args->{$key};
        }

for build probably better variant is :

push(@conds, {"me.BUILD" => {-in =>args{build}});

after some investigation ... wheen is groupid defined .. parameter build is computed in very undetermistic way and in many cases returns empty array which leads to search for empty builds?

for example this query: http://quasar.suse.cz/tests/overview?arch=&flavor=&machine=&test=&modules=isosize&module_re=&modules_result=failed&distri=opensuse&version=tumbleweed&groupid=1#

which results to:

Sep 22 16:47:28 quasar openqa-webui-daemon[32536]: [info] Conditions: {
Sep 22 16:47:28 quasar openqa-webui-daemon[32536]:   'modules.name' => {
Sep 22 16:47:28 quasar openqa-webui-daemon[32536]:                       '-in' => [
Sep 22 16:47:28 quasar openqa-webui-daemon[32536]:                                  'isosize'
Sep 22 16:47:28 quasar openqa-webui-daemon[32536]:                                ]
Sep 22 16:47:28 quasar openqa-webui-daemon[32536]:                     }
Sep 22 16:47:28 quasar openqa-webui-daemon[32536]: }
Sep 22 16:47:28 quasar openqa-webui-daemon[32536]: {
Sep 22 16:47:28 quasar openqa-webui-daemon[32536]:   'modules.result' => {
Sep 22 16:47:28 quasar openqa-webui-daemon[32536]:                         '-in' => [
Sep 22 16:47:28 quasar openqa-webui-daemon[32536]:                                    'failed'
Sep 22 16:47:28 quasar openqa-webui-daemon[32536]:                                  ]
Sep 22 16:47:28 quasar openqa-webui-daemon[32536]:                       }
Sep 22 16:47:28 quasar openqa-webui-daemon[32536]: }
Sep 22 16:47:28 quasar openqa-webui-daemon[32536]: {
Sep 22 16:47:28 quasar openqa-webui-daemon[32536]:   'me.clone_id' => undef
Sep 22 16:47:28 quasar openqa-webui-daemon[32536]: }
Sep 22 16:47:28 quasar openqa-webui-daemon[32536]: {
Sep 22 16:47:28 quasar openqa-webui-daemon[32536]:   'me.group_id' => {
Sep 22 16:47:28 quasar openqa-webui-daemon[32536]:                      '-in' => [
Sep 22 16:47:28 quasar openqa-webui-daemon[32536]:                                 1
Sep 22 16:47:28 quasar openqa-webui-daemon[32536]:                               ]
Sep 22 16:47:28 quasar openqa-webui-daemon[32536]:                    }
Sep 22 16:47:28 quasar openqa-webui-daemon[32536]: }
Sep 22 16:47:28 quasar openqa-webui-daemon[32536]: {
Sep 22 16:47:28 quasar openqa-webui-daemon[32536]:   'me.BUILD' => []
Sep 22 16:47:28 quasar openqa-webui-daemon[32536]: }
Sep 22 16:47:28 quasar openqa-webui-daemon[32536]: {
Sep 22 16:47:28 quasar openqa-webui-daemon[32536]:   'me.DISTRI' => 'opensuse'
Sep 22 16:47:28 quasar openqa-webui-daemon[32536]: }
Sep 22 16:47:28 quasar openqa-webui-daemon[32536]: {
Sep 22 16:47:28 quasar openqa-webui-daemon[32536]:   'me.VERSION' => 'tumbleweed'
Sep 22 16:47:28 quasar openqa-webui-daemon[32536]: }

#8 Updated by osukup 2 months ago

  • Assignee set to osukup

#9 Updated by osukup 2 months ago

  • Category changed from Feature requests to Concrete Bugs
  • Status changed from New to Feedback

#10 Updated by okurz 2 months ago

  • Status changed from Feedback to Resolved

https://openqa.opensuse.org/tests/overview?groupid=24&module_re=openqa looks good now. To me this looks actually fixed. Thank you

Also available in: Atom PDF