Project

General

Profile

Actions

action #45890

closed

openQA Tests - coordination #36709: [functional][y][epic] Improve informing about test scenarios

[tools] testsuite description in the bug reporting template

Added by okurz over 5 years ago. Updated about 5 years ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
Feature requests
Target version:
Start date:
2019-01-09
Due date:
% Done:

0%

Estimated time:

Description

Acceptance criteria

  • AC1: Bugs created from the openQA webui include the testsuite description in the bug description pre-filled text

Suggestions


Related issues 1 (0 open1 closed)

Has duplicate openQA Project - action #46628: [tools] openQA create bug link creates bad textRejectedokurz2019-01-24

Actions
Actions #1

Updated by okurz over 5 years ago

  • Parent task set to #36709
Actions #2

Updated by JERiveraMoya over 5 years ago

Taking a look how the db is configured to access the field.

Actions #3

Updated by JERiveraMoya over 5 years ago

  • Assignee set to JERiveraMoya
Actions #4

Updated by JERiveraMoya over 5 years ago

  • Status changed from Workable to Feedback
Actions #5

Updated by riafarov over 5 years ago

Good job!

Actions #6

Updated by JERiveraMoya over 5 years ago

Thanks! :)
Waiting for openQA deployment for verification.

Actions #7

Updated by riafarov over 5 years ago

Waiting for the deployment.

Actions #8

Updated by okurz over 5 years ago

  • Status changed from Feedback to Resolved
[16/01/2019 11:37:36] <okurz> updated o3 webui
[16/01/2019 11:38:06] <okurz> jrivera: you can check the template on o3 now, works for me :)
[16/01/2019 11:38:25] <okurz> Martchus: I updated the o3 webui, you might want to check the live log changes, e.g. wrapping
[16/01/2019 11:40:06] <okurz> Martchus: I would say wrapping works even though I was not a friend of the idea from the beginning
Actions #9

Updated by okurz over 5 years ago

  • Status changed from Resolved to Feedback
  • Priority changed from Normal to High

Hi @JERiveraMoya it seems the feature works fine on o3 but not on osd since the deployment of yesterday. E.g. see the report #46577 mentioning
"openQA test in scenario OpenQA::Schema::Result::TestSuites=HASH(0xa00d558) fails in". This seems to happen regardless if the testsuite actually has a description or not. E.g. creating a ticket from https://openqa.suse.de/tests/2393846#step/boot_to_desktop/4 which has a testsuite description seems to cause the same. But again, on o3 it works fine, in both cases of the testsuite having a description or not. Could you please take a look into this soon? I am sure nsinger and mkittler can provide help regarding the instance or debugging.

Actions #10

Updated by JERiveraMoya over 5 years ago

Asking both, it is quite strange that the new section is not even visible.

Actions #11

Updated by okurz over 5 years ago

  • Has duplicate action #46628: [tools] openQA create bug link creates bad text added
Actions #12

Updated by JERiveraMoya over 5 years ago

After talking with @nsinger we can see that at the bottom of the web ui the version is displayed and matches perfectly with the latest commit at the time of checking yesterday in the repo so both are in the same version:
" openQA is licensed GPL-2.0 - Version 4.6.1548078204.c9f60161 " on O3
" openQA is licensed GPL-2.0 - Version 4.6.1548078204.c9f60161 " on OSD
I am working on setup dev env for openQA, it is not completely functional but at least I could check that a failing job for sle could display properly the description.

Actions #13

Updated by JERiveraMoya over 5 years ago

  • Assignee changed from JERiveraMoya to mkittler
Actions #14

Updated by coolo over 5 years ago

did you guys talk or why is this assigned to marius?

Actions #15

Updated by okurz over 5 years ago

yes, they talked, see #testing

Actions #16

Updated by mkittler over 5 years ago

  • Status changed from Feedback to In Progress

The test suite description in the example mentioned by @okurz is simply not rendered because TEST is set to offline_sles11sp4_media_ha+geo_sdk_all but there's only a testsuite called offline_sles11sp4_media_ha. I assume that everything after the plus sign should be ignored to determine the test suite description.

Actions #17

Updated by okurz over 5 years ago

I doubt it's as simple as that. There is also #46628 and #46577 mentioning tests having names without +

Actions #18

Updated by mkittler over 5 years ago

The OpenQA::Schema::Result::TestSuites=HASH(0x9daa1d8) problem is another thing than the test suite description missing.

I added a test for it within https://github.com/os-autoinst/openQA/pull/1974 but couldn't reproduce it (also not when running tests in Docker). Not sure yet what exactly triggers that.

Actions #19

Updated by okurz about 5 years ago

  • Subject changed from [functional][y] testsuite description in the bug reporting template to [tools] testsuite description in the bug reporting template

adapting team assignment as mkittler took over.

Actions #20

Updated by mkittler about 5 years ago

I can't reproduce the OpenQA::Schema::Result::TestSuites=HASH(0x9daa1d8) problem locally:

  • I tried to execute the test case I've added within Docker and couldn't reproduce.
  • I also imported a job (with testresults) from OSD to test with the exact same job (settings). I also couldn't reproduce.

I also tested on o3 and could not reproduce. Besides, the relevant code also seems right.

So I have no clue how to fix this without live-debugging on OSD.

Actions #21

Updated by okurz about 5 years ago

mkittler wrote:

So I have no clue how to fix this without live-debugging on OSD.

Well, do that. Go DevOps! ;)

Actions #22

Updated by mkittler about 5 years ago

  • Due date deleted (2019-01-29)

I'm not seriously considering to mess around on OSD for an issue like this. However, I'm also clueless how to fix it right now. So I'm removing the due date.

Actions #23

Updated by okurz about 5 years ago

@mkittler can I please ask you to handle this issue with higher importance. Users of openqa.suse.de continue to report incomplete bug reports because they do not understand that the template should be crosschecked by the reporter before sending and filled with more useful information. I am sure this will start to annoy bug assignees when we do not change this soon.

Actions #24

Updated by mkittler about 5 years ago

  • Target version changed from Milestone 22 to Current Sprint

I better add it to our sprint then so I wouldn't forget about it.

I need to bring at least one of the staging instances in a usable state again anyways. Maybe I can reproduce the issue on the staging instance then.

Actions #25

Updated by mkittler about 5 years ago

I can't reproduce it on the staging instances. But to do live debugging on OSD I would need to restart the openqa-webui service there a few times. I think that's not the best idea...

Actions #26

Updated by mkittler about 5 years ago

I will try to investigate this using a disk dump from OSD. But maybe we could also try to simply turn the machine off and on again? It feels kind of pointless investigating so much time into this issue which only occurs on one machine when maybe a simple reboot would solve it.

Actions #27

Updated by okurz about 5 years ago

@mkittler I can see that you take care of the issue but because of the gravity of the issue could you please try to adress it together with more members of the tools team, e.g. nsinger and coolo? E.g. see https://bugzilla.suse.com/show_bug.cgi?id=1125222 as yet another example of people creating bug reports with useless confusing content "OpenQA::Schema::Result::TestSuites=HASH(0xa46b6e8)" which will most likely piss off even more people and make them think that openQA is not providing helpful information. How about we try a partial revert, hot fix, hot debugging, etc.? Yes, maybe a reboot, but please do it as a team!

Actions #28

Updated by mkittler about 5 years ago

I have already suggested the reboot to nick. Live-debugging and a partial revert seem more disruptive and inefficient considering that a simple reboot might help.

Nick wanted to provide a dump of the OSD system for me to debug with. But because he was in home office on Monday and is on vacation for the rest of the week we couldn't make any progress 'as a team'.

Actions #29

Updated by okurz about 5 years ago

  • Assignee changed from mkittler to okurz

fine, I will look into this myself

Actions #30

Updated by okurz about 5 years ago

  • Status changed from In Progress to Resolved

There was (still?) the file /etc/openqa/templates/branding/openqa.suse.de/external_reporting.html.ep which I renamed now to ….old . The diff:

# diff -Naur /usr/share/openqa/templates/branding/openSUSE/external_reporting.html.ep /etc/openqa/templates/branding/openqa.suse.de/external_reporting.html.ep.old
--- /usr/share/openqa/templates/branding/openSUSE/external_reporting.html.ep    2019-01-21 14:43:24.000000000 +0100
+++ /etc/openqa/templates/branding/openqa.suse.de/external_reporting.html.ep.old        2017-12-02 12:01:45.254850697 +0100
@@ -2,20 +2,19 @@
 % my $step_url = url_for('step')->to_abs;
 % my $module = stash('moduleid');
 % my %product_details = ();
-% my $scenario_description = $job->scenario_description // '';

 <% my %distri_to_product_url_new = (
     sle => 'https://bugzilla.suse.com/enter_bug.cgi',
     opensuse => 'https://bugzilla.opensuse.org/enter_bug.cgi',
+    casp => 'https://bugzilla.suse.com/enter_bug.cgi',
     caasp => 'https://bugzilla.suse.com/enter_bug.cgi',
-    openqa => 'https://progress.opensuse.org/projects/openqav3/issues/new',
-    kubic => 'https://bugzilla.opensuse.org/enter_bug.cgi'
+    openqa => 'https://progress.opensuse.org/projects/openqav3/issues/new'
 );%>
 <% my %distri_to_prod = (
     sle => 'SUSE Linux Enterprise',
     opensuse => 'openSUSE',
+    casp => 'SUSE Container as a Service Platform',
     caasp => 'SUSE CaaS Platform',
-    kubic => 'openSUSE'
 ); %>
 <% my %flavor_to_prod_sle = (
     Server => 'Server',
@@ -48,6 +47,9 @@
 % elsif ($job->DISTRI eq 'opensuse') {
 %     $product = $job->VERSION eq 'Tumbleweed' ? 'Tumbleweed' : 'Distribution';
 % }
+% elsif ($job->DISTRI eq 'casp') {
+%     $product = $job->VERSION;
+% }
 % elsif ($job->DISTRI eq 'caasp') {
 %     $product = $job->VERSION =~ s/\.[0-9]//r;
 % }
@@ -59,7 +61,7 @@
 %     my ($job) = @_;
 %     return '[' . $job->BUILD . '](' . url_for('test', testid => $job->id)->to_abs .  ')';
 % }
-% my $scenario = $job->scenario_name;
+% my $scenario = $job->scenario;
 % my $first_known_bad = build_link($job) . ' (current job)';
 % my $last_good = '(unknown)';
 % for my $prev ($job->_previous_scenario_jobs) {
@@ -75,9 +77,6 @@
 openQA test in scenario $scenario fails in
 [$module]($step_url)

-## Test suite description
-$scenario_description
-

 ## Reproducible

@@ -104,9 +103,6 @@
 %    $product_details{product} = "$distri $product";
 %    $product_details{bug_file_loc} = $step_url;
 % }
-% if ($distri eq 'kubic') {
-%    $product_details{component} = 'Kubic';
-% }
 % my $product_url_new = $distri_to_product_url_new{$job->DISTRI};
 % if ($product) {
     %= stepaction_for('Report product bug' => url_for($product_url_new)->query(%product_details), 'fa-bug', 'report product_bug');

which differed in the one part that matters here:

-% my $scenario = $job->scenario_name;
+% my $scenario = $job->scenario;

which can of course explain the problem as not a string was returned.

I assume this will fix the problem and we actually do not need more to do. I consider it a bit unfortunate that it took 20 days to reach this conclusion only now.

Actions #31

Updated by mkittler about 5 years ago

That explains everything. I would never have expected it to load templates from /etc/openqa/templates/branding. Did you use strace, or how did you find out?

Actions #32

Updated by okurz about 5 years ago

mkittler wrote:

That explains everything. I would never have expected it to load templates from /etc/openqa/templates/branding. Did you use strace, or how did you find out?

Well, I remembered that the bug reports are created with the bug reporting template and wanted to debug that by using this local overlay to not need to change files in /usr/share . Then I suspected that there might be that local overlay already in place so I checked for the file existance. After I saw that it is there I compared the file with a diff and found that it is using the old "scenario" reference.

Actions

Also available in: Atom PDF