Project

General

Profile

Actions

action #60118

closed

load_templates does not load JobGroups (unless they exist)

Added by dancermak over 4 years ago. Updated about 4 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Regressions/Crashes
Target version:
Start date:
2019-11-21
Due date:
% Done:

0%

Estimated time:

Description

I have tried to setup a local openQA instance in a container as described in the official documentation and after getting all the tests (http://open.qa/docs/#_getting_tests), I tried to fetch the job groups, templates, etc from o3. Unfortunately using dump_templates & load_templates fails:

openqa1:/ # /usr/share/openqa/script/dump_templates --host openqa.opensuse.org > templates.json
openqa1:/ # /usr/share/openqa/script/load_templates --host localhost  templates.json
ERROR: 400 - Bad Request
{ error => ["Job group OBS Unstable not found"], error_status => 400 }
ERROR: 400 - Bad Request
{
  error => [{ message => "Missing property.", path => "/scenarios" }],
  error_status => 400,
}
ERROR: 400 - Bad Request
{
  error => ["Job group openSUSE Leap 15.0 Incidents not found"],
  error_status => 400,
}
ERROR: 400 - Bad Request
{
  error => ["Job group Staging Projects not found"],
  error_status => 400,
}
ERROR: 400 - Bad Request
{
  error => ["Job group openSUSE Leap 42.3 Updates not found"],
  error_status => 400,
}
ERROR: 400 - Bad Request
{
  error => ["Job group openSUSE Leap Staging Projects not found"],
  error_status => 400,
}
ERROR: 400 - Bad Request
{
  error => ["Job group openSUSE Leap 15.1 AArch64 Images not found"],
  error_status => 400,
}
ERROR: 400 - Bad Request
{
  error => ["Job group openSUSE Leap 15.1 Updates not found"],
  error_status => 400,
}

  SNIP

ERROR: 414 - Request-URI Too Long
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Strict//EN\"\n  \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd\">\n<html xmlns=\"http://www.w3.
org/1999/xhtml\" lang=\"en\" xml:lang=\"en\">\n<head>\n<title>Submitted URI too large!</title>\n<link rev=\"made\" href=\"mailto:%5bno%20address%20given%5d\" />\n<style type=\"text/css\"><!--
/*--><![CDATA[/*><!--*/ \n    body { color: #000000; background-color: #FFFFFF; }\n    a:link { color: #0000CC; }\n    p, address {margin-left: 3em;}\n    span {font-size: smaller;}\n/*]]>*/-
-></style>\n</head>\n\n<body>\n<h1>Submitted URI too large!</h1>\n<p>\n\n\n    The length of the requested URL exceeds the capacity limit for\n\tthis server. The request cannot be processed.\
n   \n</p>\n<p>\nIf you think this is a server error, please contact\nthe <a href=\"mailto:%5bno%20address%20given%5d\">webmaster</a>.\n\n</p>\n\n<h2>Error 414</h2>\n<address>\n  <a href=\"/\
">openqa1</a><br />\n  <span>Apache</span>\n</address>\n</body>\n</html>\n\n"

  SNIP

{
  JobGroups    => { added => 0, of => 40 },
  JobTemplates => { added => 0, of => 0 },
  Machines     => { added => 49, of => 49 },
  Products     => { added => 140, of => 140 },
  TestSuites   => { added => 331, of => 331 },
}

I have attached the templates.json file that was produced by dump_templates.


Files

templates.json (445 KB) templates.json created by dump_templates --host openqa.opensuse.org (not actually a json file) dancermak, 2019-11-21 14:04
Actions #1

Updated by livdywan over 4 years ago

  • Subject changed from /usr/share/openqa/script/load_templates does not load JobGroups to load_templates does not load JobGroups (unless they exist)
  • Status changed from New to Workable
  • Assignee set to livdywan
  • Target version set to Current Sprint

Thanks for filing the bug!

I think what's happening here is that the script tries to update job groups without ensuring they actually exist.

Actions #2

Updated by livdywan over 4 years ago

  • Description updated (diff)
Actions #3

Updated by dancermak over 4 years ago

cdywan wrote:

Thanks for filing the bug!

I think what's happening here is that the script tries to update job groups without ensuring they actually exist.

Would that mean, that if I create the groups manually, that the script will automatically populate them? Since I tried that with the openSUSE Tumbleweed group and it did not get filled with the data from o3.

Actions #4

Updated by livdywan about 4 years ago

  • Status changed from Workable to In Progress

I'm writing unit tests atm as a prerequisite to fixing this.

Actions #7

Updated by livdywan about 4 years ago

With the earlier fixes and new tests in, what's left is making sure job groups always get created - which also makes importing the same group(s) twice an error, which should fail unless the clean option is used as well.

Btw just a side note but by default dump_templates writes Perl data. You need to add --json to actually get JSON output, as suggested by the filename i.e. dump_templates --host openqa.opensuse.org > templates.json but the problem description is valid regardless.

Actions #8

Updated by livdywan about 4 years ago

See gh#2770 for the fix, and gh#2776 for a semi-related branch to improve robustness of the script.

Actions #9

Updated by okurz about 4 years ago

https://github.com/os-autoinst/openQA/pull/2770 merged. Let me ask you what is maybe obvious, just to be sure: Did you also test the use case that dancermak provided, i.e. import templates into a real empty openQA instance?

Actions #10

Updated by livdywan about 4 years ago

  • Status changed from In Progress to Feedback

okurz wrote:

https://github.com/os-autoinst/openQA/pull/2770 merged. Let me ask you what is maybe obvious, just to be sure: Did you also test the use case that dancermak provided, i.e. import templates into a real empty openQA instance?

Yes. And it should work. Now that it's merged I was planning to double-check that with @dancermak

Actions #11

Updated by dancermak about 4 years ago

  • Status changed from Feedback to Closed

I have just created a test openQA instance and can confirm that load_templates now does the job as expected.

Thank you for fixing it!

Actions #12

Updated by okurz about 4 years ago

  • Status changed from Closed to Resolved

thanks for crosschecking. For ticket status: We use "Resolved" only, better avoid "Closed" as described on https://progress.opensuse.org/projects/openqav3/wiki/#ticket-workflow

Actions

Also available in: Atom PDF