Project

General

Profile

Actions

action #111710

closed

[qa-tools] [tools] remove usage of *_TEST_TEMPLATE vars in qem-bot and openQA media definition if favor of *_TEST_REPOS size:M

Added by osukup almost 2 years ago. Updated over 1 year ago.

Status:
Resolved
Priority:
Normal
Assignee:
Target version:
Start date:
2022-05-27
Due date:
2022-09-02
% Done:

0%

Estimated time:

Description

Current state:

Now QEM uses in aggregate jobs pretty complicated system to add repositories under test to jobs itself:

qem-bot post jobs with *_TEST_ISUES variable.
In osd are defined media with templates in corresponding *_TEST_TEMPLATE

when is job started from this vars is calculated variable MAINT_TEST_REPO using SCC_ADDONS

used code for this transformation:

main_common.pm -> https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/lib/main_common.pm#L773-L789
SLE main.pm -> https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/products/sle/main.pm#L323-L373

Cons:

  • pretty complex settings for medium with high probability of error
  • code used for this in os-autoinst-distri-opensuse is pretty complex

Proposal:

modify qem-bot to post new vars (*_TEST_REPO) with corresponding repositories and the use simpler code to construct MAINT_TEST_REPO by simply joining this vars still using SCC_ADDONS to join correct products/modules

qem-bot has all needed info for generate this vars which removes one layer of possible human errors (still can be problem in qa-metadata, but it is simpler to spot and debug)

Pro's:

  • remove mess from media settings
  • cleaner/simpler code in os-autoinst-distri-opensuse

Cons:

  • lights more complex code in qem-bot ( Aggregate class )
  • longer command line presented by qem-bot for aggregates

Acceptance criteria

AC1: implement needed changes in qem-bot
AC2: implement needed changes in os-autoinst-distri-opensuse
AC3: (optional) cleanup media definitions in OSD

O3

Note: this changes only handling for OSD, O3 uses different bot so poo is not related ( but can be also implemented ? on O3 side)


Related issues 1 (0 open1 closed)

Related to openQA Tests - action #109707: [qe-core] Detect when a maintenance update has been already released before running the testResolveddvenkatachala2022-06-09

Actions
Actions #1

Updated by jbaier_cz almost 2 years ago

That sounds like a good idea. I believe the bot can handle it without adding much complexity. There is also yet another bonus, without the calculation in "runtime" it should be easier and more transparent to clone a job with different subset if needed.

Actions #2

Updated by okurz almost 2 years ago

  • Target version set to future

I agree with the problem statement. Not sure if moving more complexity into qem-bot is good though.

Actions #3

Updated by osukup almost 2 years ago

okurz wrote:

I agree with the problem statement. Not sure if moving more complexity into qem-bot is good though.

in reality for qem-bot increase of complexity is pretty minimal --> needs only modify this https://github.com/openSUSE/qem-bot/blob/master/openqabot/types/aggregate.py#L167-L168 loop ( +- 2-3 lines of code ?)

on other side it removes from os-autoinst-distri more than 20 lines of code and many more from OSD media definitions

Actions #4

Updated by osukup almost 2 years ago

jbaier_cz wrote:

That sounds like a good idea. I believe the bot can handle it without adding much complexity. There is also yet another bonus, without the calculation in "runtime" it should be easier and more transparent to clone a job with different subset if needed.

there still be some calculation online, but less complex

Actions #5

Updated by osukup almost 2 years ago

  • Subject changed from [qa-tools] remove usage of *_TEST_TEMPLATE vars in qem-bot and openQA media definition to [qa-tools] remove usage of *_TEST_TEMPLATE vars in qem-bot and openQA media definition if favor of *_TEST_REPOS
Actions #6

Updated by dvenkatachala over 1 year ago

  • Related to action #109707: [qe-core] Detect when a maintenance update has been already released before running the test added
Actions #7

Updated by osukup over 1 year ago

@okurz @cdywan we should increase priority / add it in workable ( as handling this vars is pretty big part of poo#109707 ) and we don't want duplicate complex code in os-autoinst-distri-openSUSE + openQA itself.

Actions #8

Updated by osukup over 1 year ago

  • Subject changed from [qa-tools] remove usage of *_TEST_TEMPLATE vars in qem-bot and openQA media definition if favor of *_TEST_REPOS to [qa-tools] [tools] remove usage of *_TEST_TEMPLATE vars in qem-bot and openQA media definition if favor of *_TEST_REPOS
Actions #9

Updated by okurz over 1 year ago

Sure, as soon as we have the current "normal" in the backlog resolved this would be one candidate

Actions #10

Updated by livdywan over 1 year ago

  • Status changed from New to Feedback
  • Assignee set to osukup
  • Target version changed from future to Ready
Actions #11

Updated by osukup over 1 year ago

  • Description updated (diff)
  • Status changed from Feedback to In Progress

qem-bot part implemented.

for full use it needs changes in os-autoinst-distri-openSUSE

Actions #12

Updated by openqa_review over 1 year ago

  • Due date set to 2022-08-23

Setting due date based on mean cycle time of SUSE QE Tools

Actions #13

Updated by okurz over 1 year ago

https://openqa.suse.de/tests/9296092#settings is a job using the new format with the _TEST_REPOS variables. Could you please check that the according code of https://github.com/os-autoinst/scripts/pull/177 still works with the new format? Maybe it's enough to simply extend https://github.com/os-autoinst/scripts/blob/master/openqa-trigger-bisect-jobs#L76 to look for _TEST_(ISSUES|REPOS) and maybe the sorting and grouping already works automatically even with repositories

Actions #14

Updated by mkittler over 1 year ago

  • Subject changed from [qa-tools] [tools] remove usage of *_TEST_TEMPLATE vars in qem-bot and openQA media definition if favor of *_TEST_REPOS to [qa-tools] [tools] remove usage of *_TEST_TEMPLATE vars in qem-bot and openQA media definition if favor of *_TEST_REPOS size:M
Actions #15

Updated by osukup over 1 year ago

  • Status changed from In Progress to Feedback

merged changes in os-auinst-distri-opensuse

so now AC1 and AC2 fullfilled

Actions #16

Updated by okurz over 1 year ago

Please look into #111710#note-13

Actions #17

Updated by osukup over 1 year ago

okurz wrote:

Please look into #111710#note-13

bisect code needs some adjustment , on OSD are now TEST_ISSUES irrelevant + in reality code itself now don't work as excepted. Because it not undefine MAINT_TEST_REPO variable. If its defined (and in clone by default is defined ) whole *_TEST magic is skipped

Actions #18

Updated by osukup over 1 year ago

https://github.com/os-autoinst/scripts/pull/182 -- add support for REPOS and fixed issue with MAINT_TEST_REPOS

unfortunately it has still faulty logic while triggering jobs...

Actions #19

Updated by ph03nix over 1 year ago

This change broke our PublicCloud ARM64 test runs by changing the architecture of the _TEST_REPO to the architecture of the helper VM instead of using the *_TEST_TEMPLATE.
See e.g. https://openqa.suse.de/tests/9369145#settings where BASE_TEST_REPOS is x86_64, while it should be aarch64 (as previously defines in the BASE_TEST_TEMPLATE).

https://github.com/openSUSE/qem-bot/pull/65 introduces a temporary workaround while we search for a better solution. The issue is handled in #115652

Addendum: metadata change: https://gitlab.suse.de/qa-maintenance/metadata/-/merge_requests/681

Actions #21

Updated by osukup over 1 year ago

ph03nix wrote:

This change broke our PublicCloud ARM64 test runs by changing the architecture of the _TEST_REPO to the architecture of the helper VM instead of using the *_TEST_TEMPLATE.
See e.g. https://openqa.suse.de/tests/9369145#settings where BASE_TEST_REPOS is x86_64, while it should be aarch64 (as previously defines in the BASE_TEST_TEMPLATE).

prety nonstandard use of _TEST_TEMPLATE..

https://github.com/openSUSE/qem-bot/pull/65 introduces a temporary workaround while we search for a better solution. The issue is handled in #115652

ook, workaround meged , have fun

Actions #22

Updated by livdywan over 1 year ago

  • Due date changed from 2022-08-23 to 2022-09-02

Considering the impact on e.g. PC tests and limited team capacity I'd say we give this some more time and see where we're at early next week.

Actions #23

Updated by osukup over 1 year ago

  • Status changed from Feedback to Resolved

It works more than two weeks without known problems (except QAC case which was caused by 'non-standard' and practically incorrect usage of templates)*

  • workaround from QAC merged into qem-bot
Actions

Also available in: Atom PDF