openSUSE Project Management Tool: Issueshttps://progress.opensuse.org/https://progress.opensuse.org/themes/openSUSE/favicon/favicon.ico?15829177842023-12-21T16:45:31ZopenSUSE Project Management Tool
Redmine openQA Infrastructure - action #152857 (Resolved): [tools] alert ping between hosts timeout prox...https://progress.opensuse.org/issues/1528572023-12-21T16:45:31Zosukup
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p><a href="https://stats.openqa-monitor.qa.suse.de/d/EML0bpuGk/monitoring?viewPanel=4&orgId=1" class="external">https://stats.openqa-monitor.qa.suse.de/d/EML0bpuGk/monitoring?viewPanel=4&orgId=1</a></p>
<p>looks like proxy.scc.de is down ..</p>
<p><a href="https://suse.slack.com/archives/C029APBKLGK/p1703170652751919" class="external">https://suse.slack.com/archives/C029APBKLGK/p1703170652751919</a></p>
<p>Q: who is responsible for proxy.scc.suse.de and where is running ?</p>
<a name="Rollback-actions"></a>
<h2 >Rollback actions<a href="#Rollback-actions" class="wiki-anchor">¶</a></h2>
<ul>
<li><em>DONE</em> Remove silence "alertname=Packet loss between worker hosts and other hosts alert" from <a href="https://monitor.qa.suse.de/alerting/silences" class="external">https://monitor.qa.suse.de/alerting/silences</a></li>
</ul>
openQA Infrastructure - action #152827 (Resolved): [tools] cron service updating clamav database ...https://progress.opensuse.org/issues/1528272023-12-21T09:17:42Zosukup
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>From /var/spool/mail/cvdupdate on both instances of the osd and o3 web UI:</p>
<pre><code>From cvdupdate@localhost Thu Dec 21 10:00:01 2023
Return-Path: <cvdupdate@localhost>
X-Original-To: cvdupdate
Delivered-To: cvdupdate@localhost
Received: by localhost (Postfix, from userid 17307)
id BC86134590; Thu, 21 Dec 2023 10:00:01 +0100 (CET)
From: "(Cron Daemon)" <cvdupdate@localhost>
To: cvdupdate@localhost
Subject: Cron <cvdupdate@openqa> /home/cvdupdate/.local/bin/cvdupdate update
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Auto-Submitted: auto-generated
Precedence: bulk
X-Cron-Env: <XDG_SESSION_ID=c21238>
X-Cron-Env: <XDG_RUNTIME_DIR=/run/user/17307>
X-Cron-Env: <DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/17307/bus>
X-Cron-Env: <XDG_SESSION_TYPE=unspecified>
X-Cron-Env: <XDG_SESSION_CLASS=background>
X-Cron-Env: <LANG=en_US.UTF-8>
X-Cron-Env: <SHELL=/bin/sh>
X-Cron-Env: <HOME=/home/cvdupdate>
X-Cron-Env: <PATH=/usr/bin:/bin>
X-Cron-Env: <LOGNAME=cvdupdate>
X-Cron-Env: <USER=cvdupdate>
Message-Id: <20231221090001.BC86134590@localhost>
Date: Thu, 21 Dec 2023 10:00:01 +0100 (CET)
Traceback (most recent call last):
File "/home/cvdupdate/.local/bin/cvdupdate", line 11, in <module>
sys.exit(cli())
File "/home/cvdupdate/.local/lib/python3.6/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/home/cvdupdate/.local/lib/python3.6/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/home/cvdupdate/.local/lib/python3.6/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/cvdupdate/.local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/cvdupdate/.local/lib/python3.6/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/home/cvdupdate/.local/lib/python3.6/site-packages/click/decorators.py", line 21, in new_func
return f(get_current_context(), *args, **kwargs)
File "/home/cvdupdate/.local/lib/python3.6/site-packages/cvdupdate/__main__.py", line 259, in update_alias
ctx.forward(db_update)
File "/home/cvdupdate/.local/lib/python3.6/site-packages/click/core.py", line 628, in forward
return self.invoke(cmd, **kwargs)
File "/home/cvdupdate/.local/lib/python3.6/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/home/cvdupdate/.local/lib/python3.6/site-packages/cvdupdate/__main__.py", line 100, in db_update
m = CVDUpdate(config=config, verbose=verbose)
File "/home/cvdupdate/.local/lib/python3.6/site-packages/cvdupdate/cvdupdate.py", line 119, in __init__
nameserver)
File "/home/cvdupdate/.local/lib/python3.6/site-packages/cvdupdate/cvdupdate.py", line 184, in _read_config
self.config = json.load(config_file)
File "/usr/lib64/python3.6/json/__init__.py", line 299, in load
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
File "/usr/lib64/python3.6/json/__init__.py", line 354, in loads
return _default_decoder.decode(s)
File "/usr/lib64/python3.6/json/decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib64/python3.6/json/decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
</code></pre>
<a name="Acceptance-criteria"></a>
<h2 >Acceptance criteria<a href="#Acceptance-criteria" class="wiki-anchor">¶</a></h2>
<ul>
<li><strong>AC1:</strong> Clamav is no longer raising exceptions on o3</li>
<li><strong>AC2:</strong> Clamav is no longer raising exceptions on osd</li>
</ul>
<a name="Suggestions"></a>
<h2 >Suggestions<a href="#Suggestions" class="wiki-anchor">¶</a></h2>
<ul>
<li>Remove clamav service - install on o3, remove from salt on osd</li>
</ul>
openQA Infrastructure - action #152741 (Resolved): [tools] gitlab CI - openqa_review failed with ...https://progress.opensuse.org/issues/1527412023-12-18T15:46:57Zosukup
<p>Looks like osd wasn't able to reply to api longer than 30 sec, can be a random network problem or too complicated query? </p>
<p><a href="https://gitlab.suse.de/openqa/openqa-review/-/jobs/2077520" class="external">https://gitlab.suse.de/openqa/openqa-review/-/jobs/2077520</a></p>
<pre><code>usr/bin/openqa-review --host https://openqa.suse.de -n -r -T --query-issue-status --no-empty-sections --include-softfails --running-threshold=2 --exclude-job-groups '^(Released|Development|old|EOL)' --reminder-comment-on-issues --save --save-dir /tmp/tmp.1LmmaKoNx7 --job-groups '^SLE.*15.*(Functional)'
..............................WARNING:urllib3.connectionpool:Retrying (Retry(total=6, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='openqa.suse.de', port=443): Read timed out. (read timeout=30)")': /api/v1/parent_groups
..................................WARNING:urllib3.connectionpool:Retrying (Retry(total=5, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='openqa.suse.de', port=443): Read timed out. (read timeout=30)")': /api/v1/parent_groups
......................................WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='openqa.suse.de', port=443): Read timed out. (read timeout=30)")': /api/v1/parent_groups
............................................................WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='openqa.suse.de', port=443): Read timed out. (read timeout=30)")': /api/v1/parent_groups
..............................................................WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='openqa.suse.de', port=443): Read timed out. (read timeout=30)")': /api/v1/parent_groups
..............................................................................................WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='openqa.suse.de', port=443): Read timed out. (read timeout=30)")': /api/v1/parent_groups
...................................................................................................................................................................WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='openqa.suse.de', port=443): Read timed out. (read timeout=30)")': /api/v1/parent_groups
..............................WARNING:openqa_review.browser:Request to https://openqa.suse.de/api/v1/parent_groups was not successful after 7 retries: HTTPSConnectionPool(host='openqa.suse.de', port=443): Max retries exceeded with url: /api/v1/parent_groups (Caused by ReadTimeoutError("HTTPSConnectionPool(host='openqa.suse.de', port=443): Read timed out. (read timeout=30)"))
Traceback (most recent call last):
File "/usr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 467, in _make_request
self._validate_conn(conn)
File "/usr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1096, in _validate_conn
conn.connect()
File "/usr/lib/python3.11/site-packages/urllib3/connection.py", line 642, in connect
sock_and_verified = _ssl_wrap_socket_and_match_hostname(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/urllib3/connection.py", line 782, in _ssl_wrap_socket_and_match_hostname
ssl_sock = ssl_wrap_socket(
^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/urllib3/util/ssl_.py", line 470, in ssl_wrap_socket
ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_in_tls, server_hostname)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/urllib3/util/ssl_.py", line 514, in _ssl_wrap_socket_impl
return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/ssl.py", line 517, in wrap_socket
return self.sslsocket_class._create(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/ssl.py", line 1108, in _create
self.do_handshake()
File "/usr/lib64/python3.11/ssl.py", line 1379, in do_handshake
self._sslobj.do_handshake()
TimeoutError: _ssl.c:989: The handshake operation timed out
The above exception was the direct cause of the following exception:
</code></pre> openQA Project - action #135407 (Resolved): [tools] Measure to mitigate websockets overload by wo...https://progress.opensuse.org/issues/1354072023-09-08T11:39:06Zosukup
<a name="Motivation"></a>
<h2 >Motivation<a href="#Motivation" class="wiki-anchor">¶</a></h2>
<p>Consolidate all steps we took to mitigate <a class="issue tracker-6 status-3 priority-4 priority-default closed child parent" title="coordination: [epic] OSD openQA refuses to assign jobs, >3k scheduled not being picked up, no alert (Resolved)" href="https://progress.opensuse.org/issues/135122">#135122</a> and how to revert it.</p>
<p>1) stopped workers:</p>
<p>used:<br>
<code>sudo salt 'worker3[1,2,3,4,5,6]*' cmd.run 'sudo systemctl disable --now telegraf $(systemctl list-units | grep openqa-worker-auto-restart | cut -d "." -f 1 | xargs)'\<br>
&& for i in {1..6}; do sudo salt-key -y -d "worker3$i*"; done</code></p>
<p>revert:<br>
<code>for i in {1..6}; do sudo salt-key -y -a "worker3$i*";done && sudo salt 'worker3[1,2,3,4,5,6]*' state.apply</code></p>
<p>2) Lowered amount workers</p>
<p>used:<br>
<a href="https://gitlab.suse.de/openqa/salt-pillars-openqa/-/merge_requests/606" class="external">https://gitlab.suse.de/openqa/salt-pillars-openqa/-/merge_requests/606</a></p>
<p>revert: <br>
revert mentioned MR in GitLab</p>
<a name="Acceptance-criteria"></a>
<h2 >Acceptance criteria<a href="#Acceptance-criteria" class="wiki-anchor">¶</a></h2>
<ul>
<li><strong>AC1:</strong> Ensure step #1 has been reverted</li>
<li><strong>AC2</strong>: DONE Ensure step #2 has been reverted</li>
</ul>
<a name="Suggestions"></a>
<h2 >Suggestions<a href="#Suggestions" class="wiki-anchor">¶</a></h2>
<ul>
<li>Maybe don't bring them all back at once (and be prepared to remove them again in case of new performance issues)</li>
<li>In case of new performance issues make sure to strace the openqa-scheduler and openqa-websockets processes</li>
</ul>
openQA Infrastructure - action #132860 (Resolved): openqa-piworker is unstable and needs regular ...https://progress.opensuse.org/issues/1328602023-07-17T08:39:49Zosukup
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p><a href="https://gitlab.suse.de/openqa/salt-pillars-openqa/-/jobs/1694765" class="external">https://gitlab.suse.de/openqa/salt-pillars-openqa/-/jobs/1694765</a></p>
<p>only thing found in logs:<br>
salt_ping.log:</p>
<pre><code>Currently the following minions are down:
8d7
< "openqa-piworker.qa.suse.de"
===================
</code></pre>
<a name="Acceptance-criteria"></a>
<h2 >Acceptance criteria<a href="#Acceptance-criteria" class="wiki-anchor">¶</a></h2>
<ul>
<li><strong>AC1:</strong> we are able to process openQA Raspberry Pi bare-metal jobs consistently over some days</li>
</ul>
<a name="Suggestions"></a>
<h2 >Suggestions<a href="#Suggestions" class="wiki-anchor">¶</a></h2>
<ul>
<li><p>Identify the cause for regression</p>
<ul>
<li>likely something related to the hardware RTC</li>
<li>try if it just works with Leap 15.5 because we wanted to upgrade anyway</li>
<li>could be a recent kernel update so try to downgrade</li>
</ul></li>
<li><p>If it is really necessary and you exhausted all other remote-controllable options then go to the office, unplug RTC, reinstall the system assuming it was a borked system and corruption, or whatever</p></li>
<li><p>As Plan Y (if options A to X failed) buy wifi&bluetooth adapter for a IPMI controllable server and use that instead to connect to the rpi bare metal test instances</p></li>
</ul>
<a name="Rollback-steps"></a>
<h2 >Rollback steps<a href="#Rollback-steps" class="wiki-anchor">¶</a></h2>
<ul>
<li>Add back salt key with <code>ssh osd "sudo salt-key -y -a openqa-piworker.qa.suse.de"</code></li>
</ul>
openQA Project - action #129946 (Resolved): [tools][ci][ui] Randomly failing t/ui/15-comments.thttps://progress.opensuse.org/issues/1299462023-05-27T15:02:13Zosukup
<p><a href="https://app.circleci.com/pipelines/github/os-autoinst/openQA/11604/workflows/8145ab21-0f53-4eaa-ba39-8b1b91436f3d/jobs/108379">https://app.circleci.com/pipelines/github/os-autoinst/openQA/11604/workflows/8145ab21-0f53-4eaa-ba39-8b1b91436f3d/jobs/108379</a></p>
<p>from log:</p>
<pre><code class="text syntaxhl" data-language="text">[14:31:56] t/ui/15-comments.t ......................... 11/?
# Failed test 'comment text entered'
# at /home/squamata/project/t/ui/../lib/OpenQA/SeleniumTest.pm line 322.
# Looks like you failed 1 test of 8.
# Failed test 'group overview: /group_overview/1001'
# at t/ui/15-comments.t line 484.
# Looks like you failed 1 test of 4.
[14:31:56] t/ui/15-comments.t ......................... 12/?
# Failed test 'editing when logged in as regular user'
# at t/ui/15-comments.t line 485.
executeScript: unexpected alert open: {Alert text : The comment text mustn't be empty.} at /home/squamata/project/t/ui/../lib/OpenQA/SeleniumTest.pm:81 at /home/squamata/project/t/ui/../lib/OpenQA/SeleniumTest.pm line 84.
OpenQA::SeleniumTest::__ANON__(Test::Selenium::Chrome=HASH(0x55fdfec89a48), "Error while executing command: executeScript: unexpected aler"..., HASH(0x55fdff2179b8), HASH(0x55fdff21ec80)) called at /usr/lib/perl5/vendor_perl/5.26.1/Selenium/Remote/Driver.pm line 356
Selenium::Remote::Driver::catch {...} ("Error while executing command: executeScript: unexpected aler"...) called at /usr/lib/perl5/vendor_perl/5.26.1/Try/Tiny.pm line 123
Try::Tiny::try(CODE(0x55fdff21b690), Try::Tiny::Catch=REF(0x55fdff228708)) called at /usr/lib/perl5/vendor_perl/5.26.1/Selenium/Remote/Driver.pm line 361
Selenium::Remote::Driver::__ANON__(CODE(0x55fdfe908b10), Test::Selenium::Chrome=HASH(0x55fdfec89a48), HASH(0x55fdff2179b8), HASH(0x55fdff21ec80)) called at (eval 1718)[/usr/lib/perl5/vendor_perl/5.26.1/Class/Method/Modifiers.pm:89] line 1
Selenium::Remote::Driver::__ANON__(Test::Selenium::Chrome=HASH(0x55fdfec89a48), HASH(0x55fdff2179b8), HASH(0x55fdff21ec80)) called at (eval 1720)[/usr/lib/perl5/vendor_perl/5.26.1/Class/Method/Modifiers.pm:148] line 2
Selenium::Remote::Driver::_execute_command(Test::Selenium::Chrome=HASH(0x55fdfec89a48), HASH(0x55fdff2179b8), HASH(0x55fdff21ec80)) called at /usr/lib/perl5/vendor_perl/5.26.1/Selenium/Remote/Driver.pm line 1053
Selenium::Remote::Driver::execute_script(Test::Selenium::Chrome=HASH(0x55fdfec89a48), "return window.jQuery \x{26}\x{26} jQuery.active === 0") called at /home/squamata/project/t/ui/../lib/OpenQA/SeleniumTest.pm line 163
OpenQA::SeleniumTest::wait_for_ajax("msg", "comment with pinning for group added by regular user") called at t/ui/15-comments.t line 74
main::write_comment("pinned-description ... The description", "comment with pinning for group added by regular user") called at t/ui/15-comments.t line 471
main::__ANON__() called at /usr/lib/perl5/5.26.1/Test/Builder.pm line 309
eval {...} called at /usr/lib/perl5/5.26.1/Test/Builder.pm line 309
Test::Builder::subtest(Test::Builder=HASH(0x55fdf2476540), "group overview: /group_overview/1001", CODE(0x55fdff066e50)) called at /usr/lib/perl5/5.26.1/Test/More.pm line 807
Test::More::subtest("group overview: /group_overview/1001", CODE(0x55fdff066e50)) called at t/ui/15-comments.t line 484
main::__ANON__() called at /usr/lib/perl5/5.26.1/Test/Builder.pm line 309
eval {...} called at /usr/lib/perl5/5.26.1/Test/Builder.pm line 309
Test::Builder::subtest(Test::Builder=HASH(0x55fdf2476540), "editing when logged in as regular user", CODE(0x55fdff01d0e0)) called at /usr/lib/perl5/5.26.1/Test/More.pm line 807
Test::More::subtest("editing when logged in as regular user", CODE(0x55fdff01d0e0)) called at t/ui/15-comments.t line 485
[14:31:56] t/ui/15-comments.t ......................... 13/? # Tests were run but no plan was declared and done_testing() was not seen.
[14:31:56] t/ui/15-comments.t ......................... Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/13 subtests
</code></pre> QA - action #123523 (Resolved): [tools] osc-plugin-qam throws traceback on Tumbleweed size:Mhttps://progress.opensuse.org/issues/1235232023-01-23T10:07:23Zosukup
<p>as reported here: <a href="https://suse.slack.com/archives/C02CUAYSFDL/p1674466273700499" class="external">https://suse.slack.com/archives/C02CUAYSFDL/p1674466273700499</a></p>
<p>it looks like new osc-1.0.0 has change in <code>do_*something*</code> API and it breaks osc-plugin-qam:</p>
<pre><code>/usr/lib/python3.10/site-packages/osc/cmdln.py:267: FutureWarning: do_qam() handler has deprecated signature. It takes the following args: ['subcmd', 'opts', 'args', 'kwargs'], while it should be taking ['subcmd', 'opts'] and handling positional arguments explicitly via @cmdln.option.
warnings.warn(
Traceback (most recent call last):
File "/usr/bin/osc", line 33, in <module>
sys.exit(load_entry_point('osc==1.0.0b3', 'console_scripts', 'osc')())
File "/usr/lib/python3.10/site-packages/osc/babysitter.py", line 208, in main
sys.exit(run(commandline.Osc()))
File "/usr/lib/python3.10/site-packages/osc/babysitter.py", line 56, in run
return prg.main(argv)
File "/usr/lib/python3.10/site-packages/osc/cmdln.py", line 274, in main
cmd(self.options.command, self.options, *self.args)
File "/usr/lib/osc-plugins/qam.py", line 648, in do_qam
interp.optparser = cmdln.SubCmdOptionParser()
AttributeError: module 'osc.cmdln' has no attribute 'SubCmdOptionParser'
error: Assign failed: Command '['osc', '-A', 'https://api.suse.de', 'qam', 'assign', '288537']' returned non-zero exit status 1.
</code></pre>
<a name="Suggestions"></a>
<h2 >Suggestions<a href="#Suggestions" class="wiki-anchor">¶</a></h2>
<ul>
<li>Look at maybe-related PR in osc: <a href="https://github.com/openSUSE/osc/pull/991" class="external">https://github.com/openSUSE/osc/pull/991</a></li>
</ul>
QA - action #115103 (Resolved): [tools] osc-plugin-qam reads Incident Priority from IBS but that ...https://progress.opensuse.org/issues/1151032022-08-08T13:32:38Zosukup
<p>In past Incident priority was stored as Incident attribute in IBS.<br>
In recent time maintenance stopped using this and Priority is held by SMELT, and attribute is used only for over-ride.</p>
<p>So osc-plugin-qam needs to read priority from IBS and if its empty read correct value from SMELT</p>
<a name="Acceptance-criteria"></a>
<h2 >Acceptance criteria<a href="#Acceptance-criteria" class="wiki-anchor">¶</a></h2>
<ul>
<li><strong>AC1</strong>: correct value of priority is shown</li>
</ul>
QA - action #111710 (Resolved): [qa-tools] [tools] remove usage of *_TEST_TEMPLATE vars in qem-b...https://progress.opensuse.org/issues/1117102022-05-27T14:23:36Zosukup
<a name="Current-state"></a>
<h2 >Current state:<a href="#Current-state" class="wiki-anchor">¶</a></h2>
<p>Now QEM uses in aggregate jobs pretty complicated system to add repositories under test to jobs itself:</p>
<p><em>qem-bot</em> post jobs with <code>*_TEST_ISUES</code> variable.<br>
In <em>osd</em> are defined media with templates in corresponding <code>*_TEST_TEMPLATE</code></p>
<p>when is job started from this vars is calculated variable <code>MAINT_TEST_REPO</code> using <code>SCC_ADDONS</code></p>
<p>used code for this transformation:</p>
<p><code>main_common.pm</code> -> <a href="https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/lib/main_common.pm#L773-L789" class="external">https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/lib/main_common.pm#L773-L789</a><br>
SLE <code>main.pm</code> -> <a href="https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/products/sle/main.pm#L323-L373" class="external">https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/products/sle/main.pm#L323-L373</a></p>
<a name="Cons"></a>
<h3 >Cons:<a href="#Cons" class="wiki-anchor">¶</a></h3>
<ul>
<li>pretty complex settings for medium with high probability of error</li>
<li>code used for this in os-autoinst-distri-opensuse is pretty complex</li>
</ul>
<a name="Proposal"></a>
<h2 >Proposal:<a href="#Proposal" class="wiki-anchor">¶</a></h2>
<p>modify <em>qem-bot</em> to post new vars (<code>*_TEST_REPO</code>) with corresponding repositories and the use simpler code to construct <code>MAINT_TEST_REPO</code> by simply joining this vars still using SCC_ADDONS to join correct products/modules</p>
<p>qem-bot has all needed info for generate this vars which removes one layer of possible human errors (still can be problem in qa-metadata, but it is simpler to spot and debug)</p>
<a name="Pros"></a>
<h3 >Pro's:<a href="#Pros" class="wiki-anchor">¶</a></h3>
<ul>
<li>remove mess from media settings</li>
<li>cleaner/simpler code in os-autoinst-distri-opensuse</li>
</ul>
<a name="Cons-2"></a>
<h3 >Cons:<a href="#Cons-2" class="wiki-anchor">¶</a></h3>
<ul>
<li>lights more complex code in qem-bot ( Aggregate class )</li>
<li>longer command line presented by qem-bot for aggregates</li>
</ul>
<a name="Acceptance-criteria"></a>
<h2 >Acceptance criteria<a href="#Acceptance-criteria" class="wiki-anchor">¶</a></h2>
<p><strong>AC1</strong>: implement needed changes in qem-bot<br>
<strong>AC2</strong>: implement needed changes in os-autoinst-distri-opensuse<br>
<strong>AC3</strong>: (optional) cleanup media definitions in OSD</p>
<a name="O3"></a>
<h2 >O3<a href="#O3" class="wiki-anchor">¶</a></h2>
<p>Note: this changes only handling for OSD, O3 uses different <code>bot</code> so poo is not related ( but can be also implemented ? on O3 side)</p>
QA - action #111506 (Resolved): qa-tools: qem-bot - Development results leaked to dashboard size:Mhttps://progress.opensuse.org/issues/1115062022-05-24T08:17:00Zosukup
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>Development group 'pstivanin-security@Server-DVD-Updates' results synced to qem dashboard</p>
<p>We already filter out development jobs in <a href="https://github.com/openSUSE/qem-bot/blob/master/openqabot/aggrsync.py#L53-L68" class="external">https://github.com/openSUSE/qem-bot/blob/master/openqabot/aggrsync.py#L53-L68</a> in case of aggregates or <a href="https://github.com/openSUSE/qem-bot/blob/master/openqabot/incsyncres.py#L48-L61" class="external">https://github.com/openSUSE/qem-bot/blob/master/openqabot/incsyncres.py#L48-L61</a> but today <a href="https://openqa.suse.de/tests/overview?version=15-SP3&groupid=431&flavor=Server-DVD-Updates&distri=sle&build=20220523-1" class="external">pstivanin-security</a> leaked to qem-dashboard in <a href="http://dashboard.qam.suse.de/incident/24199" class="external">http://dashboard.qam.suse.de/incident/24199</a> when it shouldn't show it.</p>
<a name="Acceptance-criteria"></a>
<h2 >Acceptance criteria<a href="#Acceptance-criteria" class="wiki-anchor">¶</a></h2>
<ul>
<li><strong>AC1:</strong> Development jobs should not be part of incident approval</li>
<li><strong>AC2:</strong> Dashboard no longer shows development jobs</li>
</ul>
<a name="Suggestions"></a>
<h2 >Suggestions<a href="#Suggestions" class="wiki-anchor">¶</a></h2>
<ul>
<li>Don't let the bot submit development jobs to the dashboard</li>
<li>Prevent duplication in <a href="https://github.com/openSUSE/qem-bot/blob/master/openqabot/aggrsync.py#L53-L68" class="external">https://github.com/openSUSE/qem-bot/blob/master/openqabot/aggrsync.py#L53-L68</a> and <a href="https://github.com/openSUSE/qem-bot/blob/master/openqabot/incsyncres.py#L48-L61" class="external">https://github.com/openSUSE/qem-bot/blob/master/openqabot/incsyncres.py#L48-L61</a></li>
<li>Also exclude jobs that are part of any parent job group regardless of the job group names</li>
</ul>
<a name="Further-notes"></a>
<h2 >Further notes<a href="#Further-notes" class="wiki-anchor">¶</a></h2>
<ul>
<li>A "development job" is a job in a job group or in a parent job group where the name contains "Development".</li>
</ul>
openQA Project - action #101478 (Resolved): openqa-review pipeline failed because details-* JSON ...https://progress.opensuse.org/issues/1014782021-10-26T07:14:02Zosukup
<p><a href="https://gitlab.suse.de/openqa/openqa-review/-/jobs/658778" class="external">https://gitlab.suse.de/openqa/openqa-review/-/jobs/658778</a></p>
<pre><code>+ /usr/bin/openqa-review --host https://openqa.suse.de -n -r -T --query-issue-status --no-empty-sections --include-softfails --running-threshold=2 --exclude-job-groups '^(Released|Development|old|EOL)' --reminder-comment-on-issues --save --save-dir /tmp/tmp.UXQt70yio3 --skip-passed
WARNING:openqa_review.browser:Unable to decode JSON for [{"build_version_sort":0,"carry_over_bugrefs":1,"default_keep_important_logs_in_days":"90","default_keep_important_results_in_days":"0","default_keep_logs_in_days":"10","default_keep_results_in_days":"21","default_priority":40,"description":"https:\/\/confluence.suse.com\/pages\/viewpage.action?pageId=723878219\r\n\r\nResponsible persons: Stefan Barth <stephan.barth@suse.com>, Heiko Rommel <heiko.rommel@suse.com>","exclusively_kept_asset_size":0,"id":36,"name":"Maintenance: On Submission","size_limit_gb":null,"sort_order":6},{"build_version_sort":0,"carry_over_bugrefs":1,"default_keep_important_logs_in_days":90,"default_keep_important_results_in_days":0,"default_keep_logs_in_days":10,"default_keep_results_in_days":70,"default_priority":50,"description":"","exclusively_kept_asset_size":0,"id":8,"name":"Maintenance: Single Incidents","size_limit_gb":null,"sort_order":7},{"build_version_sort":0,"carry_over_bugrefs":1,"default_keep_important_logs_in_days":120,"default_keep_important_results_in_days":0,"default_keep_logs_in_days":30,"default_keep_results_in_days":90,"default_priority":50,"description":"","exclusively_kept_asset_size":0,"id":1,"name":"Released","size_limit_gb":null,"sort_order":14},{"build_version_sort":1,"carry_over_bugrefs":1,"default_keep_important_logs_in_days":"90","default_keep_important_results_in_days":"0","default_keep_logs_in_days":"10","default_keep_results_in_days":"21","default_priority":50,"description":"","exclusively_kept_asset_size":0,"id":32,"name":"SLE Micro","size_limit_gb":null,"sort_order":4},{"build_version_sort":1,"carry_over_bugrefs":1,"default_keep_important_logs_in_days":"90","default_keep_important_results_in_days":"0","default_keep_logs_in_days":"10","default_keep_results_in_days":"21","default_priority":50,"description":null,"exclusively_kept_asset_size":0,"id":33,"name":"WSL","size_limit_gb":null,"sort_order":5},{"build_version_sort":0,"carry_over_bugrefs":1,"default_keep_important_logs_in_days":20,"default_keep_important_results_in_days":50,"default_keep_logs_in_days":5,"default_keep_results_in_days":12,"default_priority":60,"description":"Just grouping job groups that have no direct product relevance","exclusively_kept_asset_size":0,"id":9,"name":"Development","size_limit_gb":null,"sort_order":13},{"build_version_sort":0,"carry_over_bugrefs":1,"default_keep_important_logs_in_days":"90","default_keep_important_results_in_days":"0","default_keep_logs_in_days":"10","default_keep_results_in_days":"21","default_priority":50,"description":"","exclusively_kept_asset_size":0,"id":35,"name":"Containers","size_limit_gb":null,"sort_order":3},{"build_version_sort":0,"carry_over_bugrefs":0,"default_keep_important_logs_in_days":90,"default_keep_important_results_in_days":0,"default_keep_logs_in_days":10,"default_keep_results_in_days":70,"default_priority":50,"description":"","exclusively_kept_asset_size":0,"id":27,"name":"Public Cloud","size_limit_gb":null,"sort_order":2},{"build_version_sort":0,"carry_over_bugrefs":1,"default_keep_important_logs_in_days":90,"default_keep_important_results_in_days":0,"default_keep_logs_in_days":10,"default_keep_results_in_days":70,"default_priority":50,"description":"","exclusively_kept_asset_size":0,"id":24,"name":"Maintenance: Kiwi","size_limit_gb":null,"sort_order":12},{"build_version_sort":1,"carry_over_bugrefs":1,"default_keep_important_logs_in_days":30,"default_keep_important_results_in_days":0,"default_keep_logs_in_days":10,"default_keep_results_in_days":50,"default_priority":60,"description":"https:\/\/confluence.suse.com\/display\/openqa\/Testing+of+Quarterly+Refreshed+ISOs+in+openQA\r\n\r\nResponsible person: hrommel <heiko.rommel@suse.com>","exclusively_kept_asset_size":158186928501,"id":23,"name":"Maintenance: QR","size_limit_gb":150,"sort_order":15},{"build_version_sort":1,"carry_over_bugrefs":1,"default_keep_important_logs_in_days":"90","default_keep_important_results_in_days":"0","default_keep_logs_in_days":"10","default_keep_results_in_days":"21","default_priority":50,"description":null,"exclusively_kept_asset_size":0,"id":30,"name":"L3","size_limit_gb":null,"sort_order":11},{"build_version_sort":1,"carry_over_bugrefs":1,"default_keep_important_logs_in_days":"90","default_keep_important_results_in_days":"0","default_keep_logs_in_days":"10","default_keep_results_in_days":"21","default_priority":50,"description":null,"exclusively_kept_asset_size":0,"id":34,"name":"SLES JeOS","size_limit_gb":null,"sort_order":1},{"build_version_sort":1,"carry_over_bugrefs":1,"default_keep_important_logs_in_days":"90","default_keep_important_results_in_days":"0","default_keep_logs_in_days":"10","default_keep_results_in_days":"21","default_priority":50,"description":null,"exclusively_kept_asset_size":0,"id":28,"name":"Others","size_limit_gb":null,"sort_order":18},{"build_version_sort":1,"carry_over_bugrefs":1,"default_keep_important_logs_in_days":"90","default_keep_important_results_in_days":"0","default_keep_logs_in_days":"10","default_keep_results_in_days":"21","default_priority":50,"description":null,"exclusively_kept_asset_size":0,"id":37,"name":"EOL","size_limit_gb":null,"sort_order":16},{"build_version_sort":0,"carry_over_bugrefs":1,"default_keep_important_logs_in_days":90,"default_keep_important_results_in_days":0,"default_keep_logs_in_days":10,"default_keep_results_in_days":70,"default_priority":50,"description":"","exclusively_kept_asset_size":0,"id":21,"name":"Maintenance: Single Incidents SLE-12","size_limit_gb":null,"sort_order":8},{"build_version_sort":1,"carry_over_bugrefs":1,"default_keep_important_logs_in_days":"90","default_keep_important_results_in_days":"0","default_keep_logs_in_days":"10","default_keep_results_in_days":"21","default_priority":50,"description":null,"exclusively_kept_asset_size":0,"id":31,"name":"Maintenace: KOTD","size_limit_gb":null,"sort_order":10},{"build_version_sort":0,"carry_over_bugrefs":1,"default_keep_important_logs_in_days":90,"default_keep_important_results_in_days":0,"default_keep_logs_in_days":10,"default_keep_results_in_days":70,"default_priority":50,"description":"","exclusively_kept_asset_size":0,"id":7,"name":"Maintenance: Test Repo","size_limit_gb":null,"sort_order":9},{"build_version_sort":1,"carry_over_bugrefs":1,"default_keep_important_logs_in_days":90,"default_keep_important_results_in_days":0,"default_keep_logs_in_days":10,"default_keep_results_in_days":70,"default_priority":50,"description":"see https:\/\/wiki.suse.net\/index.php\/RD-OPS_QA\/openQA_review for the review process and responsible persons for review.\r\n\r\nAlso see [#qa-review](irc:\/\/irc.suse.de\/qa-review)","exclusively_kept_asset_size":0,"id":15,"name":"SLE 15","size_limit_gb":null,"sort_order":0},{"build_version_sort":1,"carry_over_bugrefs":1,"default_keep_important_logs_in_days":"90","default_keep_important_results_in_days":"0","default_keep_logs_in_days":"10","default_keep_results_in_days":"21","default_priority":50,"description":null,"exclusively_kept_asset_size":0,"id":38,"name":"Liberty","size_limit_gb":null,"sort_order":17}]: Expecting value: line 1 column 1 (char 0) (Content was: "https://openqa.suse.de/api/v1/parent_groups")
Traceback (most recent call last):
File "/usr/lib/python3.8/site-packages/openqa_review/browser.py", line 161, in _decode_content
content = json.loads(raw) if as_json else raw
File "/usr/lib64/python3.8/json/__init__.py", line 357, in loads
return _default_decoder.decode(s)
File "/usr/lib64/python3.8/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib64/python3.8/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
</code></pre> QA - action #96968 (Resolved): qem-dashboard jobs api extension https://progress.opensuse.org/issues/969682021-08-16T10:45:29Zosukup
<p><a href="https://gitlab.suse.de/opensuse/qem-dashboard/-/issues/14" class="external">https://gitlab.suse.de/opensuse/qem-dashboard/-/issues/14</a></p>
<p>for approve part of bot, we need access to list of jobs for exact settings (incident_settings/updates_settings)</p>
<p>proposed api is described in gitlab issue</p>
openQA Infrastructure - action #96719 (Resolved): recover imagetester with broken filesystem/hard...https://progress.opensuse.org/issues/967192021-08-10T14:36:00Zosukup
<p>During work on <a href="https://progress.opensuse.org/issues/96311" class="external">https://progress.opensuse.org/issues/96311</a> , we found imagetester wasn't updated for 2 months</p>
<p>investigate why wasn't automatic transactional update working and update imagetester.</p>
<p>now blocked by <a href="https://infra.nue.suse.com/SelfService/Display.html?id=194271" class="external">https://infra.nue.suse.com/SelfService/Display.html?id=194271</a> , because it didn't survive reboot and this host hasn't any remote management interface</p>
openQA Project - action #92665 (Resolved): Automatically validate code style for python codehttps://progress.opensuse.org/issues/926652021-05-13T09:53:58Zosukup
<a name="Motivation"></a>
<h2 >Motivation<a href="#Motivation" class="wiki-anchor">¶</a></h2>
<p>It would be great to have automated checks for more style aspects to python code</p>
<p>We use perl-Tidy and perl-Critic for perl code and strictly checks in CI, it will be nice to have same<br>
standard also for other languages.</p>
<a name="Acceptance-criteria"></a>
<h2 >Acceptance criteria<a href="#Acceptance-criteria" class="wiki-anchor">¶</a></h2>
<ul>
<li><strong>AC1</strong>: python code style is checked automatically</li>
</ul>
<a name="Acceptance-tests"></a>
<h2 >Acceptance tests<a href="#Acceptance-tests" class="wiki-anchor">¶</a></h2>
<ul>
<li><strong>AT1</strong>: Inconsistent codestyle prevents deployments</li>
</ul>
<a name="Suggestion"></a>
<h2 >Suggestion<a href="#Suggestion" class="wiki-anchor">¶</a></h2>
<ul>
<li>use <a href="https://github.com/psf/black" class="external">black</a> for python code style</li>
</ul>
openQA Tests - action #42488 (Resolved): [qam][functional][y][fast] test fails in partitioning_raidhttps://progress.opensuse.org/issues/424882018-10-15T08:39:49Zosukup
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>openQA test in scenario sle-15-Server-DVD-Incidents-Minimal-x86_64-qam-minimal-RAID1+sle15@64bit-smp fails in<br>
<a href="https://openqa.suse.de/tests/2172164/modules/partitioning_raid/steps/12" class="external">partitioning_raid</a></p>
<a name="Reproducible"></a>
<h2 >Reproducible<a href="#Reproducible" class="wiki-anchor">¶</a></h2>
<p>Fails since (at least) Build <a href="https://openqa.suse.de/tests/2160531" class="external">:7911:kmod.1538309649</a></p>
<a name="Expected-result"></a>
<h2 >Expected result<a href="#Expected-result" class="wiki-anchor">¶</a></h2>
<p>Last good: <a href="https://openqa.suse.de/tests/2138785" class="external">:8985:zypper.1538995398</a> (or more recent)</p>
<a name="Further-details"></a>
<h2 >Further details<a href="#Further-details" class="wiki-anchor">¶</a></h2>
<p>Always latest result in this scenario: <a href="https://openqa.suse.de/tests/latest?machine=64bit-smp&flavor=Server-DVD-Incidents-Minimal&test=qam-minimal-RAID1%2Bsle15&version=15&distri=sle&arch=x86_64" class="external">latest</a></p>