https://progress.opensuse.org/https://progress.opensuse.org/themes/openSUSE/favicon/favicon.ico?15829177842023-03-23T15:59:25ZopenSUSE Project Management ToolQA - action #126551: [qem-bot] Flag missing openQA jobs with qem-dashboard API size:Mhttps://progress.opensuse.org/issues/126551?journal_id=6167752023-03-23T15:59:25Zkraihsebastian.riedel@suse.com
<ul><li><strong>Related to</strong> <i><a class="issue tracker-4 status-3 priority-4 priority-default closed child behind-schedule" href="/issues/126548">action #126548</a>: [qem-dashboard] Add an API endpoint to flag openQA jobs as missing in openQA size:M</i> added</li></ul> QA - action #126551: [qem-bot] Flag missing openQA jobs with qem-dashboard API size:Mhttps://progress.opensuse.org/issues/126551?journal_id=6167782023-03-23T15:59:39Zkraihsebastian.riedel@suse.com
<ul></ul><p>Blocked by <a class="issue tracker-4 status-3 priority-4 priority-default closed child behind-schedule" title="action: [qem-dashboard] Add an API endpoint to flag openQA jobs as missing in openQA size:M (Resolved)" href="https://progress.opensuse.org/issues/126548">#126548</a>.</p>
QA - action #126551: [qem-bot] Flag missing openQA jobs with qem-dashboard API size:Mhttps://progress.opensuse.org/issues/126551?journal_id=6189352023-03-30T20:37:49Zkraihsebastian.riedel@suse.com
<ul><li><strong>Status</strong> changed from <i>Blocked</i> to <i>New</i></li><li><strong>Target version</strong> changed from <i>future</i> to <i>Ready</i></li></ul><p>Unblocked.</p>
QA - action #126551: [qem-bot] Flag missing openQA jobs with qem-dashboard API size:Mhttps://progress.opensuse.org/issues/126551?journal_id=6189442023-03-30T20:41:48Zkraihsebastian.riedel@suse.com
<ul></ul><p>The dashboard endpoint to use is <code>PATCH /api/jobs/<job_id></code>: <a href="https://github.com/openSUSE/qem-dashboard/blob/main/API.md#openqa-jobs" class="external">https://github.com/openSUSE/qem-dashboard/blob/main/API.md#openqa-jobs</a></p>
QA - action #126551: [qem-bot] Flag missing openQA jobs with qem-dashboard API size:Mhttps://progress.opensuse.org/issues/126551?journal_id=6211972023-04-06T07:22:34Zokurzokurz@suse.com
<ul><li><strong>Target version</strong> changed from <i>Ready</i> to <i>future</i></li></ul> QA - action #126551: [qem-bot] Flag missing openQA jobs with qem-dashboard API size:Mhttps://progress.opensuse.org/issues/126551?journal_id=6278632023-04-27T17:16:57Zokurzokurz@suse.com
<ul><li><strong>Target version</strong> changed from <i>future</i> to <i>Ready</i></li></ul> QA - action #126551: [qem-bot] Flag missing openQA jobs with qem-dashboard API size:Mhttps://progress.opensuse.org/issues/126551?journal_id=6295102023-05-04T10:07:29Zokurzokurz@suse.com
<ul><li><strong>Subject</strong> changed from <i>[qem-bot] Flag missing openQA jobs with qem-dashboard API</i> to <i>[qem-bot] Flag missing openQA jobs with qem-dashboard API size:M</i></li><li><strong>Description</strong> updated (<a title="View differences" href="/journals/629510/diff?detail_id=591194">diff</a>)</li><li><strong>Status</strong> changed from <i>New</i> to <i>Workable</i></li></ul> QA - action #126551: [qem-bot] Flag missing openQA jobs with qem-dashboard API size:Mhttps://progress.opensuse.org/issues/126551?journal_id=6389442023-06-02T13:01:19Zmkittlermarius.kittler@suse.com
<ul><li><strong>Assignee</strong> set to <i>mkittler</i></li></ul> QA - action #126551: [qem-bot] Flag missing openQA jobs with qem-dashboard API size:Mhttps://progress.opensuse.org/issues/126551?journal_id=6389712023-06-02T14:11:53Zmkittlermarius.kittler@suse.com
<ul><li><strong>Status</strong> changed from <i>Workable</i> to <i>In Progress</i></li></ul> QA - action #126551: [qem-bot] Flag missing openQA jobs with qem-dashboard API size:Mhttps://progress.opensuse.org/issues/126551?journal_id=6389772023-06-02T14:58:37Zmkittlermarius.kittler@suse.com
<ul><li><strong>Status</strong> changed from <i>In Progress</i> to <i>Feedback</i></li></ul><p>PR: <a href="https://github.com/openSUSE/qem-bot/pull/118" class="external">https://github.com/openSUSE/qem-bot/pull/118</a></p>
QA - action #126551: [qem-bot] Flag missing openQA jobs with qem-dashboard API size:Mhttps://progress.opensuse.org/issues/126551?journal_id=6394392023-06-05T11:35:39Zokurzokurz@suse.com
<ul><li><strong>Due date</strong> set to <i>2023-06-16</i></li></ul> QA - action #126551: [qem-bot] Flag missing openQA jobs with qem-dashboard API size:Mhttps://progress.opensuse.org/issues/126551?journal_id=6407772023-06-07T12:47:35Zlivdywanliv.dywan@suse.com
<ul></ul><p>Discussed in the Unlock:</p>
<ul>
<li>Reproduce locally in the same ubuntu:latest as run in GHA or use github codespaces to bisect</li>
<li>Change the mocked use of osd to something else to be sure that it definitily never resolves</li>
<li>Tina volunteered to try and locally break it</li>
</ul>
QA - action #126551: [qem-bot] Flag missing openQA jobs with qem-dashboard API size:Mhttps://progress.opensuse.org/issues/126551?journal_id=6421222023-06-12T10:48:08Zmkittlermarius.kittler@suse.com
<ul></ul><p>We've established that this is not a problem caused by my changes but rather incompatible versions of certain Python modules being used in the CI. I'll try to switch to using Tumbleweed for CI runs.</p>
QA - action #126551: [qem-bot] Flag missing openQA jobs with qem-dashboard API size:Mhttps://progress.opensuse.org/issues/126551?journal_id=6423022023-06-12T13:32:23Zmkittlermarius.kittler@suse.com
<ul></ul><p>This PR will fix the CI: <a href="https://github.com/openSUSE/qem-bot/pull/120" class="external">https://github.com/openSUSE/qem-bot/pull/120</a></p>
<p>Then I can rebase my other PR on it and finally get it merged.</p>
QA - action #126551: [qem-bot] Flag missing openQA jobs with qem-dashboard API size:Mhttps://progress.opensuse.org/issues/126551?journal_id=6431122023-06-14T09:37:00Zokurzokurz@suse.com
<ul></ul><p><a href="https://github.com/openSUSE/qem-bot/pull/118" class="external">https://github.com/openSUSE/qem-bot/pull/118</a> merged</p>
QA - action #126551: [qem-bot] Flag missing openQA jobs with qem-dashboard API size:Mhttps://progress.opensuse.org/issues/126551?journal_id=6431182023-06-14T09:40:08Zlivdywanliv.dywan@suse.com
<ul><li><strong>Due date</strong> changed from <i>2023-06-16</i> to <i>2023-06-23</i></li></ul><p>mkittler wrote:</p>
<blockquote>
<p>This PR will fix the CI: <a href="https://github.com/openSUSE/qem-bot/pull/120" class="external">https://github.com/openSUSE/qem-bot/pull/120</a></p>
<p>Then I can rebase my other PR on it and finally get it merged.</p>
</blockquote>
<p>Merged. Last wait another week to see if it actually works</p>
QA - action #126551: [qem-bot] Flag missing openQA jobs with qem-dashboard API size:Mhttps://progress.opensuse.org/issues/126551?journal_id=6442132023-06-16T10:48:59Zmkittlermarius.kittler@suse.com
<ul></ul><p>So far there are no jobs flagged as obsolete/missing:</p>
<pre><code>ssh root@dashboard.qam.suse.de
machinectl shell postgresql
sudo -u postgres psql dashboard_db
dashboard_db=# select count(id) from openqa_jobs where obsolete = true;
count
-------
0
(1 row)
</code></pre> QA - action #126551: [qem-bot] Flag missing openQA jobs with qem-dashboard API size:Mhttps://progress.opensuse.org/issues/126551?journal_id=6450742023-06-20T10:23:25Zmkittlermarius.kittler@suse.com
<ul></ul><p>Now some openQA jobs have been flagged:</p>
<pre><code>dashboard_db=# select concat('https://openqa.suse.de/tests/', job_id) from openqa_jobs where obsolete = true;
concat
---------------------------------------
https://openqa.suse.de/tests/11392753
https://openqa.suse.de/tests/11377163
https://openqa.suse.de/tests/11377159
https://openqa.suse.de/tests/11360616
https://openqa.suse.de/tests/11374924
https://openqa.suse.de/tests/11395877
https://openqa.suse.de/tests/11377130
https://openqa.suse.de/tests/11381027
https://openqa.suse.de/tests/11392665
https://openqa.suse.de/tests/11395867
https://openqa.suse.de/tests/11396541
https://openqa.suse.de/tests/11377151
https://openqa.suse.de/tests/11392660
https://openqa.suse.de/tests/11392666
https://openqa.suse.de/tests/11395868
(15 rows)
</code></pre>
<p>So I guess it generally works. It is just strange that this list of jobs also contains jobs that definitely <em>do</em> exists, e.g. <a href="https://openqa.suse.de/tests/11395868" class="external">https://openqa.suse.de/tests/11395868</a>. The openQA comments API (which is used by the bot and a 404 reply would lead to flagging) also returns a 200 response for this job (via <a href="https://openqa.suse.de/api/v1/jobs/11395868/comments" class="external">https://openqa.suse.de/api/v1/jobs/11395868/comments</a>). Any ideas why this could be the case?</p>
QA - action #126551: [qem-bot] Flag missing openQA jobs with qem-dashboard API size:Mhttps://progress.opensuse.org/issues/126551?journal_id=6450832023-06-20T11:16:34Zmkittlermarius.kittler@suse.com
<ul></ul><p>I could also spot relevant lines in the bot's logs, e.g. <a href="https://gitlab.suse.de/qa-maintenance/bot-ng/-/jobs/1644866:">https://gitlab.suse.de/qa-maintenance/bot-ng/-/jobs/1644866:</a></p>
<pre><code>2023-06-20 11:04:43 INFO SUSE:Maintenance:29169:300779 has at least one failed job in aggregate tests
2023-06-20 11:04:43 INFO Job setting 2016724 not found for incident 29270
2023-06-20 11:04:43 INFO Job 11397618 not found in openQA, marking as obsolete on dashboard
2023-06-20 11:04:43 INFO Found failed, not-ignored job https://openqa.suse.de/t11397618 for incident 29270
2023-06-20 11:04:43 INFO SUSE:Maintenance:29270:301195 has at least one failed job in incident tests
2023-06-20 11:04:43 INFO Job setting 2016725 not found for incident 29280
…
023-06-20 11:04:59 INFO SUSE:Maintenance:29403:301154 has at least one failed job in aggregate tests
2023-06-20 11:05:00 INFO Job 11397617 not found in openQA, marking as obsolete on dashboard
2023-06-20 11:05:00 INFO Found failed, not-ignored job https://openqa.suse.de/t11397617 for incident 29407
</code></pre>
<p>Those jobs really don't exist. They do not appear in the list of my previous comment but I've just executed the query again and now they are there as well:</p>
<pre><code>dashboard_db=# select concat('https://openqa.suse.de/tests/', job_id) from openqa_jobs where obsolete = true order by job_id;
concat
---------------------------------------
https://openqa.suse.de/tests/11360616
https://openqa.suse.de/tests/11374924
https://openqa.suse.de/tests/11377130
https://openqa.suse.de/tests/11377151
https://openqa.suse.de/tests/11377159
https://openqa.suse.de/tests/11377163
https://openqa.suse.de/tests/11381027
https://openqa.suse.de/tests/11392665
https://openqa.suse.de/tests/11392666
https://openqa.suse.de/tests/11395906
https://openqa.suse.de/tests/11395914
https://openqa.suse.de/tests/11396541
https://openqa.suse.de/tests/11396786
https://openqa.suse.de/tests/11397615
https://openqa.suse.de/tests/11397616
https://openqa.suse.de/tests/11397617
https://openqa.suse.de/tests/11397618
https://openqa.suse.de/tests/11397623
https://openqa.suse.de/tests/11397626
https://openqa.suse.de/tests/11397627
(20 rows)
</code></pre> QA - action #126551: [qem-bot] Flag missing openQA jobs with qem-dashboard API size:Mhttps://progress.opensuse.org/issues/126551?journal_id=6455032023-06-21T09:44:50Zokurzokurz@suse.com
<ul><li><strong>Status</strong> changed from <i>Feedback</i> to <i>In Progress</i></li></ul><p>as discussed in the unblock please reach out in #eng-testing or #discuss-maintenance and unless you receive horrible backlash consider the work done</p>
QA - action #126551: [qem-bot] Flag missing openQA jobs with qem-dashboard API size:Mhttps://progress.opensuse.org/issues/126551?journal_id=6458842023-06-22T11:29:53Zmkittlermarius.kittler@suse.com
<ul><li><strong>Status</strong> changed from <i>In Progress</i> to <i>Feedback</i></li></ul><p>I sent as message yesterday (to #discuss-maintenance) but haven't gotten a response yet.</p>
QA - action #126551: [qem-bot] Flag missing openQA jobs with qem-dashboard API size:Mhttps://progress.opensuse.org/issues/126551?journal_id=6459232023-06-22T12:36:19Zokurzokurz@suse.com
<ul><li><strong>Due date</strong> deleted (<del><i>2023-06-23</i></del>)</li><li><strong>Status</strong> changed from <i>Feedback</i> to <i>Resolved</i></li></ul><p>I consider no response as "no horrible backlash" :)</p>