https://progress.opensuse.org/https://progress.opensuse.org/themes/openSUSE/favicon/favicon.ico?15829177842022-05-16T09:15:29ZopenSUSE Project Management ToolopenQA Project - action #111152: Investigation jobs run because of the lack of automatic takeover size:Mhttps://progress.opensuse.org/issues/111152?journal_id=5195842022-05-16T09:15:29Ztinitatina.mueller+trick-redmine@suse.com
<ul><li><strong>Copied from</strong> <i><a class="issue tracker-4 status-3 priority-5 priority-high3 closed" href="/issues/110881">action #110881</a>: Investigation jobs run because of the lack of automatic takeover size:S</i> added</li></ul> openQA Project - action #111152: Investigation jobs run because of the lack of automatic takeover size:Mhttps://progress.opensuse.org/issues/111152?journal_id=5195882022-05-16T09:16:48Ztinitatina.mueller+trick-redmine@suse.com
<ul><li><strong>Subject</strong> changed from <i>Investigation jobs run because of the lack of automatic takeover</i> to <i>Investigation jobs run because of the lack of automatic carryover</i></li><li><strong>Description</strong> updated (<a title="View differences" href="/journals/519588/diff?detail_id=491478">diff</a>)</li></ul> openQA Project - action #111152: Investigation jobs run because of the lack of automatic takeover size:Mhttps://progress.opensuse.org/issues/111152?journal_id=5195902022-05-16T09:17:15Ztinitatina.mueller+trick-redmine@suse.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/519590/diff?detail_id=491480">diff</a>)</li></ul> openQA Project - action #111152: Investigation jobs run because of the lack of automatic takeover size:Mhttps://progress.opensuse.org/issues/111152?journal_id=5199652022-05-17T11:14:44Ztinitatina.mueller+trick-redmine@suse.com
<ul><li><strong>Subject</strong> changed from <i>Investigation jobs run because of the lack of automatic carryover</i> to <i>Investigation jobs run because of the lack of automatic takeover</i></li><li><strong>Description</strong> updated (<a title="View differences" href="/journals/519965/diff?detail_id=491855">diff</a>)</li></ul><p>I looked into the minion_jobs table but the job was already too old and the entry deleted.<br>
I was wondering why the investigation comment was made 75 minutes after the job was finished:<br>
<a href="https://openqa.suse.de/tests/8739190#comments" class="external">https://openqa.suse.de/tests/8739190#comments</a></p>
<p><a class="user active user-mention" href="https://progress.opensuse.org/users/23018">@kraih</a> suggested to increase the time to delete minion jobs. The default is 2 days. We should add a setting for it:</p>
<blockquote>
<p>probably one line in WebAPI.pm to assign the setting to $self->minion->remove_after , and the setting itself to the settings module</p>
</blockquote>
<p>SQL for searching for a certain job in the minion table:</p>
<pre><code>select id, args, notes->'hook_rc' as hook_rc, notes->'hook_result' as hook_result, created, finished from minion_jobs where jsonb_typeof(args->0) = 'number' and cast(args->0 as int) = 8739190
</code></pre> openQA Project - action #111152: Investigation jobs run because of the lack of automatic takeover size:Mhttps://progress.opensuse.org/issues/111152?journal_id=5199692022-05-17T11:25:37Ztinitatina.mueller+trick-redmine@suse.com
<ul></ul><p>I looked into the audit log if the takeover comment was even created (and then possibly deleted).<br>
It's a bit hard to tell because the comment events get logged since a few days only, and the oldest comment event in the audit log is from "2022-05-12 05:47:14" CEST I believe.<br>
The job in question finished at 05:25 UTC -> 07:25 CEST, that means the comment event should have been logged.</p>
<p>Now one problem is, comment events are logged with their id, but without the job(group) id.<br>
So if a comment is deleted, we can never connect the comment audit entry to a job.<br>
That will be fixed by <a href="https://github.com/os-autoinst/openQA/pull/4655" class="external">https://github.com/os-autoinst/openQA/pull/4655</a> once it is merged.</p>
<p>But for takeover comments, they are actually logged in the audit table with an additional entry <code>taken_over_from_job_id</code>.<br>
For the job in question that should have been <code>8292431</code>:<br>
<a href="https://openqa.suse.de/tests/8686481#comments" class="external">https://openqa.suse.de/tests/8686481#comments</a><br>
So I looked for that id:</p>
<pre><code>select * from audit_events where event = 'comment_create' and event_data like '%8292431%';
</code></pre>
<p>but got nothing.</p>
<p>So my conclusion is the takeover comment was never created.</p>
<p>But it should have been, and I confirmed that by trying it out locally and just putting the <code>carry_over_bugrefs</code> call into a normal job view for the job in question, and it called the code which would have created the comment, so all conditions for a carry over candidate were fulfilled.</p>
openQA Project - action #111152: Investigation jobs run because of the lack of automatic takeover size:Mhttps://progress.opensuse.org/issues/111152?journal_id=5199712022-05-17T11:27:44Ztinitatina.mueller+trick-redmine@suse.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/519971/diff?detail_id=491861">diff</a>)</li></ul> openQA Project - action #111152: Investigation jobs run because of the lack of automatic takeover size:Mhttps://progress.opensuse.org/issues/111152?journal_id=5207152022-05-19T09:41:45Zmkittlermarius.kittler@suse.com
<ul><li><strong>Subject</strong> changed from <i>Investigation jobs run because of the lack of automatic takeover</i> to <i>Investigation jobs run because of the lack of automatic takeover size:M</i></li><li><strong>Description</strong> updated (<a title="View differences" href="/journals/520715/diff?detail_id=492473">diff</a>)</li><li><strong>Status</strong> changed from <i>New</i> to <i>Workable</i></li></ul> openQA Project - action #111152: Investigation jobs run because of the lack of automatic takeover size:Mhttps://progress.opensuse.org/issues/111152?journal_id=5211472022-05-19T17:24:18Ztinitatina.mueller+trick-redmine@suse.com
<ul><li><strong>Status</strong> changed from <i>Workable</i> to <i>In Progress</i></li><li><strong>Assignee</strong> set to <i>tinita</i></li></ul> openQA Project - action #111152: Investigation jobs run because of the lack of automatic takeover size:Mhttps://progress.opensuse.org/issues/111152?journal_id=5211502022-05-19T17:41:31Ztinitatina.mueller+trick-redmine@suse.com
<ul></ul><p><a href="https://github.com/os-autoinst/openQA/pull/4662" class="external">https://github.com/os-autoinst/openQA/pull/4662</a> Add configuration for expiring minion jobs</p>
openQA Project - action #111152: Investigation jobs run because of the lack of automatic takeover size:Mhttps://progress.opensuse.org/issues/111152?journal_id=5212312022-05-20T04:19:19Zopenqa_reviewopenqa-review@suse.de
<ul><li><strong>Due date</strong> set to <i>2022-06-03</i></li></ul><p>Setting due date based on mean cycle time of SUSE QE Tools</p>
openQA Project - action #111152: Investigation jobs run because of the lack of automatic takeover size:Mhttps://progress.opensuse.org/issues/111152?journal_id=5215162022-05-20T10:59:42Ztinitatina.mueller+trick-redmine@suse.com
<ul></ul><p><a href="https://github.com/os-autoinst/openQA/pull/4662" class="external">https://github.com/os-autoinst/openQA/pull/4662</a> merged</p>
openQA Project - action #111152: Investigation jobs run because of the lack of automatic takeover size:Mhttps://progress.opensuse.org/issues/111152?journal_id=5215222022-05-20T11:11:03Ztinitatina.mueller+trick-redmine@suse.com
<ul><li><strong>Status</strong> changed from <i>In Progress</i> to <i>Feedback</i></li></ul><p><a href="https://github.com/os-autoinst/openQA/pull/4664" class="external">https://github.com/os-autoinst/openQA/pull/4664</a> Improve debugging of _carry_over_candidate</p>
openQA Project - action #111152: Investigation jobs run because of the lack of automatic takeover size:Mhttps://progress.opensuse.org/issues/111152?journal_id=5224132022-05-24T11:04:56Ztinitatina.mueller+trick-redmine@suse.com
<ul></ul><p><a href="https://github.com/os-autoinst/openQA/pull/4664" class="external">https://github.com/os-autoinst/openQA/pull/4664</a> merged</p>
openQA Project - action #111152: Investigation jobs run because of the lack of automatic takeover size:Mhttps://progress.opensuse.org/issues/111152?journal_id=5224282022-05-24T11:41:56Zokurzokurz@suse.com
<ul></ul><p>ok. That should suffice to cover the ACs. So I think we can resolve this ticket. We shouldn't necessarily wait for any other problem observed. Whenever that happens we can look at the logs. WDYT?</p>
openQA Project - action #111152: Investigation jobs run because of the lack of automatic takeover size:Mhttps://progress.opensuse.org/issues/111152?journal_id=5225182022-05-24T12:56:58Ztinitatina.mueller+trick-redmine@suse.com
<ul></ul><p>Well, I was waiting until it is deployed. That's necessary for resolving.<br>
I checked that it is deployed on o3, but not osd yet.</p>
openQA Project - action #111152: Investigation jobs run because of the lack of automatic takeover size:Mhttps://progress.opensuse.org/issues/111152?journal_id=5228692022-05-25T09:27:41Ztinitatina.mueller+trick-redmine@suse.com
<ul><li><strong>Status</strong> changed from <i>Feedback</i> to <i>Resolved</i></li></ul><p>Deployed on osd as well</p>