https://progress.opensuse.org/https://progress.opensuse.org/themes/openSUSE/favicon/favicon.ico?15829177842019-03-04T12:02:13ZopenSUSE Project Management ToolopenQA Project - coordination #48641: [epic] Trigger openQA tests in pull requests of any product github pull requesthttps://progress.opensuse.org/issues/48641?journal_id=1948312019-03-04T12:02:13Zokurzokurz@suse.com
<ul><li><strong>Copied from</strong> <i><a class="issue tracker-4 status-3 priority-4 priority-default closed" href="/issues/44327">action #44327</a>: Trigger tests based on git refspec/branch</i> added</li></ul> openQA Project - coordination #48641: [epic] Trigger openQA tests in pull requests of any product github pull requesthttps://progress.opensuse.org/issues/48641?journal_id=2501362019-10-15T10:21:24Zokurzokurz@suse.com
<ul><li><strong>Parent task</strong> set to <i>#58184</i></li></ul> openQA Project - coordination #48641: [epic] Trigger openQA tests in pull requests of any product github pull requesthttps://progress.opensuse.org/issues/48641?journal_id=2513272019-10-18T13:19:01Zmkittlermarius.kittler@suse.com
<ul></ul><ul>
<li>This use-case is also impaired by the inability to use custom needles (<a href="https://progress.opensuse.org/issues/56789" class="external">https://progress.opensuse.org/issues/56789</a> <a class="issue tracker-4 status-1 priority-3 priority-lowest child" title="action: New needles from git repository not working with openqa-clone-custom-git-refspec (New)" href="https://progress.opensuse.org/issues/56789">#56789</a>).</li>
<li>Besides, the implementation suggestion <code>use SCHEDULE parameter with e.g. SCHEDULE=tests/boot/boot_to_desktop,tests/run_custom</code> implies that a composability of test distributions would be required. This is of course possible by somehow embedding the base test distribution (e.g. os-autoinst-distri-opensuse) into the product repository (e.g. cobbler) adding own tests on top of it. But this sounds rather hacky and inconvenient to use. So having a light test distribution which can simply be referred to as "base test distribution" seems a desirable for this use-case.</li>
</ul>
<p>These points are likely the tricky part. Triggering the test execution itself is likely not that hard. Instead of writing a "polling bot" I would add an API route in openQA which can be added as GitHub hook.</p>
openQA Project - coordination #48641: [epic] Trigger openQA tests in pull requests of any product github pull requesthttps://progress.opensuse.org/issues/48641?journal_id=2922862020-04-12T21:24:13Zokurzokurz@suse.com
<ul></ul><p>mkittler wrote:</p>
<blockquote>
<ul>
<li>This use-case is also impaired by the inability to use custom needles (<a href="https://progress.opensuse.org/issues/56789" class="external">https://progress.opensuse.org/issues/56789</a> <a class="issue tracker-4 status-1 priority-3 priority-lowest child" title="action: New needles from git repository not working with openqa-clone-custom-git-refspec (New)" href="https://progress.opensuse.org/issues/56789">#56789</a>).</li>
</ul>
</blockquote>
<p>Maybe we can still consider any need for needles an independant requirement of the part about triggering test code which I find more important.</p>
<p>Regarding the "hacky approach" or "base test distribution" I think we have a ticket for having something like a "linux" middleware between os-autoinst and os-autoinst-distri-opensuse that can be extracted but for now and to have a proof-of-concept on which to improve upon I recommend the "hacky" approach</p>
<blockquote>
<p>[…] I would add an API route in openQA which can be added as GitHub hook.</p>
</blockquote>
<p>What would you add as an API route here?</p>
openQA Project - coordination #48641: [epic] Trigger openQA tests in pull requests of any product github pull requesthttps://progress.opensuse.org/issues/48641?journal_id=3152372020-07-28T11:31:28Zokurzokurz@suse.com
<ul><li><strong>Target version</strong> set to <i>Ready</i></li></ul> openQA Project - coordination #48641: [epic] Trigger openQA tests in pull requests of any product github pull requesthttps://progress.opensuse.org/issues/48641?journal_id=3494922020-11-11T09:51:41Zokurzokurz@suse.com
<ul><li><strong>Related to</strong> <i><a class="issue tracker-6 status-1 priority-4 priority-default child parent" href="/issues/77698">coordination #77698</a>: [epic] synchronous qemu based system level test in pull request CI runs, e.g. standalone isotovideo or openQA tests</i> added</li></ul> openQA Project - coordination #48641: [epic] Trigger openQA tests in pull requests of any product github pull requesthttps://progress.opensuse.org/issues/48641?journal_id=3522702020-11-19T16:18:11Zokurzokurz@suse.com
<ul><li><strong>Subject</strong> changed from <i>Trigger openQA tests in pull requests of any product github pull request</i> to <i>[epic] Trigger openQA tests in pull requests of any product github pull request</i></li><li><strong>Status</strong> changed from <i>New</i> to <i>Blocked</i></li><li><strong>Assignee</strong> set to <i>okurz</i></li></ul><p>waiting for <a class="issue tracker-6 status-1 priority-4 priority-default child parent" title="coordination: [epic] synchronous qemu based system level test in pull request CI runs, e.g. standalone isotovid... (New)" href="https://progress.opensuse.org/issues/77698">#77698</a> first</p>
openQA Project - coordination #48641: [epic] Trigger openQA tests in pull requests of any product github pull requesthttps://progress.opensuse.org/issues/48641?journal_id=4051452021-05-07T11:14:53Zokurzokurz@suse.com
<ul><li><strong>Target version</strong> changed from <i>Ready</i> to <i>future</i></li></ul><p><a href="https://progress.opensuse.org/issues/92022#note-4" class="external">https://progress.opensuse.org/issues/92022#note-4</a></p>
openQA Project - coordination #48641: [epic] Trigger openQA tests in pull requests of any product github pull requesthttps://progress.opensuse.org/issues/48641?journal_id=6151342023-03-20T13:30:29Zszarate
<ul><li><strong>Related to</strong> <i><a class="issue tracker-4 status-3 priority-4 priority-default closed" href="/issues/124173">action #124173</a>: [qe-core] Create status badges for verification runs</i> added</li></ul> openQA Project - coordination #48641: [epic] Trigger openQA tests in pull requests of any product github pull requesthttps://progress.opensuse.org/issues/48641?journal_id=6278752023-04-27T17:27:48Zokurzokurz@suse.com
<ul><li><strong>Target version</strong> changed from <i>future</i> to <i>Ready</i></li></ul> openQA Project - coordination #48641: [epic] Trigger openQA tests in pull requests of any product github pull requesthttps://progress.opensuse.org/issues/48641?journal_id=6358362023-05-23T12:29:40Zmkittlermarius.kittler@suse.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/635836/diff?detail_id=597085">diff</a>)</li></ul> openQA Project - coordination #48641: [epic] Trigger openQA tests in pull requests of any product github pull requesthttps://progress.opensuse.org/issues/48641?journal_id=6376512023-05-30T11:34:09Zmkittlermarius.kittler@suse.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/637651/diff?detail_id=598654">diff</a>)</li></ul><p>I've extended the "Further ideas" section because we've noticed that the webhook-based approach is not yet applicable for repos like the openQA-in-openQA test.</p>
openQA Project - coordination #48641: [epic] Trigger openQA tests in pull requests of any product github pull requesthttps://progress.opensuse.org/issues/48641?journal_id=6395562023-06-05T14:04:33Zmkittlermarius.kittler@suse.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/639556/diff?detail_id=600235">diff</a>)</li></ul> openQA Project - coordination #48641: [epic] Trigger openQA tests in pull requests of any product github pull requesthttps://progress.opensuse.org/issues/48641?journal_id=6427792023-06-13T14:39:50Zokurzokurz@suse.com
<ul></ul><p>okurz wrote:</p>
<blockquote>
<p>[…]</p>
<ol>
<li>Allow to compute the HDD image path dynamically, e.g. by providing a regex which is matched against existing assets on the openQA instance. This would allow utilizing an existing image in a dynamic way (e.g. to use the image created by the installation job for the most latest Tumbleweed snapshot as the openQA-in-openQA test does).</li>
</ol>
</blockquote>
<p>For now I don't think this is necessary as one can use a static link to an appliance, e.g. <a href="http://download.opensuse.org/tumbleweed/appliances/openSUSE-Tumbleweed-Minimal-VM.x86_64-kvm-and-xen.qcow2" class="external">http://download.opensuse.org/tumbleweed/appliances/openSUSE-Tumbleweed-Minimal-VM.x86_64-kvm-and-xen.qcow2</a></p>
openQA Project - coordination #48641: [epic] Trigger openQA tests in pull requests of any product github pull requesthttps://progress.opensuse.org/issues/48641?journal_id=6429682023-06-14T06:54:05Zlivdywanliv.dywan@suse.com
<ul></ul><p>okurz wrote:</p>
<blockquote>
<p>okurz wrote:</p>
<blockquote>
<p>[…]</p>
<ol>
<li>Allow to compute the HDD image path dynamically, e.g. by providing a regex which is matched against existing assets on the openQA instance. This would allow utilizing an existing image in a dynamic way (e.g. to use the image created by the installation job for the most latest Tumbleweed snapshot as the openQA-in-openQA test does).</li>
</ol>
</blockquote>
<p>For now I don't think this is necessary as one can use a static link to an appliance, e.g. <a href="http://download.opensuse.org/tumbleweed/appliances/openSUSE-Tumbleweed-Minimal-VM.x86_64-kvm-and-xen.qcow2" class="external">http://download.opensuse.org/tumbleweed/appliances/openSUSE-Tumbleweed-Minimal-VM.x86_64-kvm-and-xen.qcow2</a></p>
</blockquote>
<p>How? This is an external URL. You're quoting a use case for re-using the image generated by a job.</p>
openQA Project - coordination #48641: [epic] Trigger openQA tests in pull requests of any product github pull requesthttps://progress.opensuse.org/issues/48641?journal_id=6430312023-06-14T08:04:21Zokurzokurz@suse.com
<ul></ul><p>cdywan wrote:</p>
<blockquote>
<p>How? This is an external URL. You're quoting a use case for re-using the image generated by a job.</p>
</blockquote>
<p>There was never a strong need to re-use an openQA generated image though.</p>
openQA Project - coordination #48641: [epic] Trigger openQA tests in pull requests of any product github pull requesthttps://progress.opensuse.org/issues/48641?journal_id=6730972023-09-14T19:59:28Zokurzokurz@suse.com
<ul><li><strong>Target version</strong> changed from <i>Ready</i> to <i>Tools - Next</i></li></ul> openQA Project - coordination #48641: [epic] Trigger openQA tests in pull requests of any product github pull requesthttps://progress.opensuse.org/issues/48641?journal_id=6879862023-10-18T13:00:35Zokurzokurz@suse.com
<ul><li><strong>Tracker</strong> changed from <i>action</i> to <i>coordination</i></li></ul> openQA Project - coordination #48641: [epic] Trigger openQA tests in pull requests of any product github pull requesthttps://progress.opensuse.org/issues/48641?journal_id=6879892023-10-18T13:00:44Zokurzokurz@suse.com
<ul><li><strong>Subtask</strong> <i>#138203</i> added</li></ul> openQA Project - coordination #48641: [epic] Trigger openQA tests in pull requests of any product github pull requesthttps://progress.opensuse.org/issues/48641?journal_id=7354522023-11-17T13:39:26Zokurzokurz@suse.com
<ul><li><strong>Subtask</strong> <i>#150992</i> added</li></ul> openQA Project - coordination #48641: [epic] Trigger openQA tests in pull requests of any product github pull requesthttps://progress.opensuse.org/issues/48641?journal_id=7427052023-12-07T08:00:08Zokurzokurz@suse.com
<ul><li><strong>Subtask</strong> <i>#152170</i> added</li></ul> openQA Project - coordination #48641: [epic] Trigger openQA tests in pull requests of any product github pull requesthttps://progress.opensuse.org/issues/48641?journal_id=7480972023-12-27T10:11:33Zokurzokurz@suse.com
<ul><li><strong>Subtask</strong> <i>#152939</i> added</li></ul> openQA Project - coordination #48641: [epic] Trigger openQA tests in pull requests of any product github pull requesthttps://progress.opensuse.org/issues/48641?journal_id=7513152024-01-12T08:22:49Zokurzokurz@suse.com
<ul><li><strong>Subtask</strong> <i>#153460</i> added</li></ul> openQA Project - coordination #48641: [epic] Trigger openQA tests in pull requests of any product github pull requesthttps://progress.opensuse.org/issues/48641?journal_id=7647492024-02-17T18:18:53Zszarate
<ul></ul><p><a class="user active user-mention" href="https://progress.opensuse.org/users/17668">@okurz</a>, I'm having a hard time trying to understand the next step for this story. What are the next three steps to move this forward? (regardless of when, but I'm looking for order of steps, represented in tickets)</p>
<p>Also, because I don't really want to reopen <a class="issue tracker-4 status-3 priority-3 priority-lowest closed child" title="action: [timeboxed][spike solution:20h] openQA tests in pull requests to github.com/os-autoinst/os-autoin... (Resolved)" href="https://progress.opensuse.org/issues/150992">#150992</a>:</p>
<p>Trying to understand the failure rate of <a class="issue tracker-4 status-3 priority-3 priority-lowest closed child" title="action: [timeboxed][spike solution:20h] openQA tests in pull requests to github.com/os-autoinst/os-autoin... (Resolved)" href="https://progress.opensuse.org/issues/150992">#150992</a> and other consequences in the context of recent <a href="https://suse.slack.com/archives/C02CANHLANP/p1708034421966699?thread_ts=1708001470.886389&cid=C02CANHLANP" class="external">slack chat</a>, I came to notice at least one 503 (bad gateway), which reminds me of the problems I already mentioned to you, while discussing <a class="issue tracker-4 status-1 priority-4 priority-default child" title="action: Batch update for jobs (New)" href="https://progress.opensuse.org/issues/154798">#154798</a> </p>
<blockquote>
<p>That sleep 1, is needed, as otherwise the webUI will return 503's constantly</p>
</blockquote>
<p>And before moving any further, somehow that has to be addressed, which brings me again to the question of how we track usage statistics and monitor the web UI health. iirc either mojolicious ui that was showing some stats, <a class="user active user-mention" href="https://progress.opensuse.org/users/23018">@kraih</a> you recall maybe or how we solved the similar problem we were having years ago, I think it was by incrementing something with the webui workers/processes and so on?<br>
<a href="https://github.com/os-autoinst/os-autoinst-distri-opensuse/actions/runs/7666441613/job/20894254516#step:5:143" class="external">https://github.com/os-autoinst/os-autoinst-distri-opensuse/actions/runs/7666441613/job/20894254516#step:5:143</a></p>
openQA Project - coordination #48641: [epic] Trigger openQA tests in pull requests of any product github pull requesthttps://progress.opensuse.org/issues/48641?journal_id=7648002024-02-18T11:59:02Zokurzokurz@suse.com
<ul></ul><p>szarate wrote in <a href="#note-25">#note-25</a>:</p>
<blockquote>
<p><a class="user active user-mention" href="https://progress.opensuse.org/users/17668">@okurz</a>, I'm having a hard time trying to understand the next step for this story. What are the next three steps to move this forward? (regardless of when, but I'm looking for order of steps, represented in tickets)</p>
</blockquote>
<p>The next steps are the subtasks in this ticket here, e.g. <a class="issue tracker-4 status-3 priority-5 priority-high3 closed child" title="action: Trigger openQA tests mentioned in github description as part of CI size:M (Resolved)" href="https://progress.opensuse.org/issues/130934">#130934</a> and <a class="issue tracker-4 status-3 priority-3 priority-lowest closed child" title="action: Find "last build" of a product over API size:M (Resolved)" href="https://progress.opensuse.org/issues/152939">#152939</a> currently in the backlog.<br>
After those the other open subtasks or what comes up based on our experiences with those from production.</p>
<blockquote>
<p>Also, because I don't really want to reopen <a class="issue tracker-4 status-3 priority-3 priority-lowest closed child" title="action: [timeboxed][spike solution:20h] openQA tests in pull requests to github.com/os-autoinst/os-autoin... (Resolved)" href="https://progress.opensuse.org/issues/150992">#150992</a>:<br>
[...] I came to notice at least one 503 (bad gateway)</p>
</blockquote>
<p>Yes, that's a known behavior. "Bad gateway" is a bit misleading but hard to avoid. In this context here it shouldn't ever be a problem as there is retrying and the final result is monitored for anyway.</p>
<blockquote>
<p>which reminds me of the problems I already mentioned to you, while discussing <a class="issue tracker-4 status-1 priority-4 priority-default child" title="action: Batch update for jobs (New)" href="https://progress.opensuse.org/issues/154798">#154798</a> </p>
</blockquote>
<p>Yes, I would like to solve such 503's as part of that</p>
openQA Project - coordination #48641: [epic] Trigger openQA tests in pull requests of any product github pull requesthttps://progress.opensuse.org/issues/48641?journal_id=7665042024-02-22T06:45:11Zszarate
<ul></ul><p>okurz wrote in <a href="#note-26">#note-26</a>:</p>
<blockquote>
<p>szarate wrote in <a href="#note-25">#note-25</a>:</p>
<blockquote>
<p><a class="user active user-mention" href="https://progress.opensuse.org/users/17668">@okurz</a>, I'm having a hard time trying to understand the next step for this story. What are the next three steps to move this forward? (regardless of when, but I'm looking for order of steps, represented in tickets)</p>
</blockquote>
<p>The next steps are the subtasks in this ticket here, e.g. <a class="issue tracker-4 status-3 priority-5 priority-high3 closed child" title="action: Trigger openQA tests mentioned in github description as part of CI size:M (Resolved)" href="https://progress.opensuse.org/issues/130934">#130934</a> and <a class="issue tracker-4 status-3 priority-3 priority-lowest closed child" title="action: Find "last build" of a product over API size:M (Resolved)" href="https://progress.opensuse.org/issues/152939">#152939</a> currently in the backlog.<br>
After those the other open subtasks or what comes up based on our experiences with those from production.</p>
<blockquote>
<p>Also, because I don't really want to reopen <a class="issue tracker-4 status-3 priority-3 priority-lowest closed child" title="action: [timeboxed][spike solution:20h] openQA tests in pull requests to github.com/os-autoinst/os-autoin... (Resolved)" href="https://progress.opensuse.org/issues/150992">#150992</a>:<br>
[...] I came to notice at least one 503 (bad gateway)</p>
</blockquote>
<p>Yes, that's a known behavior. "Bad gateway" is a bit misleading but hard to avoid. In this context here it shouldn't ever be a problem as there is retrying and the final result is monitored for anyway.</p>
<blockquote>
<p>which reminds me of the problems I already mentioned to you, while discussing <a class="issue tracker-4 status-1 priority-4 priority-default child" title="action: Batch update for jobs (New)" href="https://progress.opensuse.org/issues/154798">#154798</a> </p>
</blockquote>
<p>Yes, I would like to solve such 503's as part of that<br>
Much clear now <3 appreciate the clarity</p>
</blockquote>