https://progress.opensuse.org/https://progress.opensuse.org/themes/openSUSE/favicon/favicon.ico?15829177842018-04-09T07:12:33ZopenSUSE Project Management ToolopenQA Project - action #34486: database of "test cases" or how to search for tests we have in openQAhttps://progress.opensuse.org/issues/34486?journal_id=1103682018-04-09T07:12:33Zszarate
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/110368/diff?detail_id=110281">diff</a>)</li></ul><p><a class="user active user-mention" href="https://progress.opensuse.org/users/17668">@okurz</a> by:</p>
<pre><code>**AC2:** The user is one and only one question, that is the search term, nothing else
</code></pre>
<p>You mean that the user has one and only one question? :).</p>
<p>I guess that for things like this, things would need to be indexed, and that filtering (eg. by build or distri) would be only for names of test suites and scenarios.</p>
openQA Project - action #34486: database of "test cases" or how to search for tests we have in openQAhttps://progress.opensuse.org/issues/34486?journal_id=1108092018-04-09T21:53:50Zokurzokurz@suse.com
<ul></ul><p>szarate wrote:</p>
<blockquote>
<p><a class="user active user-mention" href="https://progress.opensuse.org/users/17668">@okurz</a> by:</p>
<pre><code>**AC2:** The user is one and only one question, that is the search term, nothing else
</code></pre>
<p>You mean that the user has one and only one question? :).</p>
</blockquote>
<p>I am assuming that the user does not know if he means a testsuite, test module, test step, scenario, product, etc. So I envision the default to be simplistic, like google once was, just a single text field. Ok, you can provide buttons below or above the search bar for advanced stuff, help, customization but let's start simple :)</p>
<blockquote>
<p>I guess that for things like this, things would need to be indexed, and that filtering (eg. by build or distri) would be only for names of test suites and scenarios.</p>
</blockquote>
<p>What do you want to index? We have git repos -> already indexed, and we have database tables for test suites, so go with that :)</p>
openQA Project - action #34486: database of "test cases" or how to search for tests we have in openQAhttps://progress.opensuse.org/issues/34486?journal_id=1110582018-04-10T08:32:28ZEDiGiacintoedigiacinto@suse.com
<ul></ul><p>okurz wrote:</p>
<blockquote>
<p>szarate wrote:</p>
<blockquote>
<p><a class="user active user-mention" href="https://progress.opensuse.org/users/17668">@okurz</a> by:</p>
<pre><code>**AC2:** The user is one and only one question, that is the search term, nothing else
</code></pre>
<p>You mean that the user has one and only one question? :).</p>
</blockquote>
<p>I am assuming that the user does not know if he means a testsuite, test module, test step, scenario, product, etc. So I envision the default to be simplistic, like google once was, just a single text field. Ok, you can provide buttons below or above the search bar for advanced stuff, help, customization but let's start simple :)</p>
</blockquote>
<p>While i can agree with being simplistic i don't share re-implementing google.</p>
<blockquote>
<blockquote>
<p>I guess that for things like this, things would need to be indexed, and that filtering (eg. by build or distri) would be only for names of test suites and scenarios.</p>
</blockquote>
<p>What do you want to index? We have git repos -> already indexed, and we have database tables for test suites, so go with that :)</p>
</blockquote>
<p>What you propose basically is a killer (as in resources) api endpoint that does a complex query - including search in a git subtree. That's why you might want an index where to search data instead of doing the search directly whenever a user request it.</p>
openQA Project - action #34486: database of "test cases" or how to search for tests we have in openQAhttps://progress.opensuse.org/issues/34486?journal_id=1112142018-04-10T09:03:55Zrpalethorperichard.palethorpe@suse.com
<ul></ul><p>Oliver just try screen scraping the whole of OpenQA and index every page with Xapian (see <a href="https://gitlab.suse.de/alnovak/fast-bzsearch/tree/master" class="external">https://gitlab.suse.de/alnovak/fast-bzsearch/tree/master</a>). As you are basically asking for a totally unstructured text search which OpenQA has no infrastructure to support, Xapian is really fast and you already know how to screen scrape OpenQA.</p>
openQA Project - action #34486: database of "test cases" or how to search for tests we have in openQAhttps://progress.opensuse.org/issues/34486?journal_id=1301412018-06-15T20:55:07Zokurzokurz@suse.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/130141/diff?detail_id=131461">diff</a>)</li></ul> openQA Project - action #34486: database of "test cases" or how to search for tests we have in openQAhttps://progress.opensuse.org/issues/34486?journal_id=1668172018-11-20T20:09:04Zokurzokurz@suse.com
<ul><li><strong>Related to</strong> <i><a class="issue tracker-4 status-3 priority-3 priority-lowest closed behind-schedule" href="/issues/38024">action #38024</a>: [functional][y] Add ability to search for a module occurrences in all the tests of job group</i> added</li></ul> openQA Project - action #34486: database of "test cases" or how to search for tests we have in openQAhttps://progress.opensuse.org/issues/34486?journal_id=2950732020-04-22T11:48:07Zokurzokurz@suse.com
<ul><li><strong>Has duplicate</strong> <i><a class="issue tracker-4 status-6 priority-4 priority-default closed" href="/issues/65954">action #65954</a>: Create a way to check which jobs contain a test</i> added</li></ul> openQA Project - action #34486: database of "test cases" or how to search for tests we have in openQAhttps://progress.opensuse.org/issues/34486?journal_id=2950762020-04-22T11:49:54Zokurzokurz@suse.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/295076/diff?detail_id=292001">diff</a>)</li></ul> openQA Project - action #34486: database of "test cases" or how to search for tests we have in openQAhttps://progress.opensuse.org/issues/34486?journal_id=2966302020-04-28T10:12:59Zokurzokurz@suse.com
<ul><li><strong>Related to</strong> <i><a class="issue tracker-6 status-1 priority-4 priority-default parent" href="/issues/64126">coordination #64126</a>: [qe-core][epic] Identify packages that have automated indirect testing or that have sufficient build time test suite</i> added</li></ul> openQA Project - action #34486: database of "test cases" or how to search for tests we have in openQAhttps://progress.opensuse.org/issues/34486?journal_id=2999802020-05-13T14:31:36Zmkittlermarius.kittler@suse.com
<ul></ul><p>I'm really wondering about the idea to search the file system. Beside the problem pointed out by <a class="user active user-mention" href="https://progress.opensuse.org/users/25026">@EDiGiacinto</a> I'm wondering what you want to do once you've found a file in the repository. There's no relation from such a file to a concrete openQA job. But openQA jobs are what the results of that search are supposed to be, right? The acceptance criteria should clarify what kind of results the search is supposed to reveal. I assume visitors of the openQA page would expect to find concrete openQA jobs and if they wanted to search for particular files within a certain test distribution they would use a different kind of search in the first place.</p>
openQA Project - action #34486: database of "test cases" or how to search for tests we have in openQAhttps://progress.opensuse.org/issues/34486?journal_id=3000072020-05-13T17:13:48Zlivdywanliv.dywan@suse.com
<ul></ul><p>mkittler wrote:</p>
<blockquote>
<p>I'm really wondering about the idea to search the file system. Beside the problem pointed out by <a class="user active user-mention" href="https://progress.opensuse.org/users/25026">@EDiGiacinto</a> I'm wondering what you want to do once you've found a file in the repository. There's no relation from such a file to a concrete openQA job. But openQA jobs are what the results of that search are supposed to be, right? The acceptance criteria should clarify what kind of results the search is supposed to reveal. I assume visitors of the openQA page would expect to find concrete openQA jobs and if they wanted to search for particular files within a certain test distribution they would use a different kind of search in the first place.</p>
</blockquote>
<p>Not sure where searching the file system comes from. But one of the duplicates was based on this real use case:</p>
<ul>
<li>Allison Average looks at <a class="issue tracker-4 status-3 priority-4 priority-default closed" title="action: Ensure apache timeouts are higher than openQA internal ones (was: Latest version of openQA webui ... (Resolved)" href="https://progress.opensuse.org/issues/63244">#63244</a>.</li>
<li>The example URLs are dead.</li>
<li>So she wants to find jobs using <code>qa_run</code>.</li>
</ul>
openQA Project - action #34486: database of "test cases" or how to search for tests we have in openQAhttps://progress.opensuse.org/issues/34486?journal_id=3008682020-05-17T18:28:14Zokurzokurz@suse.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/300868/diff?detail_id=297844">diff</a>)</li></ul><p>mkittler wrote:</p>
<blockquote>
<p>[…] There's no relation from such a file to a concrete openQA job</p>
</blockquote>
<p>Correct. But imagine the following: A QA automation engineer wonders if any test coverage for nginx exists. So you might call <a href="https://openqa.opensuse.org/tests?match=nginx" class="external">https://openqa.opensuse.org/tests?match=nginx</a> and find nothing because there is no test suite or scenario that mentions "nginx" in the name.</p>
<blockquote>
<p>But openQA jobs are what the results of that search are supposed to be, right?</p>
</blockquote>
<p>No. As above, there might be no test suite or scenario that mention "nginx", but the openqa test suite "textmode" might include a test module from os-autoinst-distri-opensuse called "httpd.pm" that has <code>zypper -n in nginx && systemctl start nginx</code>. You can only find that by looking into the test distribution code, i.e. files on the file system or the mentioned search in the repo using the github search.</p>
<blockquote>
<p>The acceptance criteria should clarify what kind of results the search is supposed to reveal. I assume visitors of the openQA page would expect to find concrete openQA jobs and if they wanted to search for particular files within a certain test distribution they would use a different kind of search in the first place.</p>
</blockquote>
<p>My experience tells me that they don't. It happened multiple times that QA engineers create a pull request – after weeks worth of work invested! – to add test code for something that already exists and they were not aware. Why? Mainly because they <em>never</em> searched simply in os-autoinst-distri-opensuse for mentions of the package/component/file/service/module/feature to be tested.</p>
<p>I do not see the relation that cdywan has mentioned relating to <a class="issue tracker-4 status-3 priority-4 priority-default closed" title="action: Ensure apache timeouts are higher than openQA internal ones (was: Latest version of openQA webui ... (Resolved)" href="https://progress.opensuse.org/issues/63244">#63244</a> but the other related issues linked to this ticket here already show other examples of the same.</p>
<p>I think the acceptance criteria are not vague but generic and they should be. Still I tried to extend them now also with acceptance tests to show even more explicitly what I expect</p>
openQA Project - action #34486: database of "test cases" or how to search for tests we have in openQAhttps://progress.opensuse.org/issues/34486?journal_id=3008712020-05-17T18:38:16Zokurzokurz@suse.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/300871/diff?detail_id=297847">diff</a>)</li><li><strong>Status</strong> changed from <i>New</i> to <i>Workable</i></li></ul><p>Add more suggestions. If you think like this the ticket is feasible and well-specified but too big we can split it in sub tasks, e.g. "generic search API route /api/v1/search looking at test suites only (as a first step)"</p>
openQA Project - action #34486: database of "test cases" or how to search for tests we have in openQAhttps://progress.opensuse.org/issues/34486?journal_id=3062262020-06-09T09:00:07Zmkittlermarius.kittler@suse.com
<ul></ul><p>Ok, so this ticket is about combining different places to search but <em>not</em> being smart about the relations of the things we might find. So if "nginx" is found in a file of the test distribution that file's path is part of the results but there are still no related openQA jobs returned unless "nginx" shows up within the job settings or job module names.</p>
openQA Project - action #34486: database of "test cases" or how to search for tests we have in openQAhttps://progress.opensuse.org/issues/34486?journal_id=3063192020-06-09T12:43:21Zokurzokurz@suse.com
<ul></ul><p>yes</p>
openQA Project - action #34486: database of "test cases" or how to search for tests we have in openQAhttps://progress.opensuse.org/issues/34486?journal_id=3069342020-06-12T15:55:52Zlivdywanliv.dywan@suse.com
<ul><li><strong>Assignee</strong> set to <i>livdywan</i></li></ul> openQA Project - action #34486: database of "test cases" or how to search for tests we have in openQAhttps://progress.opensuse.org/issues/34486?journal_id=3115102020-07-03T16:57:30Zlivdywanliv.dywan@suse.com
<ul><li><strong>Status</strong> changed from <i>Workable</i> to <i>In Progress</i></li></ul><p>After having researched what's needed for this, I took a step back and pushed code for the following aspects:</p>
<ul>
<li>Search results view</li>
<li>AT3-4 - Search for Perl modules by filename</li>
<li>AT3-5 - Search Perl module contents</li>
</ul>
<p><a href="https://github.com/os-autoinst/openQA/pull/3237" class="external">https://github.com/os-autoinst/openQA/pull/3237</a></p>
openQA Project - action #34486: database of "test cases" or how to search for tests we have in openQAhttps://progress.opensuse.org/issues/34486?journal_id=3150902020-07-28T11:27:50Zokurzokurz@suse.com
<ul><li><strong>Target version</strong> set to <i>Ready</i></li></ul> openQA Project - action #34486: database of "test cases" or how to search for tests we have in openQAhttps://progress.opensuse.org/issues/34486?journal_id=3191292020-08-17T13:39:55Zlivdywanliv.dywan@suse.com
<ul></ul><p>The initial implementation of the global search UI as well as the underlying REST API got merged - we decided to put it in the <em>experimental</em> namespace for now.</p>
<p>As per discussion, I'm also proposing a follow-up to improve general responsiveness: <a href="https://github.com/os-autoinst/openQA/pull/3313" class="external">https://github.com/os-autoinst/openQA/pull/3313</a></p>
openQA Project - action #34486: database of "test cases" or how to search for tests we have in openQAhttps://progress.opensuse.org/issues/34486?journal_id=3196182020-08-19T12:17:36Zmgrifalconi
<ul></ul><p>Hello, I would like to propose the following use case.</p>
<ul>
<li>I am working with <code>mau-extratests</code> test suite</li>
<li>I know which perl modules it runs</li>
<li><p>I <strong>don't know</strong> how to find openqa tests to clone that run <code>mau-extratests</code></p></li>
<li><p>I search for <code>mau-extratests</code> (possibly filtering only for job groups results)</p></li>
<li><p>I get as result, between the others:</p></li>
</ul>
<p>The job template: <a href="https://openqa.suse.de/admin/job_templates/232" class="external">https://openqa.suse.de/admin/job_templates/232</a><br>
because it contains:</p>
<pre><code>- mau-extratests
- mau-filesystem
[..]
- mau-extratests-desktop:
</code></pre>
<p>Thanks!</p>
openQA Project - action #34486: database of "test cases" or how to search for tests we have in openQAhttps://progress.opensuse.org/issues/34486?journal_id=3196332020-08-19T13:08:51Zlivdywanliv.dywan@suse.com
<ul></ul><ul>
<li>Improvements to matches for file contents: <a href="https://github.com/os-autoinst/openQA/pull/3317" class="external">os-autoinst/openQA#3317</a></li>
<li>Skip arbitrary files in the test root: <a href="https://github.com/os-autoinst/openQA/pull/3318" class="external">os-autoinst/openQA#3318</a></li>
</ul>
openQA Project - action #34486: database of "test cases" or how to search for tests we have in openQAhttps://progress.opensuse.org/issues/34486?journal_id=3197202020-08-19T16:17:11Zlivdywanliv.dywan@suse.com
<ul></ul><p>mgrifalconi wrote:</p>
<blockquote>
<p>Hello, I would like to propose the following use case.</p>
<ul>
<li>I am working with <code>mau-extratests</code> test suite</li>
<li>I know which perl modules it runs</li>
<li>I <strong>don't know</strong> how to find openqa tests to clone that run <code>mau-extratests</code></li>
<li>I search for <code>mau-extratests</code> (possibly filtering only for job groups results)</li>
<li>I get as result, between the others:</li>
</ul>
<p>The job template: <a href="https://openqa.suse.de/admin/job_templates/232" class="external">https://openqa.suse.de/admin/job_templates/232</a><br>
because it contains:</p>
<pre><code>- mau-extratests
- mau-filesystem
[..]
- mau-extratests-desktop:
</code></pre></blockquote>
<p>Thank you for your feedback! I prepared a branch that searches job templates, which I think is what you want:</p>
<p><a href="https://github.com/os-autoinst/openQA/pull/3320" class="external">https://github.com/os-autoinst/openQA/pull/3320</a></p>
openQA Project - action #34486: database of "test cases" or how to search for tests we have in openQAhttps://progress.opensuse.org/issues/34486?journal_id=3204372020-08-24T13:35:09Zlivdywanliv.dywan@suse.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/320437/diff?detail_id=317611">diff</a>)</li></ul> openQA Project - action #34486: database of "test cases" or how to search for tests we have in openQAhttps://progress.opensuse.org/issues/34486?journal_id=3206352020-08-25T10:02:51Zlivdywanliv.dywan@suse.com
<ul></ul><p>More potentially interesting match types:</p>
<ul>
<li>Needles</li>
<li>Assets</li>
<li>Bugs</li>
<li><p>Job by job module</p></li>
<li><p>I will also propose a PR to handle the testrepo symlink to avoid redundant results.</p></li>
<li><p><a class="user active user-mention" href="https://progress.opensuse.org/users/22072">@mkittler</a> also suggested we display the type of result, maybe as an icon.</p></li>
<li><p>We should link results, e.g. a job template links to the job group.</p></li>
</ul>
openQA Project - action #34486: database of "test cases" or how to search for tests we have in openQAhttps://progress.opensuse.org/issues/34486?journal_id=3206742020-08-25T14:52:47Zlivdywanliv.dywan@suse.com
<ul></ul><p>cdywan wrote:</p>
<blockquote>
<ul>
<li>I will also propose a PR to handle the testrepo symlink to avoid redundant results.
<a href="https://github.com/os-autoinst/openQA/pull/3331" class="external">https://github.com/os-autoinst/openQA/pull/3331</a></li>
</ul>
</blockquote>
openQA Project - action #34486: database of "test cases" or how to search for tests we have in openQAhttps://progress.opensuse.org/issues/34486?journal_id=3216822020-09-01T11:25:45Zokurzokurz@suse.com
<ul></ul><p>As discussed in the weekly QA tools team meeting a next idea would be to return current job module results similar to what <a href="https://github.com/okurz/scripts/blob/master/openqa-db_query_last_use_of_module" class="external">https://github.com/okurz/scripts/blob/master/openqa-db_query_last_use_of_module</a> as mentioned in the description already provides.</p>
openQA Project - action #34486: database of "test cases" or how to search for tests we have in openQAhttps://progress.opensuse.org/issues/34486?journal_id=3220392020-09-02T09:20:00Zokurzokurz@suse.com
<ul><li><strong>Copied to</strong> <i><a class="issue tracker-6 status-1 priority-4 priority-default parent" href="/issues/70840">coordination #70840</a>: [epic] further features for the "search" feature</i> added</li></ul> openQA Project - action #34486: database of "test cases" or how to search for tests we have in openQAhttps://progress.opensuse.org/issues/34486?journal_id=3220512020-09-02T09:23:58Zokurzokurz@suse.com
<ul></ul><p>I moved out all further, additional ideas into <a class="issue tracker-6 status-1 priority-4 priority-default parent" title="coordination: [epic] further features for the "search" feature (New)" href="https://progress.opensuse.org/issues/70840">#70840</a> , please focus on adding the "job module results" only today to be able to resolve this ticket.</p>
openQA Project - action #34486: database of "test cases" or how to search for tests we have in openQAhttps://progress.opensuse.org/issues/34486?journal_id=3222462020-09-02T16:09:54Zlivdywanliv.dywan@suse.com
<ul></ul><p><a href="https://github.com/os-autoinst/openQA/pull/3353" class="external">https://github.com/os-autoinst/openQA/pull/3353</a></p>
openQA Project - action #34486: database of "test cases" or how to search for tests we have in openQAhttps://progress.opensuse.org/issues/34486?journal_id=3241962020-09-11T14:51:52Zlivdywanliv.dywan@suse.com
<ul><li><strong>Status</strong> changed from <i>In Progress</i> to <i>Feedback</i></li></ul> openQA Project - action #34486: database of "test cases" or how to search for tests we have in openQAhttps://progress.opensuse.org/issues/34486?journal_id=3249432020-09-16T08:40:38Zokurzokurz@suse.com
<ul><li><strong>Status</strong> changed from <i>Feedback</i> to <i>Resolved</i></li></ul><p>so <a href="https://github.com/os-autoinst/openQA/pull/3353" class="external">https://github.com/os-autoinst/openQA/pull/3353</a> is in and also deployed on osd now. Searching for job modules also works now, e.g. <a href="https://openqa.opensuse.org/search?q=kontact" class="external">https://openqa.opensuse.org/search?q=kontact</a></p>
<p>All the rest is left for <a class="issue tracker-6 status-1 priority-4 priority-default parent" title="coordination: [epic] further features for the "search" feature (New)" href="https://progress.opensuse.org/issues/70840">#70840</a></p>