Project

General

Profile

Actions

action #128360

closed

coordination #58184: [saga][epic][use case] full version control awareness within openQA

coordination #48641: [epic] Trigger openQA tests in pull requests of any product github pull request

Supporting fork based development model size:M

Added by okurz 12 months ago. Updated 10 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Feature requests
Target version:
Start date:
Due date:
% Done:

0%

Estimated time:

Description

Supporting fork based development model

Motivation

We found a nice way to trigger openQA tests within GitHub workflows but that needs openQA credentials which are for example for os-autoinst-distri-openQA only provided in the base repository but not available to forks. We must find a way to support the fork based development model, what people usually do in GitHub.

Acceptance criteria

  • AC1: PRs like https://github.com/os-autoinst/os-autoinst-distri-example/pull/19 can successfully execute openQA based CI checks.
  • AC2: The normal openQA cleanup strategy still applies, i.e. jobs are still cleaned up as normal
  • AC3: openqa.opensuse.org+os-autoinst-distri-example support this workflow from github fork repositories
  • AC4: Documentation covers the setup and use of this feature
  • AC5: The implementation is generally reusable for any openQA instance and any openQA test distribution

Suggestions

Out of scope

  • It does not actually matter which specific openQA jobs are triggered, just any that refers to the test distribution
  • It's ok if changes to scenario-definitions.yaml in the PR are not effective for the spawned jobs before merge

Related issues 3 (0 open3 closed)

Blocks openQA Project - action #126950: [openQA-in-openQA] openQA tests in pull requests to github.com/os-autoinst/os-autoinst-distri-openQA/ size:MResolvedjbaier_cz2023-03-30

Actions
Copied from openQA Project - action #127949: [spike][timeboxed:20h] Research native GitHub for running openQA tests as CI checks size:MResolvedmkittler2023-04-19

Actions
Copied to openQA Project - action #129730: Adapt http://open.qa/docs/#_running_openqa_jobs_as_ci_checks for the use of github pull_request_target size:MResolvedmkittler

Actions
Actions #1

Updated by okurz 12 months ago

  • Copied from action #127949: [spike][timeboxed:20h] Research native GitHub for running openQA tests as CI checks size:M added
Actions #2

Updated by mkittler 12 months ago

Somehow I'm not sure whether #127949 will help with that. At least for Codecov (which has a better integration with GitHub) the cases where a token is needed are still problematic (because also then the token is not there when creating a PR from a fork). This just works because Codecov doesn't require a token for public projects¹. Maybe we should provide an openQA instance that does not require a token (or in our case API credentials) as well? Or provide some limited public access on an existing instance just for that?


¹ It currently kind of does but that's considered a bug, see #128129.

Actions #3

Updated by okurz 12 months ago

  • Status changed from Blocked to New
  • Assignee deleted (okurz)
  • Priority changed from Low to Normal

Ok then let's not block on that

Actions #4

Updated by mkittler 12 months ago

If it was possible to make use of the API credentials from any fork then anybody could spawn any kind of openQA job on o3. If that is acceptable we might as well allow anybody to spawn any kind of openQA job on o3 without providing API credentials. This way we'd avoid the problem of making the credentials available to forked repos.

Supposedly this kind of access is nothing we should introduce by default. We could add an openQA setting to allow anonymous job posting and enable it only on o3.

However, o3 is also where a lot of production workload is executed. Maybe we don't want to mix that with this kind of development workload. We could setup a 2nd public instance for this.

Actions #5

Updated by okurz 12 months ago

mkittler wrote:

If it was possible to make use of the API credentials from any fork then anybody could spawn any kind of openQA job on o3.

Well, for unknown contributors openQA still asks the target repo owners for approval to run CI jobs. So whatever is working for GHA and OBS build checks we should still be able to do with openQA. How does it work for OBS build checks?

Actions #6

Updated by mkittler 12 months ago

Well, for unknown contributors openQA still asks the target …

You mean GitHub still asks … ?

How does it work for OBS build checks?

Good question. After a brief look I'd say it works for OBS because it uses a web hook. So GitHub itself (and not some possibly user-defined code running within a CI runner) makes an HTTP request to OBS using some secret stored in the repo config. Then OBS reacts to it. I suppose we could replace OBS with openQA. So we needed to implement a route handling the web hook on the openQA side. This would basically mean we'd implement #127949 first it (unlike I suspected before) it would help us with this. So maybe raise the prio of that ticket to be at least the as high as the prio of this one?

Actions #7

Updated by jbaier_cz 12 months ago

  • Blocks action #126950: [openQA-in-openQA] openQA tests in pull requests to github.com/os-autoinst/os-autoinst-distri-openQA/ size:M added
Actions #8

Updated by mkittler 12 months ago

  • Assignee set to mkittler
Actions #9

Updated by mkittler 12 months ago

  • Status changed from New to Blocked

I'll continue with #127949 which will hopefully also cover this ticket.

Actions #10

Updated by mkittler 12 months ago

Considering the approach we've taking with #128129 we could also do the same here. So we'd make the "token" public. I suppose that would mean adding a new admin level in openQA so one can have an API key/secret that only allows posting an ISO and also only with constrained parameters (like _GROUP=0 and BUILD and TEST would automatically get some prefix/appendix so these jobs would never interfere with normal dashboards). We might also want to disallow the download feature (although it is likely not problematic because it uses a whitelist anyways).

Actions #11

Updated by okurz 12 months ago

mkittler wrote:

Considering the approach we've taking with #128129 we could also do the same here. So we'd make the "token" public. I suppose that would mean adding a new admin level in openQA so one can have an API key/secret that only allows posting an ISO and also only with constrained parameters (like _GROUP=0 and BUILD and TEST would automatically get some prefix/appendix so these jobs would never interfere with normal dashboards).

But then anyone with the token could DoS us by triggering endless number of tests, right?

Actions #12

Updated by mkittler 11 months ago

Yes - so it is definitely not ideal and I prefer to continue with #127949. This one actually also allows one to flood an openQA instance with tons of jobs. One just has to push a scenario definitions YAML file that generates tons of jobs. I suppose we want to limit this before actually configuring the webhook on a production instance. However, limiting this shouldn't be a big deal.

Actions #13

Updated by mkittler 11 months ago

  • Status changed from Blocked to In Progress
Actions #14

Updated by openqa_review 11 months ago

  • Due date set to 2023-05-30

Setting due date based on mean cycle time of SUSE QE Tools

Actions #15

Updated by mkittler 11 months ago

  • Description updated (diff)
Actions #16

Updated by mkittler 11 months ago

I've been working on this today and I've just updated #127949#note-10. The pending PR: https://github.com/os-autoinst/openQA/pull/5133 (I'm using this comment because then there's only one place for what's left TODO.)

When at least the cleanup has been concluded we can consider this good enough for resolving this ticket here and create follow-up tickets for the remaining improvements.

Actions #17

Updated by okurz 11 months ago

  • Copied to action #129730: Adapt http://open.qa/docs/#_running_openqa_jobs_as_ci_checks for the use of github pull_request_target size:M added
Actions #18

Updated by okurz 11 months ago

  • Subject changed from Supporting fork based development model to Supporting fork based development model size:M
  • Description updated (diff)
Actions #19

Updated by mkittler 11 months ago

  • Status changed from In Progress to Feedback

I've tested whether https://github.com/os-autoinst/openQA/pull/5133 works for real via https://github.com/os-autoinst/os-autoinst-distri-example/pull/22 and it does. So both PRs can be merged. Then I can configure OSD and the example repository accordingly.

I've just been adding further ideas that I see out of scope of this ticket but still generally relevant to #48641.

Actions #20

Updated by mkittler 11 months ago

I've added the webhook for o3 on https://github.com/os-autoinst/os-autoinst-distri-example/pull/22 and it works. It is using the openQA user "webhooks" with its own key/secret pair.

Actions #21

Updated by mkittler 11 months ago

  • Status changed from Feedback to Resolved

The PR for the example distri that disables the old approach has just been merged. With that I'd conclude this ticket.

For further improvements, see https://progress.opensuse.org/issues/48641#Further-ideas.

Actions #22

Updated by okurz 10 months ago

  • Due date deleted (2023-05-30)
Actions

Also available in: Atom PDF