QA - coordination #91646: [saga][epic] SUSE Maintenance QA workflows with fully automated testing, approval and release
Make qem-dashboard a proper public open source project size:M
I wonder why I even need to write this, we are working at SUSE, right? :) (If still not convinced see opensource.suse.com/ and https://intra.suse.net/company/living-our-values-everyday/ and if you don't want to talk to me about this then talk to others, e.g. bzoltan1)
- AC1: https://gitlab.suse.de/opensuse/qem-dashboard/ is developed in a public place
- AC2: Deployments to SUSE internal ressources in CI pipelines still work
- Consider openqa_review as an existing example using both git clone in a gitlab pipeline and packages built
- git fetch from gitlab
I also prefer making everything open source from the start. Main reasons not to do so here were that, a) the project was just a proof of concept until recently, b) openQABot and SMELT are closed source, c) ease of deployment from our gitlab instance, since Jan already has ansible set up there for pipelines. Aside from the code being pretty much useless without having a SMELT instance, the main concern will be deployment from GitHub to the dashboard machine. Oh, and of course naming the project. :)
As I understand it there's nothing blocking us here besides finding a new name. If CI can't easily be setup on GitHub I wouldn't consider that a problem, since this is about development and not about production deployments.
Deployment to production is currently triggered by commits to the repo (after all tests pass in CI).
#11 Updated by jbaier_cz about 2 months ago
If CI can't easily be setup on GitHub I wouldn't consider that a problem, since this is about development and not about production deployments.
Well, I would consider not being able to deploy a major usability issue. What is the development for if you cannot use it?
Anyways, in the current state the deployment is orchestrated via Ansible playbook (which actually just prepares some service files and does a git checkout). This is managed in a separate gitlab project qa-maintenance/qamops and will work as long as we will have read access to the qem-dashboard code repository (which should be OK for a public GitHub repository). The only think here would be to trigger the deployment pipeline itself. Currently, we are using multi-project pipelines and we are triggering the downstream pipeline via CI inside the qem-dashboard pipeline. I see several options here, each with its own ups and downs (ordered by simplicity of the solution):
Triggering pipeline through API directly from GitHub
con: we need gitlab.suse.de to be accessible from GitHub for the webhook to work
Pull mirroring from the remote and triggering pipeline for updates
con: pull mirroring is a premium feature in GitLab
Running the deployment in a scheduled pipeline
con: not event-based, the deployed version will not be always up-to-date
Push mirroring to GitHub
con: the development will still be done in GitLab
Add OBS/IBS to the mix and use workflow integration to build a package after GitHub push and trigger package installation after successful build
con: complicated setup
#12 Updated by okurz about 2 months ago
- Running the deployment in a scheduled pipeline con: not event-based, the deployed version will not be always up-to-date
I recommend to go with this and run it every couple of minutes. I see no harm in doing that every like 10 minutes with an early-abort if the current git version matches the last deployed one.
#13 Updated by jbaier_cz about 2 months ago
The con here is a lot of wasted runs (with tight schedules, like every 10 minutes). The early abort is not very feasible without rewriting the current playbook as the git operation is at the very end (and effectively cannot be done before you make sure the project directory exists and git binary is installed; so we are talking here only about skipping systemd unit files creation). Running the playbook often has also some other drawbacks.
#16 Updated by jbaier_cz about 2 months ago
The con here is a lot of wasted runs (with tight schedules, like every 10 minutes).
As long as the pipeline can also be triggered manually, even just running it every 24 hours with the schedule would be fine.
Yes, once a day and manual option to run the pipeline seems fine to me as well.
#18 Updated by jbaier_cz about 2 months ago
Deployment book updated with https://gitlab.suse.de/qa-maintenance/qamops/-/commit/c90aa3e25ed8ca04a2d89f369b23d843804745fa
Now we just need to update the original pipeline add the manual trigger and scheduled pipeline for deploy.