Project

General

Profile

action #12506

openqa-scripts on osd/o3 should be deployed automatically (CD)

Added by okurz about 4 years ago. Updated 11 months ago.

Status:
Resolved
Priority:
Low
Assignee:
Target version:
Start date:
2016-06-26
Due date:
% Done:

0%

Estimated time:

Description

IMHO the "openqa-scripts" repo on gitlab should use continuous deployment to deploy automatically to both instances osd/o3. See discussion in #12370 for reasoning. This might be a good idea to do with salt if it can auto-deploy on git repo change, otherwise we should use jenkins.


Related issues

Related to openQA Tests - action #12370: toolchain/crash - test uses incorrectResolved2016-06-16

History

#1 Updated by okurz about 4 years ago

  • Related to action #12370: toolchain/crash - test uses incorrect added

#2 Updated by coolo almost 3 years ago

  • Priority changed from Normal to Low
  • Target version set to Ready

neither salt nor jenkins can do 'auto', but both can do polling :)

So you can have a crontab to git pull, just as we git pull the pillars.

#3 Updated by pevik almost 3 years ago

coolo wrote:

neither salt nor jenkins can do 'auto', but both can do polling :)

So you can have a crontab to git pull, just as we git pull the pillars.

At least jenkins has can trigger build after git push [1] [2] (no need to have a crontab) :).

[1] https://wiki.jenkins.io/display/JENKINS/Git+Plugin#GitPlugin-Pushnotificationfromrepository
[2] https://wiki.jenkins.io/display/JENKINS/GitHub+Plugin#GitHubPlugin-GitHubhooktriggerforGITScmpolling

#4 Updated by coolo almost 3 years ago

"Note: Custom Git hooks must be configured on the filesystem of the GitLab server. Only GitLab server administrators will be able to complete these tasks." - are you?

#5 Updated by okurz almost 3 years ago

  • Status changed from New to In Progress

ok. osd has direct access to gitlab.suse.de so I just added the following to /etc/cron.d/sles:

*/5    * * * *   root     cd /opt/openqa-scripts && git fetch --all && git checkout --force "origin/master"

so from now on the scripts coming from https://gitlab.suse.de/openqa/scripts are automatically deployed to osd at least. o3 has no direct access to gitlab.suse.de so we must look for an intermediate host which would do that deployment. Any suggestions which host that should be?

#6 Updated by okurz almost 2 years ago

  • Subject changed from openqa-scripts on osd/o3 should be deployed automatically (CD) to [functional][y] openqa-scripts on osd/o3 should be deployed automatically (CD)
  • Status changed from In Progress to Workable

By now I think the deployment to o3 (maybe the same for osd) should be done by a gitlab-runner on a host within the SUSE network that can access o3. I am sure without the involvement of y-team this will not move on.

#7 Updated by mkittler almost 2 years ago

  • Project changed from openQA Project to openQA Infrastructure
  • Category deleted (168)

Seems to be an infra issue.

#8 Updated by okurz almost 2 years ago

mkittler wrote:

Seems to be an infra issue.

Depends. One can argue the actual necessity for openqa-scripts is a "feature request" to openQA

#9 Updated by okurz about 1 year ago

  • Subject changed from [functional][y] openqa-scripts on osd/o3 should be deployed automatically (CD) to openqa-scripts on osd/o3 should be deployed automatically (CD)
  • Status changed from Workable to Feedback
  • Assignee set to okurz

Created an SSH keypair following https://docs.gitlab.com/ee/ssh/README.html#generating-a-new-ssh-key-pair with

ssh-keygen -t ed25519 -C o3-openqa-scripts-deployment

stored the public key on
https://gitlab.suse.de/openqa/scripts/-/settings/ci_cd
in a protected and masked variable SSH_PRIVATE_KEY and the output of

ssh-keyscan -p 2215 proxy-opensuse.suse.de

in a variable SSH_KNOWN_HOSTS .

On o3 I created a new user, added the public key, changed the folder /opt/openqa-scripts to be owned by that user (and still group root) and gave a password to the user, otherwise ssh login is denied by PAM authentication. Stored the password in my personal password manager but it should actually never be necessary

useradd -m -s /bin/false o3-openqa-scripts-deployment
(cd /opt/openqa-scripts && chmod -R g+w . && chown -R o3-openqa-scripts-deployment .)
echo 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIK+lN+JTRW9hNlC8E/1krDNYbMlKhV7p4sdvhJqN1WCn o3-openqa-scripts-deployment' >> /home/o3-openqa-scripts/.ssh/authorized_keys 
passwd o3-openqa-scripts 

tried to push with

GIT_SSH_COMMAND='ssh -i ~/id_ed25519_o3_openqa_scripts_deployment' git push o3-openqa-scripts@o3:/opt/openqa-scripts/

and then created https://gitlab.suse.de/openqa/scripts/merge_requests/355

and finally a successful pipeline job with the deployment done: https://gitlab.suse.de/openqa/scripts/-/jobs/102296

Thanks to nsinger for the nice pair-programming session.

I will keep this ticket open for some time to get feedback from any of you if we should also adopt osd in the same way or not.

#10 Updated by okurz 12 months ago

As discussed in https://gitlab.suse.de/openqa/salt-states-openqa/merge_requests/106#note_184745 decided to do the same approach for OSD: https://gitlab.suse.de/openqa/scripts/merge_requests/397 . Merged.

Created https://gitlab.suse.de/openqa/salt-states-openqa/merge_requests/202 to delete the cron job for updating the scripts working copy.

#11 Updated by okurz 11 months ago

  • Status changed from Feedback to Resolved

https://gitlab.suse.de/openqa/salt-states-openqa/merge_requests/202 merged. Checked manually on osd and also again o3 that there are no cron-job leftovers.

Also available in: Atom PDF