Project

General

Profile

coordination #45302

coordination #58184: [saga][epic][use case] full version control awareness within openQA, e.g. user forks and branches, fully versioned test schedules and configuration settings

[epic] smarter fetchneedles (was: fetchneedles should ensure we are always on a branch (and try to self-repair))

Added by okurz almost 3 years ago. Updated 7 months ago.

Status:
Blocked
Priority:
Low
Assignee:
okurz
Category:
Feature requests
Target version:
QA - future
Start date:
2019-12-20
Due date:
% Done:

50%

Estimated time:
(Total: 0.00 h)
Difficulty:

Description

Observation

[18/12/2018 14:23:00] <guillaume_g> okurz: https://github.com/os-autoinst/os-autoinst-needles-opensuse/commits/master seems to not be updated since 5 days! openQA should push updates
[18/12/2018 15:01:58] <okurz> guillaume_g: yes, should be. checking…

Problem

Something caused a conflict causing the repo to not stay on the master branch and en

Fixed with the following:

su geekotest
cd /var/lib/openqa/share/tests/opensuse/products/opensuse/needles/
git checkout -b fix
git add .
git commit -m "Add gnucash-tip-close-unfocussed-20181128" -a && git pull --rebase origin master
git rebase --skip
git checkout master
git rebase fix
git pull --rebase origin master && git push origin HEAD:master
git branch -D fix

On OSD we have in /etc/cron.d/SLES.CRON:

# sometimes gitlab.s.d is unresponsive and there are new needles coming in and we did not pull in time before someone created new needles from the webui so make sure this is synced at least once a day
47 3 * * * geekotest (cd /var/lib/openqa/share/tests/sle/products/sle/needles && git pull --rebase >/dev/null && git push origin HEAD:master >/dev/null 2>&1)

The "fetchneedles" could try to apply all that. Maybe something like the following would be best:

sudo -u geekotest sh -c '(cd /var/lib/openqa/share/tests/opensuse/products/opensuse/needles && <commit all locally> && <fetch all> && <rebase all, skip conflicting local changes> && <ensure on valid branch> && <push all>)'

Subtasks

openQA Infrastructure - action #61221: osd: unable to save needles, minion fails with "fatal: Unable to create '/var/lib/openqa/.../needles/.git/index.lock'"New

action #63133: SLE needles git repo looses upstream branch configuration, needles are saved but fail to be pushedResolvedokurz

Related issues

Has duplicate openQA Project - action #53672: Using https://openqa.opensuse.org/admin/needles ran into some problems on the git repo on multi-needle deleteRejected2019-07-02

History

#1 Updated by coolo about 2 years ago

  • Subject changed from fetchneedles should ensure we are always on a branch (and try to self-repair) to EPIC: smarter fetchneedles (was: fetchneedles should ensure we are always on a branch (and try to self-repair))
  • Target version set to Ready

I think we should even integrate fetchneedles into openQA and have a UI for what to checkout/update and on what schedule - taken that we git commit into it, we should also git checkout

#2 Updated by okurz about 2 years ago

  • Subject changed from EPIC: smarter fetchneedles (was: fetchneedles should ensure we are always on a branch (and try to self-repair)) to [epic] smarter fetchneedles (was: fetchneedles should ensure we are always on a branch (and try to self-repair))
  • Parent task set to #58184

#3 Updated by mkittler about 2 years ago

okurz You proposed making this a Minion job so we can prevent conflicts with the other Minion jobs to save/delete needles. That would also allow an easier integration with the web UI.

Having a brief look at the fetchneedles code it seems that is has a more advanced logic than I'd expected. Maybe we can just invoke the Bash script from Perl? On the other hand the script should be generally improved (e.g. it hard codes /var/lib as OPENQA_BASEDIR).

#4 Updated by okurz about 2 years ago

So what I think we can do is to start the fetchneedles script by a minion job which should be easy but not try to re-write fetchneedles itself for now. I would keep any further improvements to handling all tests+needles better with git and potentially do not keep global needle repo when we can handle that as git.

#5 Updated by SLindoMansilla almost 2 years ago

On a shared branch, if more than one person commits changes, the only way to not cause troubles to other is to use merge, which will not change the commit tree. Using rebase for example, will cause that all others will need to "fix" their tree.
Since commits to needle repositories can come from the repositories webui (gitlab and github) and from the openQA webui (interactive/developer mode), there is always a risk of hitting that tree divergence.

I already solved problems like these in my previous company and my suggestion for a general solution with minimal changes on the infrastructure is:

That openQA webui doesn't commit to master, but to another branch (for example a branch called o3 for openqa.opensuse.org and one branch called osd for openqa.suse.de). These branches will hold the current status of production environments. openQA will then periodically fetch and merge master branch from $origin onto o3/osd branch and push changes to $origin o3/osd.

Users/developers should then just track o3 for openSUSE needles and osd for SLE needles, and they will have the latest status of production environments.

#6 Updated by SLindoMansilla almost 2 years ago

Another problem to solve is that git was not develop so that two people can work on the same working directory at the same time. So that if they both perform an operation, they will likely cause troubles to each other.
So, the developer/interactive mode should avoid two operation at the same time. I suggest to implement an operation queue.

#7 Updated by okurz almost 2 years ago

SLindoMansilla wrote:

So, the developer/interactive mode should avoid two operation at the same time. I suggest to implement an operation queue.

It already does. needle saving is done by enqueuing gru jobs. What is not done in the same way is that fetchneedles can conflict in parallel.

#8 Updated by okurz over 1 year ago

  • Has duplicate action #53672: Using https://openqa.opensuse.org/admin/needles ran into some problems on the git repo on multi-needle delete added

#9 Updated by szarate about 1 year ago

  • Tracker changed from action to coordination

#11 Updated by okurz about 1 year ago

  • Status changed from New to Blocked
  • Assignee set to okurz

waiting for specific subtask #61221

#12 Updated by okurz 7 months ago

  • Target version changed from Ready to future

Also available in: Atom PDF