coordination #45302
closedcoordination #58184: [saga][epic][use case] full version control awareness within openQA
[epic] smarter fetchneedles (was: fetchneedles should ensure we are always on a branch (and try to self-repair))
Description
Observation¶
[18/12/2018 14:23:00] <guillaume_g> okurz: https://github.com/os-autoinst/os-autoinst-needles-opensuse/commits/master seems to not be updated since 5 days! openQA should push updates
[18/12/2018 15:01:58] <okurz> guillaume_g: yes, should be. checking…
Problem¶
Something caused a conflict causing the repo to not stay on the master branch and en
Fixed with the following:
su geekotest
cd /var/lib/openqa/share/tests/opensuse/products/opensuse/needles/
git checkout -b fix
git add .
git commit -m "Add gnucash-tip-close-unfocussed-20181128" -a && git pull --rebase origin master
git rebase --skip
git checkout master
git rebase fix
git pull --rebase origin master && git push origin HEAD:master
git branch -D fix
On OSD we have in /etc/cron.d/SLES.CRON:
# sometimes gitlab.s.d is unresponsive and there are new needles coming in and we did not pull in time before someone created new needles from the webui so make sure this is synced at least once a day
47 3 * * * geekotest (cd /var/lib/openqa/share/tests/sle/products/sle/needles && git pull --rebase >/dev/null && git push origin HEAD:master >/dev/null 2>&1)
The "fetchneedles" could try to apply all that. Maybe something like the following would be best:
sudo -u geekotest sh -c '(cd /var/lib/openqa/share/tests/opensuse/products/opensuse/needles && <commit all locally> && <fetch all> && <rebase all, skip conflicting local changes> && <ensure on valid branch> && <push all>)'
Updated by coolo almost 5 years ago
- Subject changed from fetchneedles should ensure we are always on a branch (and try to self-repair) to EPIC: smarter fetchneedles (was: fetchneedles should ensure we are always on a branch (and try to self-repair))
- Target version set to Ready
I think we should even integrate fetchneedles into openQA and have a UI for what to checkout/update and on what schedule - taken that we git commit into it, we should also git checkout
Updated by okurz almost 5 years ago
- Subject changed from EPIC: smarter fetchneedles (was: fetchneedles should ensure we are always on a branch (and try to self-repair)) to [epic] smarter fetchneedles (was: fetchneedles should ensure we are always on a branch (and try to self-repair))
- Parent task set to #58184
Updated by mkittler almost 5 years ago
@okurz You proposed making this a Minion job so we can prevent conflicts with the other Minion jobs to save/delete needles. That would also allow an easier integration with the web UI.
Having a brief look at the fetchneedles code it seems that is has a more advanced logic than I'd expected. Maybe we can just invoke the Bash script from Perl? On the other hand the script should be generally improved (e.g. it hard codes /var/lib as OPENQA_BASEDIR).
Updated by okurz almost 5 years ago
So what I think we can do is to start the fetchneedles script by a minion job which should be easy but not try to re-write fetchneedles itself for now. I would keep any further improvements to handling all tests+needles better with git and potentially do not keep global needle repo when we can handle that as git.
Updated by SLindoMansilla over 4 years ago
On a shared branch, if more than one person commits changes, the only way to not cause troubles to other is to use merge, which will not change the commit tree. Using rebase for example, will cause that all others will need to "fix" their tree.
Since commits to needle repositories can come from the repositories webui (gitlab and github) and from the openQA webui (interactive/developer mode), there is always a risk of hitting that tree divergence.
I already solved problems like these in my previous company and my suggestion for a general solution with minimal changes on the infrastructure is:
That openQA webui doesn't commit to master, but to another branch (for example a branch called o3 for openqa.opensuse.org and one branch called osd for openqa.suse.de). These branches will hold the current status of production environments. openQA will then periodically fetch and merge master branch from $origin onto o3/osd branch and push changes to $origin o3/osd.
Users/developers should then just track o3 for openSUSE needles and osd for SLE needles, and they will have the latest status of production environments.
Updated by SLindoMansilla over 4 years ago
Another problem to solve is that git was not develop so that two people can work on the same working directory at the same time. So that if they both perform an operation, they will likely cause troubles to each other.
So, the developer/interactive mode should avoid two operation at the same time. I suggest to implement an operation queue.
Updated by okurz over 4 years ago
SLindoMansilla wrote:
So, the developer/interactive mode should avoid two operation at the same time. I suggest to implement an operation queue.
It already does. needle saving is done by enqueuing gru jobs. What is not done in the same way is that fetchneedles can conflict in parallel.
Updated by okurz over 4 years ago
- Has duplicate action #53672: Using https://openqa.opensuse.org/admin/needles ran into some problems on the git repo on multi-needle delete added
Updated by szarate almost 4 years ago
- Tracker changed from action to coordination
Updated by szarate almost 4 years ago
See for the reason of tracker change: http://mailman.suse.de/mailman/private/qa-sle/2020-October/002722.html
Updated by okurz almost 4 years ago
- Status changed from New to Blocked
- Assignee set to okurz
waiting for specific subtask #61221
Updated by okurz over 3 years ago
- Target version changed from Ready to future