action #58373

Proposal: Poll test and needle repos every minute to make people wait less for updates in tests or needles

Added by okurz 4 months ago. Updated 3 months ago.

Status:ResolvedStart date:18/10/2019
Priority:NormalDue date:
Assignee:okurz% Done:

0%

Category:-
Target version:openQA Project - Done
Duration:

Description

git fetch is rather cheap and except for os-autoinst-needles-sles where we could use a gitlab CI pipeline. So I suggest we run the cron job every minute as we already set it to be not recorded in log files.


Related issues

Related to openQA Project - action #25572: [tools][needles]needles pushing can interfer with "fetchn... Resolved 26/09/2017

History

#1 Updated by okurz 4 months ago

  • Target version set to Current Sprint

Also I disabled git -C /var/lib/openqa/share/tests/openqa pull --quiet --rebase origin master on o3 in /etc/cron.d/openqa-update-git assuming that we should not need it. I will need to see if updates to os-autoinst-distri-openqa are pulled.

#2 Updated by okurz 4 months ago

  • Due date set to 19/11/2019

os-autoinst-distri-openQA is fine. I decreased the interval from 15m to 1m now on o3.

#3 Updated by okurz 3 months ago

  • Related to action #25572: [tools][needles]needles pushing can interfer with "fetchneedles" added

#4 Updated by coolo 3 months ago

Note that gitlab and github provide push notifications

#5 Updated by okurz 3 months ago

coolo wrote:

Note that gitlab and github provide push notifications

yes, for gitlab I mentioned that already in the description. There I would simply push not notifications but directly push the updates from os-autoinst-needles-sles as we already do with e.g. openqa/scripts. But for github? Where should we push to? My idea was to provide a simple API route that can be triggered, like "new tests/needles available, please fetch". Could be done as part of #45302

#6 Updated by coolo 3 months ago

There is generally interest in having those push notifications available on rabbitmq - so possibly we rely on rabbitmq for notification and leave it to magic how the github/gitlab payload ends on the bus.

#7 Updated by okurz 3 months ago

today we hit a problem. maxlin reported in https://susepaste.org/75288891 :

Failed to save yast2_snapper-empty-list-leap-20191113.
Unable to push Git commit: To github.com:os-autoinst/os-autoinst-needles-opensuse.git
 ! [rejected]          master -> master (non-fast-forward)
error: failed to push some refs to 'git@github.com:os-autoinst/os-autoinst-needles-opensuse.git'
hint: Updates were rejected because a pushed branch tip is behind its remote
hint: counterpart. Check out this branch and integrate the remote changes
hint: (e.g. 'git pull ...') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.

and from chat:

[13/11/2019 11:12:45] <maxlin> okurz, coolo: creating needle on o3 doesn't work, is it an known issue? https://susepaste.org/75288891
[13/11/2019 11:17:26] <okurz> maxlin: yes, known issue. I could have happened all the time, I just made it more likely by decreasing the git update interval from 15m to 1m :) So thanks for letting me know.
[13/11/2019 11:19:13] <maxlin> ok
[13/11/2019 11:19:44] <okurz> did you and others ignore the issue since 2019-11-11? there are quite some needles in a detached branch. I am handling this now
[13/11/2019 11:20:30] <okurz> hm, no. I also created needles and I was not aware of any error message
[13/11/2019 11:23:09] <maxlin> okurz: I can't remember that did I created any needle after 11/11, if I did, then no error for me until now :)
[13/11/2019 11:23:18] <okurz> rebased and pushed the branch, brought it back to master. Will record this in ticket, monitor and try to come up with a solution
[13/11/2019 11:23:50] <okurz> maxlin: yes, I can confirm. Could be that the branch detaching was caused by fetchneedles in the background and only then an error as you received popped up

I repaired manually once but somehow this seems as needle saving itself is fine and only fetchneedles need to handle this gracefully by "repairing". What I did as geekotest on o3:

git pull --rebase origin master 
git branch -D master && git checkout -b master
git push origin HEAD:master

so two steps that fetchneedles does not do: Ensuring that we are on a valid branch and pushing all outstanding commits in case we lost a branch. We can call test "$(git rev-parse --abbrev-ref)" = "HEAD" to check if we are detached

=> https://github.com/os-autoinst/openQA/pull/2486

EDIT: 2019-11-18: merged. To be monitored on o3 and osd. We might need an additional git branch --set-upstream-to=origin/master master

#8 Updated by okurz 3 months ago

  • Due date changed from 19/11/2019 to 26/11/2019

should check once more next week if it is still running fine on o3 with changed fetchneedles. osd still on */15

#9 Updated by okurz 3 months ago

  • Due date deleted (26/11/2019)

No problems on o3. Will propose to change osd the same.

https://gitlab.suse.de/openqa/salt-states-openqa/merge_requests/222 for osd.

#10 Updated by okurz 3 months ago

  • Status changed from Feedback to Resolved
  • Target version changed from Current Sprint to Done

done, live, no further problems observed, old problems fixed :)

Also available in: Atom PDF