Project

General

Profile

action #81180

fetchneedles fails on uncommited needles

Added by okurz 7 months ago. Updated 6 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Concrete Bugs
Target version:
Start date:
2020-12-18
Due date:
2021-01-11
% Done:

0%

Estimated time:
Difficulty:

Description

Observation

Report by user "leli" in https://chat.suse.de/channel/testing?msg=kRr8dRMqjMXwB56YB

I met issue to save needle to OSD. 'Failed to save select_patterns-minimal-system-20201218.
Unable to reset repository to origin/master: error: cannot rebase: Your index contains uncommitted changes.
error: Please commit or stash them.'

Problem

"fetchneedles" should ensure a clean state and try to repair itself where it can

History

#1 Updated by okurz 7 months ago

Could reproduce the problem calling sudo -u geekotest bash -ex /usr/share/openqa/script/fetchneedles on osd

+ '[' '' = -h ']'
+ '[' '' = --help ']'
+ : geekotest
+ : opensuse
+ : git://github.com/os-autoinst/os-autoinst-distri-opensuse.git
+ : master
+ : openqa@
+ : 'openQA web UI'
+ : opensuse
+ : 1
+ : git://github.com/os-autoinst/os-autoinst-needles-opensuse.git
+ : master
+ : 0
+ dir=/var/lib/openqa/share/tests
+ '[' -w / ']'
+ '[' 0 = 1 ']'
+ target=/var/lib/openqa/share/tests/opensuse
+ mkdir -p /var/lib/openqa/share/tests/opensuse
+ cd /var/lib/openqa/share/tests/opensuse
+ '[' '!' -d .git ']'
+ do_fetch /var/lib/openqa/share/tests/opensuse
+ target=/var/lib/openqa/share/tests/opensuse
+ git_update master
+ branch=master
+ git gc --auto --quiet
+ git fetch -q origin
+ git rebase -q origin/master
+ '[' 1 = 1 ']'
+ test -d products
+ for nd in products/*/needles
+ '[' -h /var/lib/openqa/share/tests/opensuse/products/caasp/needles ']'
+ continue
+ for nd in products/*/needles
+ '[' -h /var/lib/openqa/share/tests/opensuse/products/casp/needles ']'
+ continue
+ for nd in products/*/needles
+ '[' -h /var/lib/openqa/share/tests/opensuse/products/microos/needles ']'
+ continue
+ for nd in products/*/needles
+ '[' -h /var/lib/openqa/share/tests/opensuse/products/opensuse/needles ']'
+ cd /var/lib/openqa/share/tests/opensuse/products/opensuse/needles
+ git_update_needles
+ git_update master
+ branch=master
+ git gc --auto --quiet
+ git fetch -q origin
+ git rebase -q origin/master
++ git rev-parse --abbrev-ref --symbolic-full-name HEAD
+ '[' master = HEAD ']'
+ for nd in products/*/needles
+ '[' -h /var/lib/openqa/share/tests/opensuse/products/sle/needles ']'
+ cd /var/lib/openqa/share/tests/opensuse/products/sle/needles
+ git_update_needles
+ git_update master
+ branch=master
+ git gc --auto --quiet
+ git fetch -q origin
+ git rebase -q origin/master
error: cannot rebase: Your index contains uncommitted changes.
error: Please commit or stash them.

and the current state

# sudo su geekotest
geekotest@openqa:~/share/tests/sle/products/sle/needles> git status
On branch master
Your branch is up to date with 'origin/master'.

Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
    new file:   links_https-links-connect-suse-webpage-20201218.json
    new file:   links_https-links-connect-suse-webpage-20201218.png

Did as geekotest

git commit -m "Add links_https-links-connect-suse-webpage-20201218" -a
git rebase -q origin/master
git push origin HEAD:master

EDIT: We already changed fetchneedles in the past, e.g. in 2aadd3e56 trying to repair needle git repos on detached HEADs. But I think we do not have automatic repairs that would commit or delete files we have not pushed.

#2 Updated by openqa_review 7 months ago

  • Due date set to 2021-01-02

Setting due date based on mean cycle time of SUSE QE Tools

#3 Updated by cdywan 7 months ago

I guess you've been mending the repo manually but not looked into a concrete automatic fix yet?

As a suggestion, how about git diff-index --exit-code HEAD -- && git reset --hard HEAD? I proposed https://github.com/os-autoinst/openQA/pull/3658 - of course if you already have a better approach, I'm happy to delete it, it was just quick enough to try the idea I had briefly and push what I had.

#4 Updated by okurz 7 months ago

  • Status changed from In Progress to Feedback
  • Assignee changed from okurz to cdywan

I guess your approach is ok. I have thought about alternatives which would not delete uncommited data to not loose anything that users have created but I could not easily come up with something better right now. I approved your PR.

#5 Updated by cdywan 7 months ago

  • Due date changed from 2021-01-02 to 2021-01-11

#6 Updated by okurz 6 months ago

PR merged, deployed to osd. You can now set the force flag in salt states: https://gitlab.suse.de/openqa/salt-states-openqa/-/blob/master/etc/master/cron.d/SLES.CRON#L5

EDIT: Changed o3 already accordingly.

#8 Updated by okurz 6 months ago

MR was merged. Unfortunately now we have output where previously the script was silent. As we run this as part of cron that means that every minute we generate an email with content like the following:

# sudo -u geekotest env force=1 updateall=1 /usr/share/openqa/script/fetchneedles
HEAD is now at 9104e3b1f Merge pull request #11751 from ricardobranco777/stupid_fix
HEAD is now at 85f70a138 logoutdialog-xfce-aarch64-20210113 for opensuse-Tumbleweed-DVD-aarch64-Build20210111-create_hdd_xfce@aarch64
HEAD is now at ad96f20da yast2-install-hypervisor-and-tools-20210113 for sle-15-SP3-Online-x86_64-Build123.3-yast2_gui@64bit
HEAD is now at 85f70a138 logoutdialog-xfce-aarch64-20210113 for opensuse-Tumbleweed-DVD-aarch64-Build20210111-create_hdd_xfce@aarch64
HEAD is now at 1cd822c update the information of the testsuite
HEAD is now at 1cd822c update the information of the testsuite

this should be fixed. Please also check the runtime performance comparing the old approach with the new approach as this is called every minute for multiple repositories.

#9 Updated by cdywan 6 months ago

okurz wrote:

MR was merged. Unfortunately now we have output where previously the script was silent. As we run this as part of cron that means that every minute we generate an email with content like the following:

https://github.com/os-autoinst/openQA/pull/3671

this should be fixed. Please also check the runtime performance comparing the old approach with the new approach as this is called every minute for multiple repositories.

I was proposing an additional check with git diff-index before but during review we seemed to agree that we might as well reset all the time

#10 Updated by cdywan 6 months ago

  • Status changed from Feedback to Resolved

I assume this is okay now

Also available in: Atom PDF