action #89620
closedopenqa-clone-custom-git-refspec fails with: env: git: Permission denied
Added by ggardet_arm about 3 years ago. Updated about 3 years ago.
0%
Description
openqa-clone-git-refspec fails with: env: git: Permission denied:
[2021-03-08T13:01:42.0238 UTC] [debug] +++ worker notes +++
[37m[2021-03-08T13:01:43.081 UTC] [debug] Current version is 4.6.1613739162.3c1f4d7d [interface v20]
[0m[37m[2021-03-08T13:01:43.087 UTC] [debug] Checking out git refspec/branch 'MTE'
[0m[37m[2021-03-08T13:01:43.087 UTC] [debug] Cloning git URL 'https://github.com/ggardet/os-autoinst-distri-opensuse.git' to use as test distribution
[0m[37m[2021-03-08T13:01:43.098 UTC] [debug] env: âgitâ: Permission denied
[0mUnable to clone Git repository 'https://github.com/ggardet/os-autoinst-distri-opensuse.git#MTE' specified via CASEDIR (see log for details) at /usr/lib/os-autoinst/OpenQA/Isotovideo/Utils.pm line 69.
31628: EXIT 1
Occurrence: https://openqa.opensuse.org/tests/1661452
Updated by ggardet_arm about 3 years ago
- Subject changed from openqa-clone-git-refspec fails with: env: git: Permission denied to openqa-clone-custom-git-refspec fails with: env: git: Permission denied
Updated by okurz about 3 years ago
- Due date set to 2021-03-16
- Category set to Support
- Status changed from New to Feedback
- Assignee set to okurz
- Target version set to Ready
Hm, this looks like the worker has apparmor enabled but https://github.com/os-autoinst/openQA/blob/master/profiles/apparmor.d/usr.share.openqa.script.worker#L80 is not effective.
@ggardet_arm I assume you maintain this worker "hctw01". Can you please check if the apparmor files up-to-date? Could be there is a file like /etc/apparmor.d/...usr.share.openqa.script.worker.rpm-new
Updated by ggardet_arm about 3 years ago
No /etc/apparmor.d/usr.share.openqa.script.worker.rpmnew, only /etc/apparmor.d/usr.share.openqa.script.worker and /etc/apparmor.d/usr.share.openqa.script.worker.rpmsave
And grep -r 'git' /etc/apparmor.d/usr.share.openqa.script.worker returns only:
/usr/bin/git rix,
/usr/lib/git/git rix,
/usr/lib/git/git-remote-http rix,
/usr/share/git-core/templates/ r,
/usr/share/git-core/templates/** r,
Updated by okurz about 3 years ago
Ok, maybe it's about env then in https://github.com/os-autoinst/openQA/blob/master/profiles/apparmor.d/usr.share.openqa.script.worker#L76 ? Could you check if env is also included? You can crosscheck by switching apparmor off and recheck
Updated by ggardet_arm about 3 years ago
env entry is there as well:
/usr/bin/env rix,
If I disable apparmor, the problem remains. So, it looks like a permission error on a folder.
Updated by okurz about 3 years ago
Just to be sure, I don't know how you "disabled apparmor". Just disabling the systemd service does not make the profile ineffective. You have to explicitly disable the policies using the apparmor commands. But for crosschecking I recommend that you call sudo -u _openqa-worker isotovideo manually in a folder where you put the vars.json file of the job. At best once in a path like /tmp and once in the same directory that the worker would run, e.g. /var/lib/openqa/pool/11 as in https://openqa.opensuse.org/tests/1661452
Updated by ggardet_arm about 3 years ago
Calling manually isotovideo in /tmp clones the git repo properly:
[2021-03-09T12:37:06.295 UTC] [debug] Current version is 4.6.1613739162.3c1f4d7d [interface v20]
[2021-03-09T12:37:06.300 UTC] [debug] Checking out git refspec/branch 'MTE'
[2021-03-09T12:37:06.301 UTC] [debug] Cloning git URL 'https://github.com/ggardet/os-autoinst-distri-opensuse.git' to use as test distribution
[2021-03-09T12:37:21.295 UTC] [debug] Cloning into 'os-autoinst-distri-opensuse'...
Updating files: 100% (3239/3239), done.
I rebooted after I disabled apparmor and now the repo is cloned, but fails later:
[37m[2021-03-09T12:51:13.608 UTC] [debug] Current version is 4.6.1613739162.3c1f4d7d [interface v20]
[0m[37m[2021-03-09T12:51:13.613 UTC] [debug] Checking out git refspec/branch 'MTE'
[0m[37m[2021-03-09T12:51:13.614 UTC] [debug] Cloning git URL 'https://github.com/ggardet/os-autoinst-distri-opensuse.git' to use as test distribution
[0m[37m[2021-03-09T12:51:34.872 UTC] [debug] Cloning into 'os-autoinst-distri-opensuse'...
Updating files: 80% (2598/3239)
Updating files: 81% (2624/3239)
Updating files: 82% (2656/3239)
Updating files: 83% (2689/3239)
Updating files: 84% (2721/3239)
Updating files: 85% (2754/3239)
Updating files: 86% (2786/3239)
Updating files: 87% (2818/3239)
Updating files: 88% (2851/3239)
Updating files: 89% (2883/3239)
Updating files: 90% (2916/3239)
Updating files: 91% (2948/3239)
Updating files: 92% (2980/3239)
Updating files: 93% (3013/3239)
Updating files: 94% (3045/3239)
Updating files: 95% (3078/3239)
Updating files: 96% (3110/3239)
Updating files: 97% (3142/3239)
Updating files: 98% (3175/3239)
Updating files: 99% (3207/3239)
Updating files: 100% (3239/3239)
Updating files: 100% (3239/3239), done.
[0m[37m[2021-03-09T12:51:34.884 UTC] [debug] git hash in /var/lib/openqa/pool/11/os-autoinst-distri-opensuse: d26016bdbf408d10408d0f7d78970fe050cdc3d9
[0m[33m[2021-03-09T12:51:34.885 UTC] [info] ::: OpenQA::Isotovideo::Utils::load_test_schedule: Enforced test schedule by 'SCHEDULE' variable in action
[0mloadtest needs a script below /var/lib/openqa/pool/11/os-autoinst-distri-opensuse - tests/installation/bootloader_uefi.pm.pm is not
[37m[2021-03-09T12:51:34.886 UTC] [debug] error on tests/installation/bootloader_uefi.pm.pm: Invalid version format (0 before decimal required) at (eval 138) line 1, near "package bootloader_uefi"
syntax error at (eval 138) line 1, near "package bootloader_uefi."
BEGIN not safe after errors--compilation aborted at (eval 138) line 1.
[0merror on tests/installation/bootloader_uefi.pm.pm: Invalid version format (0 before decimal required) at (eval 138) line 1, near "package bootloader_uefi"
syntax error at (eval 138) line 1, near "package bootloader_uefi."
BEGIN not safe after errors--compilation aborted at (eval 138) line 1.
1914: EXIT 1
Maybe this is now related to: https://progress.opensuse.org/issues/67723 and https://github.com/os-autoinst/openQA/pull/3712 not merged yet?
EDIT: this is an error on my side: bootloader_uefi.pm.pm
Updated by okurz about 3 years ago
Ok, good progress. Could you do me the favor of running the apparmor profile in "complain" mode and provide us the diff for the apparmor profile if any?
Updated by ggardet_arm about 3 years ago
okurz wrote:
Ok, good progress. Could you do me the favor of running the apparmor profile in "complain" mode and provide us the diff for the apparmor profile if any?
I thought about this, but it seems apparmor is not very healthy atm and I cannot switch to complain mode.
Updated by okurz about 3 years ago
hm, ok. Given that then I can only recommend that you consider keeping apparmor disabled for the specific services and we close the ticket as resolved if that is enough for you. Or what else do you think we could do?
Updated by ggardet_arm about 3 years ago
okurz wrote:
hm, ok. Given that then I can only recommend that you consider keeping apparmor disabled for the specific services and we close the ticket as resolved if that is enough for you. Or what else do you think we could do?
Let's close it for now. I will revisit once apparmor is fixed.
Updated by ggardet_arm about 3 years ago
- Status changed from Resolved to Workable
I finally get some apparmor feedback:
Profile: /usr/share/openqa/script/worker
Operation: exec
Name: /usr/libexec/git/git
Denied: x
Profile: /usr/share/openqa/script/worker
Operation: open
Name: /usr/libexec/git/git
Denied: r
Updated by okurz about 3 years ago
Oh, could it be that the path to git differs per architecture? Because in https://github.com/os-autoinst/openQA/blob/master/profiles/apparmor.d/usr.share.openqa.script.worker#L115 we already have:
/usr/lib/git/git rix,
but not "libexec". I guess we can simply prepare a PR for that. Would you like to do that after trying it out?
Updated by ggardet_arm about 3 years ago
okurz wrote:
Oh, could it be that the path to git differs per architecture? Because in https://github.com/os-autoinst/openQA/blob/master/profiles/apparmor.d/usr.share.openqa.script.worker#L115 we already have:
/usr/lib/git/git rix,but not "libexec". I guess we can simply prepare a PR for that. Would you like to do that after trying it out?
No, this is not arch specific. As stated at https://en.opensuse.org/openSUSE:Packaging_Conventions_RPM_Macros#.25_libexecdir Tumbleweed now (since August 2020) uses /usr/libexec instead of /usr/lib for %{_libexec} macro
Updated by ggardet_arm about 3 years ago
PR available: https://github.com/os-autoinst/openQA/pull/3780
Updated by ggardet_arm about 3 years ago
- Status changed from Workable to Resolved
- Assignee changed from okurz to ggardet_arm