QA (public) » openQA Project (public)

My suggestion would be to add an instruction like the following to the openQA docs:

To be able to push via ssh, but still clone with http, configure git for the user geekotest with a url rewrite like the following:

git config --global url."git@github.com:".pushInsteadOf https://github.com/

This should work for most people, and if it isn't, then we can always work on something better.

The next alternative would be to let the user configure a mapping for github, gitlab etc. in openqa.ini and then do a local git config ... call during the clone phase.

At the end I tried to apply the suggested workaround (git config --global url."git@github.com:".pushInsteadOf https://github.com/)
The result was again the same, waiting for a passphase.

This would of course have to documented - the geekotest user would need a key without a passphrase.
That's already the case currently, when you clone over ssh explicitly.

first I need instructions to reproduce the reported error.

You could at least try the same thing on o3 to see what's happening there. If noone enabled the url rewrite there in the meantime, you will hopefully see the same as @okurz

It should be possible to prevent the password prompt for the ssh with the ssh_batchmode - have a look elsewhere in OpenQA::Git, where we already use this. Then it would fail immediately instead of prompting for password or passphrase.

For preventing the http credentials prompt, I just tried out the following:

% export GIT_TERMINAL_PROMPT=false
% export GIT_ASKPASS=
% git push origin testpush
fatal: could not read Username for 'https://github.com': terminal prompts disabled

So I suggest we set those variables in OpenQA::Git as well

reproduced in O3 problem

ybonatakis@ariel:/var/lib/openqa/share/tests/openqa/needles> sudo -u geekotest git config --list
http.version=HTTP/1.1
http.postbuffer=524288000
push.default=matching
core.repositoryformatversion=0
core.filemode=true
core.bare=false
core.logallrefupdates=true
remote.origin.url=https://github.com/os-autoinst/os-autoinst-needles-openQA
remote.origin.fetch=+refs/heads/*:refs/remotes/origin/*
branch.master.remote=origin
branch.master.merge=refs/heads/master
ybonatakis@ariel:/var/lib/openqa/share/tests/openqa/needles> sudo -u geekotest git config --global --list
push.default=matching

and indeed using `git config url."git@github.com:".pushInsteadOf https://github.com/ looks like worked.

ybonatakis@ariel:/var/lib/openqa/share/tests/openqa/needles> sudo -u geekotest git -C /var/lib/openqa/share/tests/openqa/needles push
Username for 'https://github.com': ^C
ybonatakis@ariel:/var/lib/openqa/share/tests/openqa/needles> sudo -u geekotest git config --global url."git@github.com:".pushInsteadOf https://github.com/
error: could not lock config file /var/lib/openqa/.gitconfig: Permission denied
ybonatakis@ariel:/var/lib/openqa/share/tests/openqa/needles> sudo -u geekotest git config url."git@github.com:".pushInsteadOf https://github.com/
ybonatakis@ariel:/var/lib/openqa/share/tests/openqa/needles> sudo -u geekotest git -C /var/lib/openqa/share/tests/openqa/needles push
Everything up-to-date  ## that was because the needle was already pushed previous from the UI

the config is still in place in O3 and the deletion commit is https://github.com/os-autoinst/os-autoinst-needles-openQA/commit/285a320c734cbb41090e924c636796729eda38cc

tinita wrote in #note-9:

It should be possible to prevent the password prompt for the ssh with the ssh_batchmode - have a look elsewhere in OpenQA::Git, where we already use this. Then it would fail immediately instead of prompting for password or passphrase.

For preventing the http credentials prompt, I just tried out the following:

% export GIT_TERMINAL_PROMPT=false
% export GIT_ASKPASS=
% git push origin testpush
fatal: could not read Username for 'https://github.com': terminal prompts disabled

So I suggest we set those variables in OpenQA::Git as well

According to my last comment I guess this is not needed.
Just thinking what I can provide or improve in the repo now

ybonatakis wrote in #note-15:

tinita wrote in #note-9:

So I suggest we set those variables in OpenQA::Git as well

According to my last comment I guess this is not needed.
Just thinking what I can provide or improve in the repo now

It is needed. You complained yesterday that the process was hanging. We want users to prevent from experiencing that, right?
So that if they don't have that git config, or if they don't have an ssh key (or one with a passphrase) that the process doesn't hang.

Please add those variables in OpenQA::Git, like I suggested, and also use ssh_batchmode for the git push, so we can guarantee git never asks for credentials.

edit: and I suggest to rename ssh_batchmode to just batchmode, since it will add variables for git http and ssh.

As requested, here is how I determined which env vars are needed to prevent git from prompting:

% git push origin testpush    # opens a gui prompt which I close
error: unable to read askpass response from '/usr/libexec/ssh/ssh-askpass'
Username for 'https://github.com': ^C

% GIT_ASKPASS= git push origin testpush     # opens a gui prompt which I close
Username for 'https://github.com': ^C

% GIT_TERMINAL_PROMPT=false git push origin testpush    # opens a gui prompt which I close
error: unable to read askpass response from '/usr/libexec/ssh/ssh-askpass'
fatal: could not read Username for 'https://github.com': terminal prompts disabled

% GIT_ASKPASS= GIT_TERMINAL_PROMPT=false git push origin testpush
fatal: could not read Username for 'https://github.com': terminal prompts disabled

So only the combination of the two variables does not prompt at all, which is what we want.

tinita wrote in #note-19:

As requested, here is how I determined which env vars are needed to prevent git from prompting:

% git push origin testpush    # opens a gui prompt which I close
error: unable to read askpass response from '/usr/libexec/ssh/ssh-askpass'
Username for 'https://github.com': ^C

% GIT_ASKPASS= git push origin testpush     # opens a gui prompt which I close
Username for 'https://github.com': ^C

% GIT_TERMINAL_PROMPT=false git push origin testpush    # opens a gui prompt which I close
error: unable to read askpass response from '/usr/libexec/ssh/ssh-askpass'
fatal: could not read Username for 'https://github.com': terminal prompts disabled

% GIT_ASKPASS= GIT_TERMINAL_PROMPT=false git push origin testpush
fatal: could not read Username for 'https://github.com': terminal prompts disabled

So only the combination of the two variables does not prompt at all, which is what we want.

I asked for GIT_ASKPASS= because the documentation says that if it is empty is treated as not set at all, which falls back in the default behavior. for info.

So if I understand correctly we want just to avoid the "hidden" prompt, and in place just make it print a message. I still not sure if this is the best, so bear with me as i try to think of somethning else. if not...

ybonatakis wrote in #note-20:

tinita wrote in #note-19:

I asked for GIT_ASKPASS= because the documentation says that if it is empty is treated as not set at all, which falls back in the default behavior. for info.

https://git-scm.com/docs/gitcredentials/2.42.0

If the GIT_ASKPASS environment variable is set, the program specified by the variable is invoked. A suitable prompt is provided to the program on the command line, and the user’s input is read from its standard output.

You can just set it to echo if you like that better.

So if I understand correctly we want just to avoid the "hidden" prompt, and in place just make it print a message.

yes, we want to avoid that git prompts and waits for user input, which leaves the process hanging.

And I showed how to do that.

tinita wrote in #note-21:

ybonatakis wrote in #note-20:

tinita wrote in #note-19:

I asked for GIT_ASKPASS= because the documentation says that if it is empty is treated as not set at all, which falls back in the default behavior. for info.

https://git-scm.com/docs/gitcredentials/2.42.0

If the GIT_ASKPASS environment variable is set, the program specified by the variable is invoked. A suitable prompt is provided to the program on the command line, and the user’s input is read from its standard output.

You can just set it to echo if you like that better.

So if I understand correctly we want just to avoid the "hidden" prompt, and in place just make it print a message.

yes, we want to avoid that git prompts and waits for user input, which leaves the process hanging.

And I showed how to do that.

Tina dont take it wrong but this had confused me. the ticket opened because there is a Username for 'https://github.com': Permission denied". From the descussion on slack is clear that there is no password prompt which makes the task to hang. there is your suggestion but seems to be something else.

give me some time to put some things together (if there is time until tomorrow) and lets have a chat. (keep in mind that I wont be available in the morning)

There are two tasks:

git prompts for password, process hanging¶

This only happens in some environments. Not in o3.
We observed this when trying out deleting needles in the openQA instance on your machine.
But it should be prevented by my suggestion above. It's nice that it doesn't happen in o3, but we can't rely this to be true on any openQA instance, right? openQA is a software that is not only used on o3 and osd.

Repos cloned via http should allow a push via ssh¶

This is the original problem why the ticket was created.

We normally, historically, clone needles via ssh to allow geekotest to automatically push (ssh key without passphrase).
For our new automatic git clone feature, using CASEDIR=https://... and NEEDLES_DIR=https://... in the scenario definitions, we want that people can specify http urls that are used to fetch. But they should be able to automatically push via ssh, even if the clone was done via http.
For that the pushInsteadOf is needed, which we should put into the openQA documentation: https://open.qa/docs/#_setting_up_git_support
People who setup an instance and want to use the git support are supposed to read that, so we can just add it there.

tinita wrote in #note-23:

For that the pushInsteadOf is needed, which we should put into the openQA documentation: https://open.qa/docs/#_setting_up_git_support
People who setup an instance and want to use the git support are supposed to read that, so we can just add it there.

Can we also point to that documentation within the error message?

okurz wrote in #note-24:

Can we also point to that documentation within the error message?

sure.

I think I have something to propose but I need some time. give me some time and I will provide a draft later in the afternoon

I fought all day with this. I was targeting to catch the prompt in the $stdout but I think is not possible(or so easy)
I come up with the minimum PoC proposal. Code/Commit is not ready for review. it is provided for some discussion

https://github.com/os-autoinst/openQA/pull/6140

An alternative for AC3: https://github.com/os-autoinst/openQA/pull/6141 Prevent git from prompting for credentials
tested successfully on my local instance

https://github.com/os-autoinst/openQA/pull/6142 Add docs about automatically using git ssh urls for pushing
todo: link to docs in error message

https://github.com/os-autoinst/openQA/pull/6142 merged

https://github.com/os-autoinst/openQA/pull/6144 Add link to docs if git push failed

And regarding the question about setting to the empty string via GIT_ASKPASS=: that still means that GIT_ASKPASS is set. It's just the empty string.
You can also unset such a variable again, with unset GIT_ASKPASS.

btw, something I learned along the way: how to tell the difference (unset vs. set and empty) with bash:
See also

• https://stackoverflow.com/a/13864829/93745
• https://stackoverflow.com/a/5406887/93745
• https://gist.github.com/JoeyBurzynski/f185285602b3c9f0fe708e32ad3ea06f