action #166202
closed
Unable to login to Grafana aka monitor.qa.suse.de with valid credentials
Added by livdywan 3 months ago.
Updated 2 months ago.
Category:
Regressions/Crashes
Description
Observation¶
Multiple people are finding themselves unable to login with valid credentails including @livdywan and @jbaier_cz.
Related error messages pointing to LDAP can be found in the journal:
Sep 03 09:42:26 monitor grafana[9821]: logger=authn.service t=2024-09-03T09:42:26.553806643+02:00 level=info msg="Failed to authenticate request" client=auth.client.form error="[password-auth.failed] failed to authenticate identity: unable to create LDAP client\n[password-auth.invalid] invalid password"
Sep 03 09:42:26 monitor grafana[9821]: logger=context userId=0 orgId=1 uname= t=2024-09-03T09:42:26.554038312+02:00 level=info msg=Unauthorized error="[password-auth.failed] failed to authenticate identity: unable to create LDAP client\n[password-auth.invalid] invalid password" remote_addr=@ traceID=
Suggestions¶
- Confirm how @ybonatakis is able to login anyhow (but without admin access)
- Assignee changed from livdywan to nicksinger
- Priority changed from High to Urgent
- Status changed from New to In Progress
So the web interface just reports "Login failed - Invalid username or password". Grafana logs show:
Sep 04 11:11:25 monitor grafana[18388]: logger=authn.service t=2024-09-04T11:11:25.868477384+02:00 level=warn msg="Failed to authenticate request" client=auth.client.session error="user token not found"
Sep 04 11:11:26 monitor grafana[18388]: logger=authn.service t=2024-09-04T11:11:26.307316458+02:00 level=info msg="Failed to authenticate request" client=auth.client.form error="[password-auth.failed] failed to authenticate identity: unable to create LDAP client\n[password-auth.invalid] invalid password"
Sep 04 11:11:26 monitor grafana[18388]: logger=context userId=0 orgId=1 uname= t=2024-09-04T11:11:26.307612563+02:00 level=info msg="Request Completed" method=POST path=/login status=401 remote_addr=@ time_ms=27 duration=27.412172ms size=107 referer=https://stats.openqa-monitor.qa.suse.de/login handler=/login
Same credentials work for https://powerhmc1.oqa.prg2.suse.org which also uses NIS. We reproduced the same with several team members so wrong passwords can most likely be excluded. Maybe I can copy the config from hmc1 to use in grafana as well.
while fixing an unrelated issue I saw the following after restarting grafana-server:
Sep 04 11:22:20 monitor grafana[32065]: logger=ldap t=2024-09-04T11:22:20.349832509+02:00 level=info msg="LDAP enabled, reading config file" file=/etc/grafana/ldap.toml
Sep 04 11:22:20 monitor grafana[32065]: logger=ldap.service t=2024-09-04T11:22:20.350320024+02:00 level=error msg="Failed to get LDAP config" error="Failed to load LDAP config file: toml: line 6: Key 'servers' was already created and cannot be used as an array."
- Status changed from In Progress to Feedback
The ldap config file contained the "[servers.attributes]"-section before the "[[servers]]" which is apparently a syntax error in toml. Since we abuse the ini-functions of salt we cannot ensure the proper order with the current approach. I changed it with https://gitlab.suse.de/openqa/salt-states-openqa/-/merge_requests/1262 and already verified on monitor that this works. Unfortunately until its merged, salt can mess up our file (although it shouldn't if the content is not changed).
- Priority changed from Urgent to High
As we can login again I see the urgency mitigated therefore reducing.
- Status changed from Feedback to Resolved
Also available in: Atom
PDF