action #166202
closedUnable to login to Grafana aka monitor.qa.suse.de with valid credentials
0%
Description
Observation¶
Multiple people are finding themselves unable to login with valid credentails including @livdywan and @jbaier_cz.
Related error messages pointing to LDAP can be found in the journal:
Sep 03 09:42:26 monitor grafana[9821]: logger=authn.service t=2024-09-03T09:42:26.553806643+02:00 level=info msg="Failed to authenticate request" client=auth.client.form error="[password-auth.failed] failed to authenticate identity: unable to create LDAP client\n[password-auth.invalid] invalid password"
Sep 03 09:42:26 monitor grafana[9821]: logger=context userId=0 orgId=1 uname= t=2024-09-03T09:42:26.554038312+02:00 level=info msg=Unauthorized error="[password-auth.failed] failed to authenticate identity: unable to create LDAP client\n[password-auth.invalid] invalid password" remote_addr=@ traceID=
Suggestions¶
- Confirm how @ybonatakis is able to login anyhow (but without admin access)
Updated by nicksinger 4 months ago
- Assignee changed from livdywan to nicksinger
- Priority changed from High to Urgent
Updated by nicksinger 4 months ago
- Status changed from New to In Progress
So the web interface just reports "Login failed - Invalid username or password". Grafana logs show:
Sep 04 11:11:25 monitor grafana[18388]: logger=authn.service t=2024-09-04T11:11:25.868477384+02:00 level=warn msg="Failed to authenticate request" client=auth.client.session error="user token not found"
Sep 04 11:11:26 monitor grafana[18388]: logger=authn.service t=2024-09-04T11:11:26.307316458+02:00 level=info msg="Failed to authenticate request" client=auth.client.form error="[password-auth.failed] failed to authenticate identity: unable to create LDAP client\n[password-auth.invalid] invalid password"
Sep 04 11:11:26 monitor grafana[18388]: logger=context userId=0 orgId=1 uname= t=2024-09-04T11:11:26.307612563+02:00 level=info msg="Request Completed" method=POST path=/login status=401 remote_addr=@ time_ms=27 duration=27.412172ms size=107 referer=https://stats.openqa-monitor.qa.suse.de/login handler=/login
Same credentials work for https://powerhmc1.oqa.prg2.suse.org which also uses NIS. We reproduced the same with several team members so wrong passwords can most likely be excluded. Maybe I can copy the config from hmc1 to use in grafana as well.
Updated by nicksinger 4 months ago
while fixing an unrelated issue I saw the following after restarting grafana-server:
Sep 04 11:22:20 monitor grafana[32065]: logger=ldap t=2024-09-04T11:22:20.349832509+02:00 level=info msg="LDAP enabled, reading config file" file=/etc/grafana/ldap.toml
Sep 04 11:22:20 monitor grafana[32065]: logger=ldap.service t=2024-09-04T11:22:20.350320024+02:00 level=error msg="Failed to get LDAP config" error="Failed to load LDAP config file: toml: line 6: Key 'servers' was already created and cannot be used as an array."
Updated by nicksinger 4 months ago
- Status changed from In Progress to Feedback
The ldap config file contained the "[servers.attributes]"-section before the "[[servers]]" which is apparently a syntax error in toml. Since we abuse the ini-functions of salt we cannot ensure the proper order with the current approach. I changed it with https://gitlab.suse.de/openqa/salt-states-openqa/-/merge_requests/1262 and already verified on monitor that this works. Unfortunately until its merged, salt can mess up our file (although it shouldn't if the content is not changed).
Updated by nicksinger 4 months ago
- Priority changed from Urgent to High
As we can login again I see the urgency mitigated therefore reducing.
Updated by nicksinger 3 months ago
- Status changed from Feedback to Resolved
https://gitlab.suse.de/openqa/salt-states-openqa/-/merge_requests/1262 merged, people can login again.