Project

General

Profile

action #162518

Updated by okurz 6 months ago

## Observation 

 Links in alert emails contain stats.openqa-monitor.qa.suse.de which is an alias to monitor.qa.suse.de - except I failed to login after several attemps. As I was drafting this ticket I tried again and logging in eventually worked. 

 See also [this Slack thread](https://suse.slack.com/archives/C02AJ1E568M/p1718794272644989), others in the team were able to reproduce the error and eventually able to login. 

 ## Suggestions 

 * Look into logs on monitor.qa.suse.de to find relevant messages 
   * `Jun 19 13:11:18 monitor grafana[18120]: logger=authn.service t=2024-06-19T13:11:18.159015268+02:00 level=warn msg="Failed to authenticate request" client=auth.client.session error="user token not found"` 
   * Also noticed these ones, tho they may not be related 
     * `Jun 19 12:59:18 monitor grafana[18120]: logger=sqlstore.transactions t=2024-06-19T12:59:18.470555152+02:00 level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked"` 

 ## Workaround 

 * Replace the hostname in alert URL's to get a reliably working link using monitor.qa.suse.de

Back