QA (public)

Actually I think instead of https://github.com/os-autoinst/qa-tools-backlog-assistant better automate the "text template for update comments" from https://progress.opensuse.org/projects/openqatests/wiki#SLOs-service-level-objectives

• The backlogger project implements a generic version of the Tools backlog checker. It includes a GHA which can be used without(!) copying the code. Only a queries.yaml is needed. The format is slightly changed to use a list and it only needs to be processed once. As a side effect locally running it is also nicer.
• The openQA tests backlog project uses this. It looks basically like the Tools backlog with its own page and refreshes automatically. This project contains no code (not counting GHA workflows)
  • Right now the URL's are copied and the "qa projects" are absent. I'm thinking to save queries and use them, because copying URLs is just as bad as copying code.

I prepared a branch for the automatic reminder comments based on the existing queries.yaml. The template is a bit simpler and to keep it simple only includes the priority since that can be read from the ticket.

• Right now the URL's are copied and the "qa projects" are absent. I'm thinking to save queries and use them, because copying URLs is just as bad as copying code.

I also prepared a branch to enable comments and add the missing queries to openqa-tests-backlog. For now without preparing saved queries.

cdywan wrote:

I prepared a branch for the automatic reminder comments based on the existing queries.yaml. The template is a bit simpler and to keep it simple only includes the priority since that can be read from the ticket.

• Right now the URL's are copied and the "qa projects" are absent. I'm thinking to save queries and use them, because copying URLs is just as bad as copying code.

I also prepared a branch to enable comments and add the missing queries to openqa-tests-backlog. For now without preparing saved queries.

I provided an example of a working comment based on a manual run (no comments will be added from a PR), see the PR and the comment.

Of course since this is handling JSON I ran into a case of JSONDecodeError. I could not reproduce the issue locally. What I did do is improve the error handling so "similar" issues I could come up with would produce 404 or 401 and confirm what the specific error is. Right now that's impossible to see from the pipeline.

Turns out the problem was that the PR from my fork has no access to the REDMINE_API_KEY secret. It works so long as I propose it from a branch in upstream.

By chance I discovered that the preview action also doesn't currently support this. It might be worth looking into their work-around once that's done. Tina also suggested we could use a dispatch upstream to work-around the restriction.
I would refrain from looking into work-arounds for now, though, and rather use branches on the project.

For the record, since I got confused with that briefly, reminder comments always clear the queries since they filter by updated. No need to worry about how often we send them.
I also sent out a batch of reminders just now, to confirm that.

There's one more point, maybe we should use a bot account. I'm reading up how to create one. That should be the last step

Just create a new account from a private browser window. Otherwise for now you can also use the existing "openqa_review" account

okurz wrote:

Just create a new account from a private browser window. Otherwise for now you can also use the existing "openqa_review" account

It seems that's what I have to do. Creating an account from within Redmine leaves me unable to login. I created one via IDN and replaced the API key in openqa-tests-backlog

One issue with the Within due date query was found, which was missing the "last updated by" field. The due date was not updated, causing repeated comments to be added. I addressed this by updating the query.

cdywan wrote:

One issue with the Within due date query was found, which was missing the "last updated by" field. The due date was not updated, causing repeated comments to be added. I addressed this by updating the query.

Proper fix upstream by not sending out reminders for "due date" queries merged. Workflow re-enabled.

cdywan wrote:

For the record I was keeping this in Feedback since the last fix-up contained a typo. So instead of a one-character fix I'd like to add minimal unit tests based on the previous proof of concept. This is not blocked, though, just delayed because of other on-going tasks.

As discussed I'm keeping the tests minimal. However I did need MagicMock to actually test if a comment is posted: https://github.com/kalikiana/backlogger/pull/2

Also I moved the repo out of my personal account because otherwise the team won't have full access to everything, so now it's openSUSE/backlogger

okurz wrote:

1. In https://progress.opensuse.org/projects/openqatests/wiki#SLOs-service-level-objectives explain what a bot is handling automatically already and referencing to instructions how to update the rules that the bot is following. It can be just a link to https://github.com/openSUSE/openqa-tests-backlog/blob/main/queries.yaml , right?

Good point. I added notes regarding the backlog status as well as the bot configuration.

1. As the queries in https://github.com/openSUSE/openqa-tests-backlog/blob/main/queries.yaml#L5 are not named queries but defined in place we could just define a "grace period" for each query and only act automatically if not done already by users, e.g. don't remind on urgent tickets after 7 days but only 7+2 days
2. Same as in queries for QA tools we likely should only look at the update time of tickets with no subtasks, e.g. see the definition of https://progress.opensuse.org/issues?query_id=542, to prevent cases like https://progress.opensuse.org/issues/113749#change-552298
3. Only write the same comment once or better follow the SLO about the suggestion of the "second reminder"

These points have been brought up and discussed before, so if anything that tells me these are not minor details. I suggest you file according follow-up tickets and we can make sure it's understood what the mechanics are (or an epic if you prefer, no strong preference, but please don't make this ticket a moving target).