action #106907
closedcoordination #91646: [saga][epic] SUSE Maintenance QA workflows with fully automated testing, approval and release
coordination #98457: [epic] Handle openqa-review reminder comments on very old jobs better
Exponential backoff for reminders based on previous reminders size:M
0%
Description
Motivation¶
Currently the latest comment is read, and a reminder is sent if it's at least 14 days old. This can become annoying so we can hold off some comments
Acceptance criteria¶
- AC1: Reminder comments are sent no sooner than double the previous interval (between the last two comments)
- AC2: The setting is configurable
Suggestions¶
- Use exponential backoff, see https://en.wikipedia.org/wiki/Exponential_backoff
- Read all the comments (we basically already do that)
- Parse timestamp (we already do that) and calculate time difference
- If (time - time_last) > 2 * max((time_last - time_second_to_last // 0), initial_time_interval) then add reminder
- In reminder comment add a text like "Expect the next reminder at the earliest $time_next" with $time_next being current time + 2 * current difference used
Out of scope¶
- Optional: Detect if the comments are openqa-review reminder comments
Updated by tinita almost 3 years ago
In reminder comment add a text like "Expect the next reminder at the earliest $time_next" with $time_next being current time + 2 * current difference used
I'm not sure that make sense if we are considering all comments, not only auto-review comments:
Comment n-2 (user-x): March 1st
Comment n-1 (user-y): March 3rd (2d)
Comment n (auto-review): March 7 (4d)
"Expect the next reminder at the earliest in 8 days"
Comment n+1 (user-x): March 8 (1d)
Now the interval is set back to 1 day, so auto-review would wait 2 days, so the comment about 8 days is wrong.
Updated by tinita almost 3 years ago
Question about redmine: Should field changes without a text comment be considered comments?
Updated by okurz almost 3 years ago
tinita wrote:
In reminder comment add a text like "Expect the next reminder at the earliest $time_next" with $time_next being current time + 2 * current difference used
I'm not sure that make sense if we are considering all comments, not only auto-review comments:
Comment n-2 (user-x): March 1st Comment n-1 (user-y): March 3rd (2d) Comment n (auto-review): March 7 (4d) "Expect the next reminder at the earliest in 8 days" Comment n+1 (user-x): March 8 (1d)
Now the interval is set back to 1 day, so auto-review would wait 2 days, so the comment about 8 days is wrong.
That's why we have the max in max((time_last - time_second_to_last // 0), initial_time_interval)
, in your example max(($time_current - 1d), 14) = 14
with 14 being our initial_time_interval. So I don't think that the interval is set back. The good effect is that basically any non-auto-review comment that are posted earlier than the next expected reminder would reset the exponential backoff if I understand my/our algorithm correctly so we would again have reminder comments sooner which I consider a good thing. For a stale bug we want less frequent reminders, for an active bug we want more frequent information flows.
tinita wrote:
Question about redmine: Should field changes without a text comment be considered comments?
Yes. Or more correctly what we look at anyway are "any changes in the history", not just comments. Compare to how the tabs in redmine interface are called. By default in History there is "All" selected with the option to show only "Notes" aka. "Comments" and "Changes".
Updated by tinita almost 3 years ago
any non-auto-review comment that are posted earlier than the next expected reminder would reset the exponential backoff
Well, take my example and make it 14 days, 28 days etc. instead.
And you say it yourself, it would reset the backoff.
But then the previous comment "Expect the next reminder at the earliest in x days" will be wrong.
It would be correct to say "Expect the next reminder at the earliest in x days if noone comments earlier".
Updated by jbaier_cz over 2 years ago
- Status changed from Workable to In Progress
Updated by jbaier_cz over 2 years ago
- Status changed from In Progress to Feedback
I created https://github.com/os-autoinst/openqa_review/pull/211, feedback welcomed.
Updated by jbaier_cz over 2 years ago
- Status changed from Feedback to Resolved
From looking at bsc#1193859 and bsc#1004573 I got an idea it is working correctly.