Project

General

Profile

Actions

action #106907

closed

coordination #91646: [saga][epic] SUSE Maintenance QA workflows with fully automated testing, approval and release

coordination #98457: [epic] Handle openqa-review reminder comments on very old jobs better

Exponential backoff for reminders based on previous reminders size:M

Added by okurz about 2 years ago. Updated about 2 years ago.

Status:
Resolved
Priority:
High
Assignee:
Target version:
Start date:
2022-02-16
Due date:
% Done:

0%

Estimated time:

Description

Motivation

Currently the latest comment is read, and a reminder is sent if it's at least 14 days old. This can become annoying so we can hold off some comments

Acceptance criteria

  • AC1: Reminder comments are sent no sooner than double the previous interval (between the last two comments)
  • AC2: The setting is configurable

Suggestions

  • Use exponential backoff, see https://en.wikipedia.org/wiki/Exponential_backoff
  • Read all the comments (we basically already do that)
  • Parse timestamp (we already do that) and calculate time difference
  • If (time - time_last) > 2 * max((time_last - time_second_to_last // 0), initial_time_interval) then add reminder
  • In reminder comment add a text like "Expect the next reminder at the earliest $time_next" with $time_next being current time + 2 * current difference used

Out of scope

  • Optional: Detect if the comments are openqa-review reminder comments
Actions #1

Updated by tinita about 2 years ago

  • Description updated (diff)
Actions #2

Updated by tinita about 2 years ago

In reminder comment add a text like "Expect the next reminder at the earliest $time_next" with $time_next being current time + 2 * current difference used

I'm not sure that make sense if we are considering all comments, not only auto-review comments:

Comment n-2 (user-x):    March 1st
Comment n-1 (user-y):    March 3rd (2d)
Comment n (auto-review): March 7 (4d)
  "Expect the next reminder at the earliest in 8 days"
Comment n+1 (user-x): March 8 (1d)

Now the interval is set back to 1 day, so auto-review would wait 2 days, so the comment about 8 days is wrong.

Actions #3

Updated by tinita about 2 years ago

Question about redmine: Should field changes without a text comment be considered comments?

Actions #4

Updated by okurz about 2 years ago

tinita wrote:

In reminder comment add a text like "Expect the next reminder at the earliest $time_next" with $time_next being current time + 2 * current difference used

I'm not sure that make sense if we are considering all comments, not only auto-review comments:

Comment n-2 (user-x):    March 1st
Comment n-1 (user-y):    March 3rd (2d)
Comment n (auto-review): March 7 (4d)
  "Expect the next reminder at the earliest in 8 days"
Comment n+1 (user-x): March 8 (1d)

Now the interval is set back to 1 day, so auto-review would wait 2 days, so the comment about 8 days is wrong.

That's why we have the max in max((time_last - time_second_to_last // 0), initial_time_interval), in your example max(($time_current - 1d), 14) = 14 with 14 being our initial_time_interval. So I don't think that the interval is set back. The good effect is that basically any non-auto-review comment that are posted earlier than the next expected reminder would reset the exponential backoff if I understand my/our algorithm correctly so we would again have reminder comments sooner which I consider a good thing. For a stale bug we want less frequent reminders, for an active bug we want more frequent information flows.

tinita wrote:

Question about redmine: Should field changes without a text comment be considered comments?

Yes. Or more correctly what we look at anyway are "any changes in the history", not just comments. Compare to how the tabs in redmine interface are called. By default in History there is "All" selected with the option to show only "Notes" aka. "Comments" and "Changes".

Actions #5

Updated by tinita about 2 years ago

any non-auto-review comment that are posted earlier than the next expected reminder would reset the exponential backoff

Well, take my example and make it 14 days, 28 days etc. instead.
And you say it yourself, it would reset the backoff.
But then the previous comment "Expect the next reminder at the earliest in x days" will be wrong.
It would be correct to say "Expect the next reminder at the earliest in x days if noone comments earlier".

Actions #6

Updated by okurz about 2 years ago

  • Priority changed from Normal to High
Actions #7

Updated by jbaier_cz about 2 years ago

  • Assignee set to jbaier_cz
Actions #8

Updated by jbaier_cz about 2 years ago

  • Status changed from Workable to In Progress
Actions #9

Updated by jbaier_cz about 2 years ago

  • Status changed from In Progress to Feedback
Actions #10

Updated by jbaier_cz about 2 years ago

  • Status changed from Feedback to Resolved

From looking at bsc#1193859 and bsc#1004573 I got an idea it is working correctly.

Actions

Also available in: Atom PDF