Project

General

Profile

Actions
Copy link

action #94231

closed

lessons learned from "make needle matching less forgiving"

Added by okurz almost 3 years ago. Updated almost 3 years ago.

Status:
Resolved
Priority:
Low
Assignee:
okurz
Category:
Organisational
Target version:
Ready
Start date:
2021-06-18
Due date:
% Done:

0%

Estimated time:

Description

Motivation

Despite multiple months keeping https://github.com/os-autoinst/os-autoinst/pull/1633 open and pinging test maintainers and even test maintainers asking explicitly to have the change deployed, after the change was going live many more people were struggling with the amount of test failures caused by the change.

Suggestions

  • Write to qa-team@suse.de as well for critical changes as well as chat channels
  • Please propose reverts of features (AFAIR no one suggested that and we would have been fine to do that)
  • From ph0enix: deploy on the beginning of the week, "announcement banners", staging area
  • Feature switches or configurable feature: We should have done that here, this was overlooked
  • See https://github.com/os-autoinst/os-autoinst/pull/1633#issuecomment-797467871 we should have needling with lower match ratio lower than default stated as a more explicit suggestion. From Joaquin SUSE QE Tools workshop chat notes: "The ratio is for each area in the needle, the UI doesn't highlight if a needle is created with lower or upper match ratio", so highlight that a needle is created with different match ratio than default?

Further details

The topic was discussed extensively in the SUSE QE Tools workshop 2021-06-18, recording available in https://youtu.be/wERuChD-88Y

Actions
Copy link

Also available in: Atom PDF