QA (public) » openQA Project (public)

Motivation¶

Let's discuss how #162035 turned into an L ticket.

Background¶

https://github.com/os-autoinst/openQA/pull/5175 is a feature originally contributed by Scott Clarke from the community / Debian
We retroactively filed #162035 and only came up with ACs even later.

Questions¶

1. Why couldn't we estimate the effort properly to understand that the effort would be much more than reasonable for a single ticket?
   • A1-1: Because we understood that only the open points from #162035#note-30 were necessary
   • => I1-1-1: Be more clear with what "expertise" is necessary for certain tasks (e.g. using beginner/expert tags on tickets, this ticket had no such tags)
   • => I1-1-2: Ask assignees and domain experts explicitly for agreement or disagreement
   • => I1-1-3: Avoiding having a mindset of trying to merge the code ASAP, which could lead to the wrong compromises. We didn't consider to re-estimate or split
   • => I1-1-4: Explicitly come up with a good subject when estimating a ticket
2. Why weren't we able to finish the ticket by following the agreed open action points?
   • A2-1: We didn't look at the feature as something we would use in production which is why some tried to expand the scope of the work even though that was never properly written down in the ticket.
   • => I2-1-1: Please be open about change of plans or not yet understood parts in tickets
   • => I2-1-2: We can suggest to the PR author of https://github.com/os-autoinst/openQA/pull/5175 that we would work on #157159 first and rebase afterwards but let's be fair and not just force changes
   • A2-2: We didn't consider that this could be tough for someone less experienced with our testing facilities.
   • A2-3: The vacation hand-over probably added to the time spent and confusion about how to continue.
3. Why didn't ticket assignee and domain experts flag it in time that the due date is not realistic?
   • A3-1: We didn't think the ticket had priority in our own roadmap.
   • A3-2: Some of us thought that we already "gave up" on usual reasonable expectations and we were lost in "sunken cost fallacy"
   • => I3-2-1: Let's ensure everyone that we are a diligent team and stick to the plan or reconsider explicitly otherwise :) In dailies, unblock, coordination more diligently check the state of tickets with their due date, if effort estimation is still reasonable, etc.
   • A3-3: The due date was reset meanwhile for other reasons which would have hidden how much it was delayed in total.
   • A3-4: The ACs we came up with were not specific enough.
   • A3-6: The suggestion "Saying no to a feature is always an option too" was not considered.
4. Why are we working on a feature we cannot enable in our instances (at least not in the proposed form)?
   • A4-1: Because we want to help others and not just work for our's sake
   • A4-2: Because we see the feature as reasonable use case and something we might want to do anyway in the future (with some changes on top)
   • A4-3: It was seen as a minimal effort supporting changes we may enable later.
   • => I4-1: If only specific domain experts can lead a ticket then they should do it
5. Why did we not just do the missing "100% statement coverage" first? See #162035#note-34
   • A5-1: By not following on what we agreed upon and "question everything" without planning it in tickets
6. What should we do for the future regarding "picking up open pull requests by others"

Acceptance criteria¶

• AC1: A Five-Whys analysis has been conducted and results documented
• AC2: Improvements are planned

Suggestions¶

• Bring up in retro
• Conduct "Five-Whys" analysis for the topic
• Identify follow-up tasks in tickets
• Organize a call to conduct the 5 whys