Project

General

Profile

Actions

action #157159

open

coordination #154777: [saga][epic] Shareable os-autoinst and test distribution plugins

coordination #162131: [epic] future version control related features in openQA

Show version of candidate needles the test ran on in the "diff view" on the test details page

Added by mkittler 3 months ago. Updated 9 days ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
Feature requests
Target version:
Start date:
Due date:
% Done:

0%

Estimated time:

Description

Motivation

  • It is less confusing to see the candidate needles how they actually were at the time the test run (AC1).
  • Specifying Git repositories via CASEDIR/NEEDLES_DIR is made a first-class citizen and candidates needles are still shown at all when specifying a repository not checked on the web UI host.

Acceptance criteria

  • AC1: The "diff view" on the test details pages shows the candidate needles (images and areas) how they were defined at the time the test run (when this feature is enabled in openqa.ini).
  • AC2: The needle editor still allows to create new needles based on the latest version of existing needle images/areas.
  • AC3: AC1 is also fulfilled if CASEDIR/NEEDLES_DIR points to a Git repository that is now checked out under $OPENQA_BASEDIR/openqa/share/tests.
    • Maybe we want to declare this AC optional as it complicates things. If it ends up not implemented as part of this ticket a follow-up ticket is created instead.
  • AC4: If the ref cannot be shown (e.g. overridden after a force push) an indication that the latest version is shown instead should be displayed.
  • AC5: If the ref can be shown an indication should be displayed so users are made aware of the new feature.
  • AC6: Concurrent access and other usages of the Git repository (e.g. via fetchneedles) don't lead to errors or wrong needles being displayed.
  • AC7: A follow-up ticket for improving the cleanup strategy has been created. (It has not been created yet because its details will highly depend on the outcome of this ticket.)

Suggestions

  • See what has already been done as part on https://github.com/os-autoinst/openQA/pull/5175 as part of #154783.
    • Missing functionality for ACs
      • For AC2 the change needs to be adjusted so that the needle editor is only affected when enabled via an additional configuration.
      • For AC3 the code needs to check whether the repository needs to be cloned first and do that. Maybe we don't want to allow cloning arbitrary repositories, though.
      • For AC4 and AC5 it would make most sense to simply add a note on top of the image in the "diff view".
      • For AC6 the use of FETCH_HEAD needed to be avoided and maybe additional locking is required.
    • Further concerns
      • We might run into GitHub's rate limiting as this change always does a git fetch (except the ref is still cached).
      • It would cease to work for forks if GitHub stops allowing to fetch refs from forks by only using the original repository as remote. To avoid this we needed to add and use remotes as needed according to the Git URL in CASEDIR/NEEDLES_DIR. (But right now things should work for forks, see #154783#note-12.)
  • UI tests need to be implemented, see https://github.com/Martchus/openQA/pull/new/needles_from_casedir_tests for a start.
  • Consider recording the candidate needle images and area definitions when the test runs as part of the test result in case the approach from https://github.com/os-autoinst/openQA/pull/5175 doesn't work.

Out of scope

  • Improve the cleanup
  • Handle things sensibly within the needle editor
  • Track not only the time but also the ref in the last seen/match statistics

Related issues 4 (3 open1 closed)

Related to openQA Project - action #156922: Run os-autoinst-distri-openQA directly from git without anything related in o3:/var/lib/openqa/share/testsBlockedokurz

Actions
Follows openQA Project - action #154783: [spike][timeboxed:10h] Run os-autoinst-distri-example directly from git and ensure candidate needles show up on the web UI size:SResolvedmkittler

Actions
Precedes openQA Project - action #157165: The needle editor allows to create needles based on the ref the current test ran onNew2024-03-13

Actions
Copied to openQA Project - action #157231: Show version of candidate needles the test ran on in the "diff view" on the test details page - all the restNew2024-03-14

Actions
Actions #1

Updated by mkittler 3 months ago

  • Due date set to 2024-03-22
  • Start date changed from 2024-03-13 to 2024-03-22
  • Follows action #154783: [spike][timeboxed:10h] Run os-autoinst-distri-example directly from git and ensure candidate needles show up on the web UI size:S added
Actions #2

Updated by mkittler 3 months ago

  • Description updated (diff)
Actions #3

Updated by okurz 3 months ago

  • Due date deleted (2024-03-22)
  • Target version set to Tools - Next
  • Start date deleted (2024-03-22)
Actions #4

Updated by mkittler 3 months ago

  • Description updated (diff)
Actions #5

Updated by mkittler 3 months ago

  • Precedes action #157165: The needle editor allows to create needles based on the ref the current test ran on added
Actions #6

Updated by okurz 3 months ago

  • Related to action #156922: Run os-autoinst-distri-openQA directly from git without anything related in o3:/var/lib/openqa/share/tests added
Actions #7

Updated by okurz 3 months ago

  • Copied to action #157231: Show version of candidate needles the test ran on in the "diff view" on the test details page - all the rest added
Actions #8

Updated by okurz 9 days ago

  • Target version changed from Tools - Next to future
  • Parent task changed from #152847 to #162131
Actions

Also available in: Atom PDF