Project

General

Profile

Actions

coordination #13812

open

[epic][dashboard] openQA Dashboard ideas

Added by dheidler about 8 years ago. Updated 4 months ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
Feature requests
Target version:
Start date:
2017-01-10
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)

Description

Related use cases

Especially UC1 and UC5

ideas

  • Failing tests
  • Consider some testsuites as more important than other testsuites #13780
  • Filter on index page for specific groups -> gh#923
  • Group test results by build number, related ticked: #13742
  • finish at least one build per day -> Current workflow: QA reviewer decides this and can mark one build as important, feature request #15844
  • priority of bugs in review reports -> openqa_review#25
  • on index page add link to filter box (down) because "no one scrolls down" -> https://gitlab.suse.de/openqa/salt-states-openqa/merge_requests/19
  • "milestones" as special tags (asked by ast) -> http://open.qa/docs/#_tag_builds_with_special_comments_on_group_overview
  • special mode of Dashboard mode which allow to see summary of all job groups in optimized to full screen view #17886
  • Statistics over
    • failed/passed ratio per build for each product
    • number of executed testsuites per build for each product
  • List overall "todo: review" entries
  • Overall status indicator (eg. status light / 'Ampel')
  • "the oldest still open bug"
  • "the most flaky scenario / module / architecture"
  • back-references to bugs
    • "reverse" carry over: when old job failed and new job passed -> post comment into bug that would have been carried over to new job if it would have failed (related to okurz wiki )
  • scenario result and state change in each tests result
  • track all changes (eg. infrastructure, test changed, product changed, ...)
  • measure completion time of builds, also per architecture
  • tests that have not run for long time
  • live update of bug status (e.g. subscribe a special user to email and trigger events based on email reception)
  • personal customized view vs. static hierarchical dashboard view
  • "weather of scenario", e.g. is it failing all the time or just sporadically
  • better integration of status from external jobs and testing systems (see below)
  • customized, personal filter pre-saved
  • dheidler: learn about bugzilla integration in "smash" (e.g. ask tgoettlicher)

implementation ideas

better integration of status from external jobs and testing systems

To use openQA also as the main reporting platform relying on other tools we have in our infrastructure (at SUSE) there are already test runners feeding back the information e.g. in junit format. Also, using the client script jobs can be started and their result set along with comments, e.g. for tags. Example:

job_id=$(client --host http://localhost jobs post TEST=foo_external _GROUP="openSUSE Tumbleweed AArch64" BUILD=1 | sed -n 's/^{ id => \([0-9]\+\) }$/\1/p')
client --host http://localhost jobs/${job_id}/set_done post result=passed
client --host http://localhost jobs/${job_id}/comments post text=bsc#1234

references

could be inspired by sonar, e.g. see https://sonarqube.com/ and https://sonarqube.com/dashboard?id=c-family%3ASamba&did=148, but also jenkins Radiator View Plugin or jenkins eXtreme Feedback Panel Plugin or https://jenkins.io/blog/2016/01/10/beautiful-jenkins-dashboard/
… or https://github.com/influxdata/chronograf

And don't forget about http://testsuites.qa.suse.cz -> dead by now


Subtasks 3 (0 open3 closed)

action #15844: [tools]finish at least one build per dayResolvedokurz2017-01-10

Actions
action #17886: [dashboard] Create full screen view for openQA dashboardResolvedilausuch2017-03-22

Actions
action #18000: [dashboard] Better integration of status from external jobs and testing systemsResolvedokurz2017-03-24

Actions

Related issues 5 (1 open4 closed)

Related to openQA Project - action #10148: better notification and user feedbackResolved2015-11-13

Actions
Related to openQA Project - action #10188: [tools][dashboard]Tracking open bugs based on productResolveddheidler2016-01-12

Actions
Related to openQA Project - action #13712: [tools][dashboard]overview page for multiple products at onceResolvedokurz2016-06-21

Actions
Related to openQA Project - action #13714: [process][redmine] backlink references in openqa comments to tickets inside the corresponding tickets, like githubNew2016-09-13

Actions
Related to openQA Project - coordination #39719: [saga][epic] Detection of "known failures" for stable tests, easy test results review and easy tracking of known issuesResolvedokurz2018-05-23

Actions
Actions #3

Updated by mkittler about 8 years ago

  • Description updated (diff)
Actions #4

Updated by okurz about 8 years ago

  • Description updated (diff)
Actions #5

Updated by okurz about 8 years ago

  • Description updated (diff)
Actions #6

Updated by okurz about 8 years ago

  • Related to action #10148: better notification and user feedback added
Actions #7

Updated by okurz about 8 years ago

  • Related to action #10188: [tools][dashboard]Tracking open bugs based on product added
Actions #8

Updated by okurz about 8 years ago

  • Description updated (diff)
Actions #9

Updated by okurz about 8 years ago

  • Related to action #13712: [tools][dashboard]overview page for multiple products at once added
Actions #10

Updated by okurz about 8 years ago

  • Description updated (diff)
  • Status changed from Feedback to In Progress

Add notes from today's meeting.

Actions #11

Updated by okurz about 8 years ago

  • Description updated (diff)
Actions #12

Updated by okurz about 8 years ago

  • Description updated (diff)

updated resolved ideas

Actions #13

Updated by mkittler about 8 years ago

  • Description updated (diff)
Actions #14

Updated by okurz about 8 years ago

  • Related to action #13714: [process][redmine] backlink references in openqa comments to tickets inside the corresponding tickets, like github added
Actions #16

Updated by okurz almost 8 years ago

  • Description updated (diff)
Actions #17

Updated by okurz almost 8 years ago

  • Description updated (diff)
Actions #18

Updated by mkittler almost 8 years ago

  • Description updated (diff)
Actions #19

Updated by okurz almost 8 years ago

pull out "finish at least one build per day" as it's own ticket: #15844

Actions #20

Updated by okurz almost 8 years ago

  • Description updated (diff)
Actions #21

Updated by okurz almost 8 years ago

  • Category set to Feature requests
Actions #22

Updated by okurz almost 8 years ago

  • Description updated (diff)
Actions #23

Updated by okurz over 7 years ago

  • Description updated (diff)

incorporate content from #17886

Actions #24

Updated by okurz over 7 years ago

  • Has duplicate action #17886: [dashboard] Create full screen view for openQA dashboard added
Actions #25

Updated by okurz over 7 years ago

  • Has duplicate deleted (action #17886: [dashboard] Create full screen view for openQA dashboard)
Actions #26

Updated by okurz over 7 years ago

  • Description updated (diff)
Actions #27

Updated by RBrownSUSE over 7 years ago

  • Subject changed from openQA Dashboard ideas to [tools]openQA Dashboard ideas
Actions #28

Updated by RBrownSUSE over 7 years ago

  • Subject changed from [tools]openQA Dashboard ideas to [tools][dashboard]openQA Dashboard ideas
Actions #29

Updated by okurz over 6 years ago

  • Description updated (diff)
  • Status changed from In Progress to Workable

crossed of the completed parts and reordered to show the still open ideas below the completed ones.

Actions #30

Updated by okurz over 6 years ago

  • Description updated (diff)

http://testsuites.qa.suse.cz seems to be dead by now

Actions #31

Updated by okurz about 6 years ago

  • Related to coordination #39719: [saga][epic] Detection of "known failures" for stable tests, easy test results review and easy tracking of known issues added
Actions #32

Updated by okurz over 4 years ago

  • Subject changed from [tools][dashboard]openQA Dashboard ideas to [epic][dashboard] openQA Dashboard ideas
  • Status changed from Workable to Blocked
  • Assignee set to okurz
Actions #33

Updated by okurz over 4 years ago

  • Target version set to Ready
Actions #34

Updated by okurz about 4 years ago

  • Status changed from Blocked to New
  • Assignee deleted (okurz)
  • Target version changed from Ready to future

#17886 is a subtask that got resolved. With this I would leave the further ideas for "future".

Actions #35

Updated by szarate about 4 years ago

  • Tracker changed from action to coordination
Actions

Also available in: Atom PDF