Project

General

Profile

action #33211

[sle15sp1][desktop] Better handling wayland and x11 testing results reviewing.

Added by yfjiang over 3 years ago. Updated 3 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Enhancement to existing tests
Target version:
-
Start date:
2018-04-04
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Difficulty:

Description

Currently the desktop test cases are organized as a bit scattered:

  • Desktop Applications -> * (Wayland)
  • Functional -> extra_tests_on_gnome (X11)
  • Functional -> gnome (X11)
  • Functional -> we-module (Wayland)
  • Functional -> yast2_gui (X11)

Maybe there are more to be revealed.

Basically wheather the test cases running on X11 or Wayland is currently
based on how the system, upon which the test cases is running, was installed.

  • For the installation adapted within the test case, it depends on the status of Workstation Extension and selected system role
  • For the test cases depending on the standalone installation case (e.g. create_hdd_*), it depends on whether the installation case has Workstation Extension and GNOME system role

The goals can be phrased as:

  1. Have a more organized way to tell openQA result readers that the testing was running on Wayland or X11
  2. Migrate all those X11 cases into Wayland, so that we could make sure the GUI test cases already automated should be run on Wayland

Subtasks

action #34231: [sle][functional][desktop] Add a create_hdd_* testsuite for SLES+WEResolvedqkzhu

action #34612: [sle15][desktop] Rename all wayland related testsuitesResolvedqkzhu

action #34615: [sle15][desktop] Migrate all wayland testsuites to SLES+WE by START_AFTER_TEST=create_hdd_gnome+weResolvedqkzhu

History

#2 Updated by yfjiang over 3 years ago

Here is a correction of the description based on the latest comments here:

http://bugzilla.suse.com/show_bug.cgi?id=1083611#c14

The SLES+WE would not get wayland by default, so the current status
is more clear that most of tests in Functional group are X11 based:

Desktop Applications -> * (Wayland)
Functional -> extra_tests_on_gnome (X11)
Functional -> gnome (X11)
Functional -> we-module (X11)
Functional -> yast2_gui (X11)

#3 Updated by qkzhu over 3 years ago

Hi Yifan,

All cases under Desktop Applications were blocked by bsc#1083665 for two weeks, we could test all these cases on SLES15+WE to make sure the product quality of GNOME stack.
Testsuite create_hdd_gnome+we was added 9 hours ago, so I think we could:

  1. Rename all wayland related testsuites:

desktopapps-documentation -> wayland-desktopapps-documentation
desktopapps-gnome -> wayland-desktopapps-gnome
desktopapps-other -> wayland-desktopapps-other
desktopapps-webbrowser-core -> wayland-desktopapps-webbrowser-core
desktopapps-webbrowser-extra -> wayland-desktopapps-webbrowser-extra

  1. Migrate above testsuites to SLES+WE by START_AFTER_TEST=create_hdd_gnome+we:

desktopapps-documentation -> x11-desktopapps-documentation
desktopapps-gnome -> x11-desktopapps-gnome
desktopapps-other -> x11-desktopapps-other
desktopapps-webbrowser-core -> x11-desktopapps-webbrowser-core
desktopapps-webbrowser-extra -> x11-desktopapps-webbrowser-extra

WDYT?

#4 Updated by yfjiang over 3 years ago

Hi Chingkai,

I like your idea, so that we can test wayland on SLED, and x11 on SLES+WE. The combination looks orthogonal and enables much wider testing coverage.

#5 Updated by okurz over 3 years ago

  • Category set to Enhancement to existing tests

#6 Updated by yfjiang over 3 years ago

Side notes of the latest status: most of the core stuff has been done by chingkai, the remained pending items:

  • yast2_gui on Wayland
  • a part of extra_tests_on_gnome on Wayland

#7 Updated by yfjiang about 3 years ago

  • Subject changed from [sle15][desktop] Better handling wayland and x11 testing results reviewing. to [sle15sp1][desktop] Better handling wayland and x11 testing results reviewing.

#8 Updated by GraceWang 3 months ago

  • Status changed from New to Resolved

All the sub-tasks have been resolved. Hence this ticket can be closed now.

Also available in: Atom PDF