Project

General

Profile

Core » History » Version 8

szarate, 2023-05-26 10:19

1 2 tjyrinki_suse
# QE Core
2 1 okurz
3
(this chapter has seen changes in 2020-11 regarding QSF -> QE Core / QE Yast change)
4
5 2 tjyrinki_suse
**QE Core** (formerly QSF, QA SLE Functional) and of the core functionality of the SUSE SLE products. The squad is comprised of members of QE Integration - [SUSE QA SLE Nbg](https://wiki.suse.net/index.php/SUSE-Quality_Assurance/Organization/Members_and_Responsibilities#QA_SLE_NBG_Team), including [SUSE QA SLE Prg](https://wiki.suse.net/index.php/SUSE-Quality_Assurance/Organization/Members_and_Responsibilities#QA_SLE_PRG_Team) - and QE Maintenance people (formerly "QAM"). The [SLE Departement](https://wiki.suse.net/index.php/SUSE-Quality_Assurance/SLE_Department#QSF_.28QA_SLE_Functional.29) page describes our QA responsibilities. We focus on our automatic tests running in [openQA](https://openqa.suse.de) under the job groups "Functional" and "Core" (maintenance SLE releases), for example [SLE 15 / Functional](https://openqa.suse.de/group_overview/110). We back our automatic tests with exploratory manual tests, especially for the product milestone builds. Additionally we care about corresponding openSUSE openQA tests (see as well https://openqa.opensuse.org).
6 1 okurz
7 4 szarate
## Scope of QE Core:
8 3 tjyrinki_suse
9
Responsibilities:
10
11
    Maintaining as top level authority os-autoinst-distri-opensuse git repository and Core maintained test suites
12
    Organizing overall configuration changes like new products/versions (but can delegate to other teams)
13
    Maintain Functional / Core job groups on OSD (see eg schedule/functional in os-autoinst-distri-opensuse)
14
    Maintain modules where QE Core is the maintainer
15
    Validation of install image builds (new product, QU) as far as tests are in our domain
16
    Create new tests for basic userspace packages, if there's an important lack of test coverage (for example, a regression that slipped past)
17
    Creating new ways of end-to-end test scenarios that combine pieces of software traditionally tested by single squads.
18
    QE Core has some extra freedoms to do more testing innovation (cross-squad) and drive changes.
19
    Triage tickets with [qe-core] tag.
20
    Fix problems with maintained test modules.
21
22 4 szarate
### TL;DR
23 1 okurz
24 8 szarate
If you want to file a ticket for QE Core, add "[qe-core]" to the Subject line. If you are reporting a broken test, use the tag `bugbusters`.
25 1 okurz
26 4 szarate
See [[Core#What we do|What we do]] for more detailed information regarding our backlog.
27 1 okurz
28
## Test plan
29
30
When looking for coverage of certain components or use cases keep the [openQA glossary](http://open.qa/docs/#concept) in mind. It is important to understand that "tests in openQA" could be a scenario, for example a "textmode installation run", a combined multi-machine scenario, for example "a remote ssh based installation using X-forwarding", or a test module, for example "vim", which checks if the vim editor is correctly installed, provides correct rendering and basic functionality. You are welcome to contact any member of the team to ask for more clarification about this.
31
32
In detail the following areas are tested as part of "SLE functional":
33
34
* different hardware setups (UEFI, acpi)
35 2 tjyrinki_suse
* support for localization
36
37 3 tjyrinki_suse
Virtualization and Migration have separate squads for them:
38 1 okurz
* openSUSE: virtualization - some "virtualization" tests are active on o3 with reduced set compared to SLE coverage (on behalf of QA SLE virtualization due to team capacity constraints, clarified in QA SLE coordination meeting 2018-03-28)
39
* openSUSE: migration - comparable to "virtualization", a reduced set compared to SLE coverage is active on o3 (on behalf of QA SLE migration due to team capacity constraints, clarified in QA SLE coordination meeting 2018-04)
40
41
### QE Core
42
43
"Testing is the future, and the future starts with you"
44
45
* Current definitions can be found at https://confluence.suse.com/display/qasle/Tests+Maintained+by+QE+Core, 
46
47 3 tjyrinki_suse
Note: Link mentioned above is WIP; QE-Core's work has impact on the openSUSE community as well, to keep the community in sync, either https://progress.opensuse.org/projects/qa/wiki/Core or a better place has to be used to share what is the scope of work, always keeping to a unique source of truth, that is available to the community, keeping SLE's specific information, available to SUSE employees only. 
48 1 okurz
49
* Latest report based on openQA test results SLE12: http://s.qa.suse.de/test-status-sle12-functional , SLE15: http://s.qa.suse.de/test-status-sle15-functional
50
51
## In new organization also covered by QE Core and others
52
53
* quarterly updated media: former QA Maintenance (QAM) is now part of the various QE squads. However, QU media does happen together with Maintenance Coordination that is not part of these squads.
54
55
**The rest of the page is possibly interesting, but has not been updated since QSF-U changed to QE Core and included development maintenance SLE release tests in the same categories as Functional job group**
56 2 tjyrinki_suse
57 1 okurz
## What we do
58 2 tjyrinki_suse
59 1 okurz
We collected opinions, personal experiences and preferences starting with the following four topics: What are fun-tasks ("new tests", "collaborate", "do it right"), what parts are annoying ("old & sporadic issues"), what do we think is expected from qsf-u ("be quick", "keep stuff running", "assess quality") and what we should definitely keep doing to prevent stakeholders becoming disappointed ("build validation", "communication & support").
60
61
### How we work on our backlog
62
63 4 szarate
* Tickets have a due-date only when it is required or there's a business need or impacts other teams
64 1 okurz
* we pick up tickets that have not been previously discussed
65 4 szarate
* We are flexible on what we work.
66 1 okurz
67 5 szarate
* We strive to have no [un-triaged](https://progress.opensuse.org/projects/openqatests/wiki#ticket-backlog-triaging) tickets, see also (Not public yet) [Guidelines for triaging](https://progress.opensuse.org/projects/suseqa/wiki#Guidelines-for-triaging). 
68 1 okurz
69 4 szarate
* SLAs:
70 5 szarate
 * First goal is "to remove the urgency", see current [Service Level Objectives](https://progress.opensuse.org/projects/openqatests/wiki#SLOs-service-level-objectives), removing the urgency often doesn't necessarily mean to fully close the ticket, but to understand what it is about, and have a plan on how to tackle it, specially true for maintenance updates, as failing openQA tests imply no updates being auto-approved.
71 1 okurz
72
* everybody should set priority + milestone in obvious cases, e.g. new reproducible test failures in multiple critical scenarios, in general case the PO decides
73
74 4 szarate
### Where to find our backlog
75
76
* Full backlog and tickets to triage can be found at: https://is.gd/aLWTdv
77
* Example of a sprint: https://is.gd/qe_core_backlog_example
78
79 1 okurz
### How we like to choose our battles
80
81
We self-assessed our tasks on a scale from "administrative" to "creative" and found in the following descending order: daily test review (very "administrative"), ticket triaging, milestone validation, code review, create needles, infrastructure issues, fix and cleanup tests, find bugs while fixing failing tests, find bugs while designing new tests, new automated tests (very "creative"). Then we found we appreciate if our work has a fair share of both sides. Probably a good ratio is 60% creative plus 40% administrative tasks. Both types have their advantages and we should try to keep the healthy balance.
82
83 6 szarate
### What do we (really) *care* about?
84 1 okurz
85 6 szarate
#### Team Priorities
86 1 okurz
87 7 szarate
1. Maintenance Updates for SLE and openSUSE Leap, and Quarterly Updates.
88
1. Feature testing and Manual validation of said features.
89
1. Current SLE Product in development (eg, SLE-15SP5 by the time of this writing), SUSE ALP, and par with openSUSE Tumbleweed and Leap in development.
90
1. Team's internal projects and cross-squad collaboration.
91
1. Requests out of the QE department (eg: When the team should schedule external testsuites).
92
1. Test Requests with no owner.
93 1 okurz
94 7 szarate
Note: This is a mirror of https://confluence.suse.com/display/qasle/QE+Core#QECore-Teampriorities
95
96 1 okurz
From this list strictly no "product" gives us direct revenue however most likely SLE(S) (as well as SLES HA and SLE migration) are good examples of direct connection to revenue (based on SLE subscriptions). Conducting a poll in the team has revealed that 3 persons see "SLE(S)" as our main product and 3 see "os-autoinst-distri-opensuse+backend+needles" as the main product. We mainly agreed that however we can not *own* a product like "SLE" because that product is mainly not under our control.
97 6 szarate
98
Visualizing "cost of testing" vs. "risk of business impact" showed that both metrics have an inverse dependency, e.g. on a range from "upstream source code" over "package self-tests", "openSUSE Factory staging", "Tumbleweed", "SLE" we consider SLE to have the highest business risk attached and therefore defines our priority however testing at upstream source level is considered most effective to prevent higher cost of bugs or issues. Our conclusion is that we must ensure that the high-risk SLE base has its quality assured while supporting a quality assurance process as early as possible in the development process. package self-tests as well as the openQA staging tests are seen as a useful approach in that direction as well as "domain specfic specialist QA engineers" working closely together with according in-house development parties.
99 1 okurz
100
## Documentation
101
102
This documentation should only be interesting for the team QA SLE functional. If you find that some of the following topics are interesting for other people, please extract those topics to another wiki section.
103
104
### QA SLE functional Dashboards
105
106
In room 3.2.15 from Nuremberg office are two dedicated laptops each with a monitor attached showing a selected overview of openQA test resuls with important builds from SLE and openSUSE.
107
Such laptops are configured with a root account with the default password for production machines. First point of contact: [slindomansilla.suse.com](mailto:slindomansilla@suse.com), (okurz@suse.de)[mailto:okurz@suse.de]
108
109
* ''dashboard-osd-3215.suse.de'': Showing current view of openqa.suse.de filtered for some job group results, e.g. "Functional"
110
* ''dashboard-o3-3215.suse.de'': Showing current view of openqa.opensuse.org filtered for some job group results which we took responsibility to review and are mostly interested in
111
112
### dashboard-osd-3215
113
114
* OS: openSUSE Tumbleweed
115
* Services: ssh, mosh, vnc, x2x
116
* Users:
117
** root
118
** dashboard
119
* VNC: `vncviewer dashboard-osd-3215`
120
* X2X: `ssh -XC dashboard@dashboard-osd-3215 x2x -west -to :0.0`
121
** (attaches the dashboard monitor as an extra display to the left of your screens. Then move the mouse over and the attached X11 server will capture mouse and keyboard)
122
123
#### Content of /home/dashboard/.xinitrc
124
125
```
126
#
127
# Source common code shared between the
128
# X session and X init scripts
129
#
130
. /etc/X11/xinit/xinitrc.common
131
132
xset -dpms
133
xset s off
134
xset s noblank
135
[...]
136
#
137
# Add your own lines here...
138
#
139
$HOME/bin/osd_dashboard &
140
```
141
142
#### Content of /home/dashboard/bin/osd_dashboard
143
144
```
145
#!/bin/bash
146
147
DISPLAY=:0 unclutter &
148
149
DISPLAY=:0 xset -dpms
150
DISPLAY=:0 xset s off
151
DISPLAY=:0 xset s noblank
152
153
url="${url:-"https://openqa.suse.de/?group=SLE+15+%2F+%28Functional%7CAutoyast%29&default_expanded=1&limit_builds=3&time_limit_days=14&show_tags=1&fullscreen=1#"}"
154
DISPLAY=:0 chromium --kiosk "$url"
155
```
156
157
#### Cron job:
158
159
```
160
Min     H       DoM     Mo      DoW     Command
161
*	*	*	*	*	/home/dashboard/bin/reload_chromium
162
```
163
164
#### Content of /home/dashboard/bin/reload_chromium
165
166
```
167
#!/bin/bash
168
169
DISPLAY=:0 xset -dpms
170
DISPLAY=:0 xset s off
171
DISPLAY=:0 xset s noblank
172
173
DISPLAY=:0 xdotool windowactivate $(DISPLAY=:0 xdotool search --class Chromium)
174
DISPLAY=:0 xdotool key F5
175
DISPLAY=:0 xdotool windowactivate $(DISPLAY=:0 xdotool getactivewindow)
176
```
177
178
#### Issues:
179
180
* ''When the screen shows a different part of the web page''
181
** a simple mouse scroll through vnc or x2x may suffice.
182
* ''When the builds displayed are freeze without showing a new build, it usually means that midori, the browser displaying the info on the screen, crashed.''
183
** you can try to restart midori this way:
184
*** ps aux | grep midori
185
*** kill $pid
186
*** /home/dashboard/bin/osd_dashboard
187
** If this also doesn't work, restart the machine.
188
189
190
### dashboard-o3
191
192
* Raspberry Pi 3B+
193
* IP: `10.160.65.207`
194
195
#### Content of /home/tux/.xinitrc
196
```
197
#!/bin/bash
198
199
unclutter &
200
openbox &
201
xset s off
202
xset -dpms
203
sleep 5
204
url="https://openqa.opensuse.org?group=openSUSE Tumbleweed\$|openSUSE Leap [0-9]{2}.?[0-9]*\$|openSUSE Leap.\*JeOS\$|openSUSE Krypton|openQA|GNOME Next&limit_builds=2&time_limit_days=14&&show_tags=1&fullscreen=1#build-results"
205
chromium --kiosk "$url" &
206
207
while sleep 300 ; do
208
        xdotool windowactivate $(xdotool search --class Chromium)
209
        xdotool key F5
210
        xdotool windowactivate $(xdotool getactivewindow)
211
done
212
```
213
214
#### Content of /usr/share/lightdm/lightdm.conf.d/50-suse-defaults.conf
215
```
216
[Seat:*]
217
pam-service = lightdm
218
pam-autologin-service = lightdm-autologin
219
pam-greeter-service = lightdm-greeter
220
xserver-command=/usr/bin/X
221
session-wrapper=/etc/X11/xdm/Xsession
222
greeter-setup-script=/etc/X11/xdm/Xsetup
223
session-setup-script=/etc/X11/xdm/Xstartup
224
session-cleanup-script=/etc/X11/xdm/Xreset
225
autologin-user=tux
226 4 szarate
autologin-timeout=0
227
```
228
229
## Old stuff:
230
231
Some older links:
232
* overview of current openQA SLE12SP5 tests with progress ticket references: https://openqa.suse.de/tests/overview?distri=sle&version=12-SP5&groupid=139&groupid=142
233
* fate tickets for SLE12SP5 feature testing: based on http://s.qa.suse.de/qa_sle_functional_feature_tests_sle12sp5 new report based on all tickets with milestone before SLE12SP5 GM, http://s.qa.suse.de/qa_sle_functional_feature_tests_sle15sp1 for SLE15SP1
234 1 okurz
* only "blocker" or "shipstopper" bugs on "interesting products" for SLE15 http://s.qa.suse.de/qa_sle_functional_bug_query_sle15_2, http://s.qa/qa_sle_bugs_sle12_2 for SLE12
235
* Better organization of planned work can be seen at the [SUSE QA](https://progress.opensuse.org/projects/suseqa) project (which is not public).