Project

General

Profile

Actions

coordination #20580

closed

[sle][functional][epic] sle 15 setup

Added by okurz almost 7 years ago. Updated over 3 years ago.

Status:
Resolved
Priority:
Urgent
Assignee:
Category:
New test
Start date:
2017-07-20
Due date:
2017-09-27
% Done:

100%

Estimated time:
(Total: 0.00 h)
Difficulty:

Description

goal

New job group Functional which should be filled with a useful test coverage.

current state

riafarov and okurz have migrated all test suites from "Functional: Server" job group to "Functional" sub group of SLE 15: https://openqa.suse.de/group_overview/110
We also have triggered all test suites for x86_64 architecture in order to get an overview of common issues. After identifying most common problems and fixing them for this job group, we may migrate other job groups in similar way.

tasks

  • DONE: dumped templates for functional job group, adjusted for sle15, load
  • DONE: temporarily adjusted sync and trigger cron job to run daily only, not every 5 minutes as most new builds won't bring changes to actually fix job failures
  • DONE: incorporate review process
  • DONE: focus on former create_hdd_minimal_base+sdk scenario as a major dependency for many other job groups, e.g. create_hdd_development (clone existing sle15 job, add repos from dist.suse.de with ADDON_URL or similar, etc.)
  • DONE: fix installation only test jobs
  • DONE: fix jobs previously relying on server packages, addons, modules by implementing a workaround based on custom repos and isos
  • DONE: crosscheck other job groups, e.g. "Regression: Autoyast"
  • DONE: delete the scenarios that do not apply anymore or rename and adjust accordingly
  • optional: implement a script based solution to compare SLE15 coverage with SLE12SP3 covering multiple job groups so that we can be sure we did not forget any important scenario

further details

okurz created new job group Functional which should be filled with a useful test coverage. To make sure we don't forget anything I would

1) dump_templates the existing sle 12 sp3 functional job group and load_templates that into the new group and then sort out the useless parts and change the scenarios that need to be adapted

Alternatives are
2) same as above but move everything into the corresponding "test development" job group of SLE 15: https://openqa.suse.de/group_overview/110
and then selectively move over jobs.

3) Pick scenario by scenario whatever is considered useful and working. Challenge: We also have triggered all probably need a script to compare the existing test suites for x86_64 architecture in order to get an overview of common issues. After identifying most common problems coverage with the current coverage, evaluate the difference and fixing them for this job group, make sure we may migrate other job groups in similar way.

Attached json file was used to create job templates.


Files

sle15.json (200 KB) sle15.json riafarov, 2017-08-01 11:50

Subtasks 14 (0 open14 closed)

action #20646: [sle][functional][sle15] Evaluate the task copying the test templates from SLE 12 SP3 to SLE 15Resolvedriafarov2017-07-20

Actions
action #20648: [sle][functional][sle15] Evaluate creating template entries on template file in os-autoinst-distri-opensuse for SLE 15Resolvedriafarov2017-07-20

Actions
action #20984: [sle][functional][sle15] adapt minimal_base+sdk_create_hdd test for sle 15Resolvedriafarov2017-08-01

Actions
action #21044: [sle][functional][sle15] adapt create_hdd_textmode test suite for sle 15Resolvedriafarov2017-08-03

Actions
action #23438: [sle][functional][sle15] decide how and start using SCC using SUSEConnectResolvedriafarov2017-08-17

Actions
action #23448: [sles][functional][sle15] Introduce module to select product during installationResolvedriafarov2017-08-18

Actions
action #23460: [sles][functional] test fails in change_desktop - Missing packagesResolvedokurz2017-08-18

Actions
action #23462: [sles][functional] test fails in welcome - broken frames in ncurses in yast installationRejectedokurz2017-08-18

Actions
action #23466: [sles][functional] test fails in addon_products_sle - Missing packages for SDK moduleRejected2017-08-18

Actions
action #23492: [sle][functional]test fails in scc_registration - openQA SLE media are missing SCC_URLResolvedriafarov2017-08-202017-09-27

Actions
action #23504: [sle][functional] Adapt WE tests for SLE 15Resolvedriafarov2017-08-21

Actions
action #20824: [sles][functional] test fails in change_desktop - Failed to select default product pattern, later then fails with booting into wrong sessionResolvedokurz2017-07-27

Actions
action #23856: [sle][functional] Adjust test to the changes in welcome screenResolvedriafarov2017-09-01

Actions
action #23938: [sle][functional] Remove workarounds for broken dependencies and fail accordinglyResolvedriafarov2017-09-062017-09-27

Actions

Related issues 8 (0 open8 closed)

Related to openQA Tests - action #20582: [sle][functional] 'if check_var('VERSION, 15)" should be changed to "sle_version_at_least('15')"Resolvedriafarov2017-07-18

Actions
Related to openQA Tests - action #20630: [sles][functional] test fails in start_install - Create a workaround for missing checksums and unsigned files and perform installationResolvedSLindoMansilla2017-07-20

Actions
Related to openQA Tests - action #23402: [sles][functional] Adjust create_hdd_gnome test suite for sle15Resolvedriafarov2017-08-16

Actions
Related to openQA Tests - action #23736: [sle][functional] Adjust autoinst profiles for sle 15Resolvedriafarov2017-08-28

Actions
Related to openQA Tests - action #24704: [sles][functional] test fails in partitioning - Adapt the test to handle the new GPT as default. Boot partition should be bigger.Rejectedjorauch2017-09-062017-09-27

Actions
Related to openQA Tests - action #24706: [sles][functional] test fails in first_boot - Adapt test to workaround / accept behavior to continue with console testsResolvedSLindoMansilla2017-09-062017-10-11

Actions
Related to openQA Tests - action #25158: [sles][opensuse][functional]test fails in yast2_apparmorResolvedokurz2017-09-112017-09-27

Actions
Related to openQA Tests - coordination #23886: [sle][functional][desktop][sle15][epic] initial setupResolvedqkzhu2017-10-122017-11-08

Actions
Actions

Also available in: Atom PDF