Project

General

Profile

coordination #71242

[epic] Unify validation in the installed system

Added by JERiveraMoya about 1 year ago. Updated 6 months ago.

Status:
In Progress
Priority:
Normal
Assignee:
Target version:
Start date:
2021-01-08
Due date:
% Done:

90%

Estimated time:
(Total: 0.00 h)

Description

As a Quality Engineer I would like to validate that all the promises that the installer does are met in the installed system.

Therefore we need a set of validations which met the following criteria:

  • Single-purpose validation, enough atomic to be used in any other scenario.
  • Utilize test data and have a common structure for all the scenarios.
  • Use always common library for parsing the output (currently there are 2-3 implementation of the same thing which need to be unified).
  • Validations are focused on the verification of the configuration that the installer does, change the system and validate it is not in scope.
  • Validation are multi-architecture and product.
  • Validations are focused mainly in what is visible in the stages of the installer with the following order of preference:
    • User selection/actions: test data for those should be reusable by YuiRestClient.
    • defaults displayed in the stages of the installer for a specific scenario (any control that is visible which the user didn't interact with)
  • Additional validation can be focused on (the ones not visible in any stage of the installer):
    • documentation and other defaults of the test scenario.
    • any other thing the tester/developers would consider important as well.
  • Validation are easy to expand adding a few more test data and perhaps some code, i.e: check a new setting in the same file should be cheap.
  • Consider speed of validation, reading just once the output and doing the rest in Perl.
  • When validation fails should be very easy to understand the problem (expected vs actual results)
  • All scenarios in YaST group should apply those validations. Consider not to repeat the validation in similar scenarios, specially for defaults.
  • Stages of the installer could be related to more than one screen, so validation modules will not match 1 to 1 with a dialog during installation, instead they will cover specific functionality of a particular stage, or in other words, all the dialogs related with some stage, i.e.: validate networking, validate software.
  • Scope of this validation is huge, so main target will be partitioning, networking, software, services and additionally some others like Booting options, Timezone, etc.

Examples:

  • Test case to validate stage Network Settings configuration for interface, dhcp, hostname, routing could be checked in the installed system.
  • Test case to validate stage System role when selecting Text Mode we could figure out a simple validation of X11 and find out what should not be in the system comparing with a gnome installation.
  • Test case to validate stage Suggested Partitioning subvolume actions could be a good candidate, clicking on 'see details' there are info that we could check.
  • Test case to validate stage Installation Settings could be split in multiple modules to check every section. Clicking on each item there are more promises that the software does that could be tested individually and deserve attention even if the user does not navigate to them during the installation, but those are the promises anyway.

Subtasks

action #81902: Check available EULA translations in SLESClosedriafarov

action #88313: Verify network configuration in the installed systemClosedJERiveraMoya

action #88319: Unify config files validation implemented in verify_config_files and yast2_kdump.pmClosedriafarov

action #88615: Improve validation of System Role by checking that the exact number of patterns were installedClosedoorlov

action #91755: Split functionality for validating partitioning in partitions_validator_utils into more reusable functionsClosedJERiveraMoya

action #91758: Validate raid partitioning layout using test data instead of regexClosedriafarov

action #92101: Wipe dasd disk before partitioningClosedJERiveraMoya

action #92107: Validate encryption when the partition is not mountedClosedsyrianidou_sofia

action #92116: Remove validate_ext4_fs and validate_file_systemRejected

action #94069: Unify test data for guided partitioningNew

History

#1 Updated by JERiveraMoya about 1 year ago

  • Description updated (diff)

#2 Updated by riafarov about 1 year ago

  • Project changed from openQA Tests to qe-yast
  • Category deleted (Refactor/Code Improvements)

#3 Updated by JERiveraMoya 12 months ago

  • Subject changed from [y] Rethink approach to validate the installed system to [epic] Unify validation in the installed system
  • Description updated (diff)

#4 Updated by JERiveraMoya 12 months ago

  • Description updated (diff)

#5 Updated by riafarov 11 months ago

  • Tracker changed from action to coordination

#6 Updated by JERiveraMoya 8 months ago

  • Status changed from New to In Progress
  • Assignee set to JERiveraMoya

Also available in: Atom PDF