Project

General

Profile

Actions

coordination #91142

open

[qe-core][epic] add product QE exclusive tests to QEM runs to increase coverage

Added by hurhaj almost 3 years ago. Updated 10 days ago.

Status:
Workable
Priority:
Normal
Assignee:
-
Category:
New test
Target version:
-
Start date:
2021-05-31
Due date:
% Done:

66%

Estimated time:
(Total: 0.00 h)
Difficulty:

Description

After comparing test coverage of SLE 15 SP2 and SP3 in opneQA I identified several tests that are exclusive to either pre-release or post-release testing and could be easily used to increase our coverage just by using what we already have.

It is possible that are some errors and some of the tests from the list are already being used. It is also just my opinion, therefor review from other colleagues is more then welcome.

These are the tests that run only before release and could be used also after the release (product QE -> QEM)

QE Core:
tests/console/btrfsmaintenance.pm
tests/console/journald_fss.pm
tests/console/lvm_thin_check.pm
tests/console/ndctl.pm
tests/console/network_hostname.pm
tests/console/nvme_checks.pm
tests/console/openvswitch_ssl.pm
tests/console/snapper_cleanup_timeline.pm
tests/console/systemd_nspawn.pm
tests/console/verify_default_target.pm
tests/console/verify_network.pm
tests/console/vsftpd.pm

QE Kernel:
tests/btrfs-progs/generate_report.pm
tests/btrfs-progs/install.pm
tests/btrfs-progs/run.pm
tests/kernel/blktests.pm
tests/kernel/kdump.pm
tests/kernel/numa_irqbalance.pm
tests/kernel/pressure_stall_information.pm
tests/kernel/tuned.pm

QE Yast:
tests/console/yast2_cmdline.pm
tests/console/yast2_dns_server.pm
tests/console/yast2_kdump.pm
tests/console/yast2_lan_hostname.pm
tests/console/yast2_ntpclient.pm
tests/console/yast2_proxy.pm
tests/console/yast2_rmt.pm
tests/console/yast2_samba.pm
tests/console/yast2_settings.pm
tests/console/yast2_snapper_ncurses.pm
tests/console/yast2_vnc.pm
tests/network/wireguard.pm
tests/security/yast2_apparmor/manually_add_profile.pm
tests/security/yast2_apparmor/scan_audit_logs.pm
tests/security/yast2_apparmor/settings_disable_enable_apparmor.pm
tests/security/yast2_apparmor/settings_toggle_profile_mode.pm
tests/security/yast2_users/add_users.pm

QE Security:
tests/security/grub_auth/grub_authorization.pm
tests/security/ima/evmctl_ima_sign.pm
tests/security/ima/evm_protection_digital_signatures.pm
tests/security/ima/evm_protection_hmacs.pm
tests/security/ima/evm_setup.pm
tests/security/ima/evm_verify.pm
tests/security/ima/ima_appraisal_audit.pm
tests/security/ima/ima_appraisal_digital_signatures.pm
tests/security/ima/ima_appraisal_hashes.pm
tests/security/ima/ima_kernel_cmdline_hash.pm
tests/security/ima/ima_kernel_cmdline_template.pm
tests/security/ima/ima_measurement_audit.pm
tests/security/ima/ima_measurement.pm
tests/security/ima/ima_setup.pm
tests/security/ima/ima_verify.pm
tests/security/krb5/krb5_crypt_nfs_client.pm
tests/security/krb5/krb5_crypt_nfs_server.pm
tests/security/krb5/krb5_crypt_prepare.pm
tests/security/krb5/krb5_crypt_setup_client.pm
tests/security/krb5/krb5_crypt_setup_kdc.pm
tests/security/krb5/krb5_crypt_setup_server.pm
tests/security/krb5/krb5_crypt_ssh_client.pm
tests/security/krb5/krb5_crypt_ssh_server.pm
tests/security/lynis/lynis_analyze_system_audit.pm
tests/security/lynis/lynis_harden_index.pm
tests/security/lynis/lynis_perform_system_audit.pm
tests/security/lynis/lynis_setup.pm
tests/security/mokutil_sign.pm
tests/security/nproc_limits.pm
tests/security/pam/pam_basic_function.pm
tests/security/pam/pam_config.pm
tests/security/pam/pam_login.pm
tests/security/pam/pam_mount.pm
tests/security/pam/pam_su.pm
tests/security/selinux/audit2allow.pm
tests/security/selinux/chcat.pm
tests/security/selinux/chcon.pm
tests/security/selinux/enforcing_mode_setup.pm
tests/security/selinux/fixfiles.pm
tests/security/selinux/print_se_context.pm
tests/security/selinux/restorecon.pm
tests/security/selinux/selinux_setup.pm
tests/security/selinux/selinux_smoke.pm
tests/security/selinux/semanage_boolean.pm
tests/security/selinux/semanage_fcontext.pm
tests/security/selinux/semodule.pm
tests/security/selinux/sestatus.pm
tests/security/selinux/setsebool.pm
tests/security/swtpm/swtpm_env_setup.pm
tests/security/swtpm/swtpm_verify.pm
tests/security/tpm2/tpm2_engine/tpm2_engine_ecdsa_operation.pm
tests/security/tpm2/tpm2_engine/tpm2_engine_info.pm
tests/security/tpm2/tpm2_engine/tpm2_engine_random_data.pm
tests/security/tpm2/tpm2_engine/tpm2_engine_rsa_operation.pm
tests/security/tpm2/tpm2_engine/tpm2_engine_self_sign.pm
tests/security/tpm2/tpm2_env_setup.pm
tests/security/tpm2/tpm2_tools/tpm2_tools_auth.pm
tests/security/tpm2/tpm2_tools/tpm2_tools_encrypt.pm
tests/security/tpm2/tpm2_tools/tpm2_tools_self_contain_tool.pm
tests/security/tpm2/tpm2_tools/tpm2_tools_sign_verify.pm

[QAC]
tests/wicked/aggregate/ref/t19_teaming_ab_nsna_ping.pm
tests/wicked/aggregate/sut/t19_teaming_ab_nsna_ping.pm
tests/wicked/ipv6/ref/t01_managed_on.pm
tests/wicked/ipv6/ref/t02_managed_on.pm
tests/wicked/ipv6/sut/t01_managed_on.pm
tests/wicked/ipv6/sut/t02_managed_on.pm
tests/wicked/startandstop/sut/t15_bridge_ifreload_all_bond.pm
tests/wicked/startandstop/sut/t16_bridge_ifreload_physical.pm


Subtasks 10 (2 open8 closed)

qe-yam - action #91175: [qe-yast][qem] Add existing Product QE tests to QEMRejected

Actions
action #91178: [kernel-default][qem] Add existing kernel Product QE tests to QEMRejected

Actions
action #91190: [kernel][qem] Add existing btrfs-progs Product QE tests to QEMNewpcervinka

Actions
coordination #91193: [epic][qe-core][qem] Add existing console Product QE tests to QEMNew2021-05-31

Actions
action #93264: [qe-core][qem] Add journald_fss, openvswitch_ssl, vsftpd tests to QEMResolvedpunkioudi2021-05-31

Actions
action #93270: [qe-core][qem] Schedule vsftpd in s390x runsResolvedpunkioudi2021-07-21

Actions
action #93814: [qe-core][qem] Schedule network_hostname in maintenance runsResolvedpunkioudi2021-06-10

Actions
coordination #91199: [security][epic][qem] Add existing security Product QE tests to QEMResolved2021-06-10

Actions
action #93766: [security][qem] Add existing security Product QE tests to QEM for 15-SP3Resolvedpstivanin

Actions
action #93769: [security][qem] Add existing security Product QE tests to QEM for older SLE versions - 15-SP2, 12-SP5Resolvedpstivanin2021-06-10

Actions
Actions #1

Updated by tjyrinki_suse almost 3 years ago

  • Tracker changed from action to coordination
  • Subject changed from add product QE exclusive tests to QEM runs to increase coverage to [qe-core][epic] add product QE exclusive tests to QEM runs to increase coverage
Actions #2

Updated by tjyrinki_suse almost 3 years ago

  • Description updated (diff)
Actions #3

Updated by tjyrinki_suse almost 3 years ago

  • Category set to New test
  • Status changed from New to Workable
Actions

Also available in: Atom PDF