Project

General

Profile

coordination #91142

[qe-core][epic] add product QE exclusive tests to QEM runs to increase coverage

Added by hurhaj 6 months ago. Updated 3 months ago.

Status:
Workable
Priority:
Normal
Assignee:
-
Category:
New test
Target version:
-
Start date:
2021-05-31
Due date:
% Done:

45%

Estimated time:
(Total: 0.00 h)
Difficulty:

Description

After comparing test coverage of SLE 15 SP2 and SP3 in opneQA I identified several tests that are exclusive to either pre-release or post-release testing and could be easily used to increase our coverage just by using what we already have.

It is possible that are some errors and some of the tests from the list are already being used. It is also just my opinion, therefor review from other colleagues is more then welcome.

These are the tests that run only before release and could be used also after the release (product QE -> QEM)

QE Core:
tests/console/btrfsmaintenance.pm
tests/console/journald_fss.pm
tests/console/lvm_thin_check.pm
tests/console/ndctl.pm
tests/console/network_hostname.pm
tests/console/nvme_checks.pm
tests/console/openvswitch_ssl.pm
tests/console/snapper_cleanup_timeline.pm
tests/console/systemd_nspawn.pm
tests/console/verify_default_target.pm
tests/console/verify_network.pm
tests/console/vsftpd.pm

QE Kernel:
tests/btrfs-progs/generate_report.pm
tests/btrfs-progs/install.pm
tests/btrfs-progs/run.pm
tests/kernel/blktests.pm
tests/kernel/kdump.pm
tests/kernel/numa_irqbalance.pm
tests/kernel/pressure_stall_information.pm
tests/kernel/tuned.pm

QE Yast:
tests/console/yast2_cmdline.pm
tests/console/yast2_dns_server.pm
tests/console/yast2_kdump.pm
tests/console/yast2_lan_hostname.pm
tests/console/yast2_ntpclient.pm
tests/console/yast2_proxy.pm
tests/console/yast2_rmt.pm
tests/console/yast2_samba.pm
tests/console/yast2_settings.pm
tests/console/yast2_snapper_ncurses.pm
tests/console/yast2_vnc.pm
tests/network/wireguard.pm
tests/security/yast2_apparmor/manually_add_profile.pm
tests/security/yast2_apparmor/scan_audit_logs.pm
tests/security/yast2_apparmor/settings_disable_enable_apparmor.pm
tests/security/yast2_apparmor/settings_toggle_profile_mode.pm
tests/security/yast2_users/add_users.pm

QE Security:
tests/security/grub_auth/grub_authorization.pm
tests/security/ima/evmctl_ima_sign.pm
tests/security/ima/evm_protection_digital_signatures.pm
tests/security/ima/evm_protection_hmacs.pm
tests/security/ima/evm_setup.pm
tests/security/ima/evm_verify.pm
tests/security/ima/ima_appraisal_audit.pm
tests/security/ima/ima_appraisal_digital_signatures.pm
tests/security/ima/ima_appraisal_hashes.pm
tests/security/ima/ima_kernel_cmdline_hash.pm
tests/security/ima/ima_kernel_cmdline_template.pm
tests/security/ima/ima_measurement_audit.pm
tests/security/ima/ima_measurement.pm
tests/security/ima/ima_setup.pm
tests/security/ima/ima_verify.pm
tests/security/krb5/krb5_crypt_nfs_client.pm
tests/security/krb5/krb5_crypt_nfs_server.pm
tests/security/krb5/krb5_crypt_prepare.pm
tests/security/krb5/krb5_crypt_setup_client.pm
tests/security/krb5/krb5_crypt_setup_kdc.pm
tests/security/krb5/krb5_crypt_setup_server.pm
tests/security/krb5/krb5_crypt_ssh_client.pm
tests/security/krb5/krb5_crypt_ssh_server.pm
tests/security/lynis/lynis_analyze_system_audit.pm
tests/security/lynis/lynis_harden_index.pm
tests/security/lynis/lynis_perform_system_audit.pm
tests/security/lynis/lynis_setup.pm
tests/security/mokutil_sign.pm
tests/security/nproc_limits.pm
tests/security/pam/pam_basic_function.pm
tests/security/pam/pam_config.pm
tests/security/pam/pam_login.pm
tests/security/pam/pam_mount.pm
tests/security/pam/pam_su.pm
tests/security/selinux/audit2allow.pm
tests/security/selinux/chcat.pm
tests/security/selinux/chcon.pm
tests/security/selinux/enforcing_mode_setup.pm
tests/security/selinux/fixfiles.pm
tests/security/selinux/print_se_context.pm
tests/security/selinux/restorecon.pm
tests/security/selinux/selinux_setup.pm
tests/security/selinux/selinux_smoke.pm
tests/security/selinux/semanage_boolean.pm
tests/security/selinux/semanage_fcontext.pm
tests/security/selinux/semodule.pm
tests/security/selinux/sestatus.pm
tests/security/selinux/setsebool.pm
tests/security/swtpm/swtpm_env_setup.pm
tests/security/swtpm/swtpm_verify.pm
tests/security/tpm2/tpm2_engine/tpm2_engine_ecdsa_operation.pm
tests/security/tpm2/tpm2_engine/tpm2_engine_info.pm
tests/security/tpm2/tpm2_engine/tpm2_engine_random_data.pm
tests/security/tpm2/tpm2_engine/tpm2_engine_rsa_operation.pm
tests/security/tpm2/tpm2_engine/tpm2_engine_self_sign.pm
tests/security/tpm2/tpm2_env_setup.pm
tests/security/tpm2/tpm2_tools/tpm2_tools_auth.pm
tests/security/tpm2/tpm2_tools/tpm2_tools_encrypt.pm
tests/security/tpm2/tpm2_tools/tpm2_tools_self_contain_tool.pm
tests/security/tpm2/tpm2_tools/tpm2_tools_sign_verify.pm

[QAC]
tests/wicked/aggregate/ref/t19_teaming_ab_nsna_ping.pm
tests/wicked/aggregate/sut/t19_teaming_ab_nsna_ping.pm
tests/wicked/ipv6/ref/t01_managed_on.pm
tests/wicked/ipv6/ref/t02_managed_on.pm
tests/wicked/ipv6/sut/t01_managed_on.pm
tests/wicked/ipv6/sut/t02_managed_on.pm
tests/wicked/startandstop/sut/t15_bridge_ifreload_all_bond.pm
tests/wicked/startandstop/sut/t16_bridge_ifreload_physical.pm


Subtasks

qe-yast - action #91175: [qe-yast][qem] Add existing Product QE tests to QEMRejected

action #91178: [kernel-default][qem] Add existing kernel Product QE tests to QEMRejected

action #91190: [kernel][qem] Add existing btrfs-progs Product QE tests to QEMNew

action #91193: [qe-core][qem] Add existing console Product QE tests to QEMIn Progress

action #93264: [qe-core][qem] Add journald_fss, openvswitch_ssl, vsftpd tests to QEMResolvedpunkioudi

action #93270: [qe-core][qem] Schedule vsftpd in s390x runsWorkablepunkioudi

action #95774: [qe-core][qem] Remove soft-fail from vsftpd.pm in s390x, when the bug is solvedBlocked

action #93814: [qe-core][qem] Schedule network_hostname in maintenance runsResolvedpunkioudi

coordination #91199: [security][epic][qem] Add existing security Product QE tests to QEMNew

action #93766: [security][qem] Add existing security Product QE tests to QEM for 15-SP3Workable

action #93769: [security][qem] Add existing security Product QE tests to QEM for older SLE versions - 15-SP2, 12-SP5New

History

#1 Updated by tjyrinki_suse 6 months ago

  • Tracker changed from action to coordination
  • Subject changed from add product QE exclusive tests to QEM runs to increase coverage to [qe-core][epic] add product QE exclusive tests to QEM runs to increase coverage

#2 Updated by tjyrinki_suse 6 months ago

  • Description updated (diff)

#3 Updated by tjyrinki_suse 6 months ago

  • Category set to New test
  • Status changed from New to Workable

Also available in: Atom PDF