Project

General

Profile

Actions
Copy link

action #121747

closed

[security] Migrate security testing to using settings in job group and yaml schedules and cleanup code

Added by tjyrinki_suse 12 months ago. Updated 10 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
pstivanin
Category:
-
Target version:
-
Start date:
Due date:
% Done:

100%

Estimated time:
64.00 h
Difficulty:
hard

Description

Currently security tests are not using the more modern openQA ways of scheduling tests, making it more cumbersome to adapt tests to new products or make changes.
We should move from using hard coded execution logic in perl code to yaml definitions.

Acceptance Criteria

  1. Remove the following functions from main_common_pm: load_security_* if possible
  2. Check if our job groups define extra parameters that could be dropped, or moved to common/nested settings for easier manageability (see opensuse-jobgroups/job_groups/alp.yaml for an example)
Actions
Copy link
 #1

Updated by pstivanin 12 months ago

  • Status changed from New to In Progress
  • Assignee set to pstivanin
Actions
Copy link
 #2

Updated by pstivanin 12 months ago

  • % Done changed from 0 to 30
Actions
Copy link
 #3

Updated by pstivanin 12 months ago

  • % Done changed from 30 to 50
Actions
Copy link
 #4

Updated by pstivanin 12 months ago

  • % Done changed from 50 to 60
Actions
Copy link
 #6

Updated by pstivanin 12 months ago

  • % Done changed from 60 to 70

Since it's not possible to migrate to yaml code that contains &&, ||, etc, we cannot migrate everything from main_common. For example, code like the following:

if (get_var("SECURITY_TEST") =~ /^crypt_/ && !is_opensuse && (get_var("BETA") || check_var("FLAVOR", "Online-QR")));

or

if (get_var("FIPS_ENABLED") && get_var("JEOS"));

must remain in main_common.

Actions
Copy link
 #7

Updated by pstivanin 12 months ago

Since load_security_console_prepare contains non migratable code, the following tests cannot be ported to yaml:

load_security_tests_crypt_core
load_security_tests_crypt_web
load_security_tests_crypt_kernel
load_security_tests_crypt_x11
load_security_tests_crypt_firefox
load_security_tests_crypt_openjdk
load_security_tests_crypt_tool
load_security_tests_crypt_libtool
load_security_tests_ipsec
load_security_tests_mmtest
load_security_tests_apparmor
load_security_tests_apparmor_profile
load_security_tests_yast2_apparmor
load_security_tests_yast2_users
load_security_tests_lynis
load_security_tests_openscap
load_security_tests_cc_audit_test
load_security_tests_check_kernel_config
load_security_tests_pam
load_security_tests_create_swtpm_hdd
load_security_tests_swtpm
load_security_tests_grub_auth
load_security_tests_tpm2
load_security_tests_fips_setup
load_security_tests_cc_audit_remote_libvirt
load_security_tests_mok_enroll

the following tests, though, seem to be migratable to yaml:

load_security_tests_crypt_krb5kdc
load_security_tests_crypt_krb5server
load_security_tests_crypt_krb5client
load_security_tests_selinux
load_security_tests_ima_measurement
load_security_tests_ima_appraisal
load_security_tests_evm_protection
load_security_tests_system_check
Actions
Copy link
 #8

Updated by pstivanin 12 months ago

  • % Done changed from 70 to 80

The following tests have been ported to yaml:

load_security_tests_crypt_krb5kdc
load_security_tests_crypt_krb5server
load_security_tests_crypt_krb5client
load_security_tests_selinux
load_security_tests_ima_measurement
load_security_tests_ima_appraisal
load_security_tests_evm_protection
load_security_tests_system_check

Only things left are to adjust main_common, check if opensuse is using those and update the jsons to point the new yaml.

Actions
Copy link
 #9

Updated by pstivanin 12 months ago

  • % Done changed from 80 to 90
Actions
Copy link
 #10

Updated by pstivanin 12 months ago

  • Status changed from In Progress to Resolved
  • % Done changed from 90 to 100

PRs merged.

Actions
Copy link
 #11

Updated by punkioudi 12 months ago

  • Status changed from Resolved to In Progress

Sorry for the noise!

In the latest 15-SP5 runs there were some failures because of "wrong" expected qcow2 images:
https://openqa.suse.de/tests/10186640
https://openqa.suse.de/tests/10187557
https://openqa.suse.de/tests/10186338
https://openqa.suse.de/tests/10187559
https://openqa.suse.de/tests/10185735
https://openqa.suse.de/tests/10185731

I re-open this ticket as they seem to be part of this cleanup, if not I can open a new one :)

Actions
Copy link
 #12

Updated by pstivanin 12 months ago

  • % Done changed from 100 to 80

Thanks Anna, I'll look into it :)

Actions
Copy link
 #13

Updated by pstivanin 12 months ago

  • % Done changed from 80 to 100
Actions
Copy link
 #14

Updated by pstivanin 12 months ago

  • Status changed from In Progress to Resolved

PR has been merged. With the next run it will pick up the changes and everything should be fine.
Closing the ticket for now.

Actions
Copy link
 #15

Updated by openqa_review 11 months ago

  • Status changed from Resolved to Feedback

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: security_tpm2_keylime_uefi@uefi
https://openqa.suse.de/tests/10256004#step/boot_to_desktop/1

To prevent further reminder comments one of the following options should be followed:

  1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
  2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
  3. The bugref in the openQA scenario is removed or replaced, e.g. label:wontfix:boo1234

Expect the next reminder at the earliest in 28 days if nothing changes in this ticket.

Actions
Copy link
 #16

Updated by pstivanin 11 months ago

  • Status changed from Feedback to Resolved
Actions
Copy link
 #17

Updated by tjyrinki_suse 10 months ago

  • Parent task set to #124137
Actions
Copy link

Also available in: Atom PDF