QA » openQA Project » openQA Tests

msmeissn wrote:

we now have a "stig" hardening in the scap-security-guide

can you add testing for this to SLES, for both:

• detection mode
• mitigation mode

and also test if regular SLES stuff continues to work after running mitigation.
(e.,g. having a installation flavor regular + stig hardening mitigation )

@msmeissn , could please offer more info (any docs/links) on the "scap-security-guide"?
We do not have any idea atm. Thanks!

After investigation found these helpful links:

1. confluence page: https://confluence.suse.com/display/SecurityCertifications/Hardening+workshop+preparation https://confluence.suse.com/display/GEHC/General+Security+Discussions
2. JIRA feature: https://jira.suse.com/browse/PM-2390 https://jira.suse.com/browse/PM-245
3. sles15sp3 GM # zypper se -s scap-security-guide

...

1. man page of scap-security-guide ... Profiles in Guide to the Secure Configuration of SUSE Linux Enterprise 15 ... Additional details can be found on the projects wiki page: https://www.github.com/OpenSCAP/scap-security-guide/wiki ...

ok, simple approaches:

install openscap-utils

on SLE15 all servicepacks: ( just replace sle15 by sle12 in SLE12)
oscap xccdf eval --profile stig /usr/share/xml/scap/ssg/content/ssg-sle15-ds.xml

this is the basic evaluation and will print to stdout. There are some output optipons too which could be used for easier scripting if needed.

There is a mitigation mode:
oscap xccdf remediate --profile stig /usr/share/xml/scap/ssg/content/ssg-sle15-ds.xml

DANGER NOTE: this WILL change your system and might disallow logins or similar, so only use in scratch vms for testing or when you are able to recover it.

You can take a look at the existing openscap tests in openqa too and inject the ssg-sle15-ds.xml or ssg-sle12-ds.xml there

Got it, thanks for the info.

Testing coverage discussion:
On Tue, Dec 21, 2021 at 03:36:42PM +0800, llzhao wrote:

Hi Marcus,

I am automating the poo#93886 - [sle][security][sle15sp4]automate
testing of scap-security-guide (opened by you before and I'm sorry for
the late).

I have some questions need your confirmation.

1. You had suggested 2 testing modes

   - One is detection:

   Run "# oscap xccdf eval --profile stig /.../ssg-sle15-ds.xml".

   This mode can be executed well and print msg to stdout and the
   return value is "2".

   For the eval outputs [2] there are lots of
   "fail/notchecked/notapplicable/..." except for "pass".

   Can I take this testing pass thought there are "fail/..." and the
   return value is "2"?

Here we can expect some fails, as the system is not fully compliant
before remediation.

I would however perhaps track the output list to see if we regress
between version updates. (e.g. track a baseline of pass/fail/not
applicable here)

*- Another is mitigation*:

Run "# oscap xccdf remediate --profile stig /.../ssg-sle15-ds.xml"

This mode can not be executed well and reports the following msg:

    WARNING: Datastream component
    'scap_org.open-scap_cref_pub-projects-security-oval-suse.linux.enterprise.15.xml'
    points out to the remote
    'https://ftp.suse.com/pub/projects/security/oval/suse.linux.enterprise.15.xml'.
    Use '--fetch-remote-resources' option to download it.
    WARNING: Skipping
    'https://ftp.suse.com/pub/projects/security/oval/suse.linux.enterprise.15.xml'
    file which is referenced from datastream
    WARNING: Skipping
    ./pub-projects-security-oval-suse.linux.enterprise.15.xml file
    which is referenced from XCCDF content
    OpenSCAP Error: Could not find latest TestResult element.
    [/home/abuild/rpmbuild/BUILD/openscap-1.3.5/src/XCCDF/xccdf_session.c:1859]

So I tried this cmd instead: "# oscap xccdf eval --profile stig
--remediate /.../ssg-sle15-ds.xml"
This cmd can be executed and print msg to stdout and the return
value "1", see [2]

For the remediate outputs [2] there are lots of "fail/error/..."
except for "fixed".

Should I take this testing as pass?

So there should at least be no errors I would hope ... :/

We need to go through those. Can you open one bug with the results?

I'm asking these 2 questions as I don't know how to handle the "Result".

1. For testing suggestion "test if regular SLES stuff continues to work
   after running mitigation"

   I will check after the mitigation the system still can be updated or
   can run "# oscap remediate/evaluation"

   Is it OK?

I think good as first step...

Can we inject this kind of "remediated" system as a base installation
into some sort of basic tests?

1. Should this test case be run on all openQA arches (x86_64, s390x, ppc64le and aarch64)?

Yes please, although I do not expect differences.

1. openQA runs for released products

   Usually Security QEs only focus on the testings for products under
   development, such as we are testing sle15sp4.

   QE maintenance team will test released products, such as "sle12 spx"
   and sle15 "sp0/1/2/3".

   In this poo the test requirement is for all sle service packs.

   So I decid only automate it for sle15sp4 then will inform QE
   maintenance team to add test runs for other sle12/15 service packs

   Are you OK with it?

Yes, this sounds good.

Actually this is wanted by Cert Team that we test the new versions of scap-security-guide.

FYI:

[1] https://progress.opensuse.org/issues/93886

([sle][security][sle15sp4]automate testing of scap-security-guide)

[2] see attachment file

phase 1: integrate oscap "eval" / "eval remote" / "remediate" / "eval after remediate" test cases into openQA; support on 4 arches
phase 2: check with the baselines after "eval"
phase 2: run basic tests on the "remediated" system
So will open 3 poos to track.