Project

General

Profile

action #89479

[qe-core] test fails in sssd - Fix sssd testing with openldap, remove use of python-pam

Added by tjyrinki_suse 8 months ago. Updated 3 months ago.

Status:
Resolved
Priority:
Urgent
Assignee:
Category:
New test
Target version:
Start date:
Due date:
% Done:

90%

Estimated time:
Difficulty:

Description

Observation

openQA test in scenario sle-15-SP3-Online-x86_64-extra_tests_textmode_phub@64bit fails in
sssd

In the past, sssd used to be tested using python-pam module, however it's unmantained and it belonged in the package hub repo, see https://bugzilla.suse.com/show_bug.cgi?id=1180120 for more information

It is still affecting other teams like public cloud, but on 15SP2 it is still being run for maintenance test repo and passing at https://openqa.suse.de/tests/5636854

In the end, this might be a complete rewrite of the sssd tests, rendering tests/sysauth/sssd.pm useless. A bit of background, those tests are a bit extensive but they were written by a developer that is long gone from the company, investing time in fixing or looking for an alternative to python-pam and rewrite the test suite is more expensive than switching to directly testing the service against the system (by means of directly using pam/passwd/getent)

Reproducible

Fails since (at least) Build 14.2

Further details

Always latest result in this scenario: latest

Suggestions

Follow the approach used by this pr, that sets up the Directory Service (389-ds in this case, but there's also the information for openldap).

Acceptance criteria

AC1: sssd is being tested without other dependencies than pam for all codestreams (openSUSE and SLE)
AC2: Move sssd tests from extra_tests_textmode_phub to extra_tests_textmode
AC3: re-enable sssd testing on public cloud (Or team is informed that they can be re-enabled)

Followup during review

See how https://progress.opensuse.org/issues/89320 was implemented and evaluate whether a subticket is needed to unify the testing strategy or they are tackling two different approaches


Related issues

Related to openQA Tests - action #55115: [qe-core][functional] test fails in sssd - Test fails switching to serial terminalResolved2019-08-05

Related to openQA Tests - action #13168: [qe-core][opensuse][sle][functional] Sudo and LDAP/SSSD for sudoersWorkable5000-01-01

Related to openQA Tests - action #93210: [migration][qe-core] sssd openldap/389-ds basic testing, modify to be suitable for migration testingNew

History

#3 Updated by tjyrinki_suse 8 months ago

  • Target version set to QE-Core: Ready

#4 Updated by tjyrinki_suse 8 months ago

  • Description updated (diff)

#5 Updated by tjyrinki_suse 8 months ago

  • Related to action #55115: [qe-core][functional] test fails in sssd - Test fails switching to serial terminal added

#6 Updated by tjyrinki_suse 8 months ago

  • Description updated (diff)

#8 Updated by tjyrinki_suse 8 months ago

  • Description updated (diff)

#9 Updated by tjyrinki_suse 8 months ago

  • Description updated (diff)

#10 Updated by punkioudi 7 months ago

  • Assignee set to punkioudi

#11 Updated by punkioudi 7 months ago

Hi tjyrinki_suse!
Could you provide some more clarifications regarding the acceptance criteria of this ticket?
If I have understood correctly, the point is to fix the sssd testing, but I miss some information like:

  • The places that this test fails
  • The new requirements for this test
  • Link for the numbers in the description
  • Acceptance criteria

Thanks :)

#12 Updated by tjyrinki_suse 7 months ago

  • Description updated (diff)
  • Status changed from Workable to New

updated description, setting New until details and plan seem solid

#13 Updated by szarate 7 months ago

  • Project changed from SUSE QA to openQA Tests
  • Subject changed from [qe-core] Fix sssd testing with openldap, remove use of python-pam to [qe-core] test fails in sssd - Fix sssd testing with openldap, remove use of python-pam
  • Description updated (diff)
  • Category set to New test
  • Assignee deleted (punkioudi)

Description updated, unasigning for now Anna, until ticket is workable and/or clear enough

#14 Updated by tjyrinki_suse 7 months ago

  • Status changed from New to Workable

#15 Updated by tonyyuan 7 months ago

  • Assignee set to tonyyuan

#17 Updated by tonyyuan 7 months ago

The removal of python-pam test cases will not reduce test coverage as they basically test the same functionalities.
In the future, we could add test cases for offline user authentication and information lookup. Junit is also used here incorrectly and should be removed to simplify the test.

#18 Updated by tjyrinki_suse 7 months ago

  • Status changed from Workable to In Progress

#19 Updated by tonyyuan 6 months ago

I am rewriting the sssd test module based on discussion on the PR. https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/12179
Below are some ideas for the new test cases:

  1. local domain won't be tested as the feature is not recommended for use by sssd developer.
  2. use "ssh -l" to test pam authentication instead of "su -l" as root because "su -l" actually does not go through PAM rules.
  3. offline authentication and identity lookup will be covered.
  4. use systemd to start ldap, sssd instead of on command line to test normal service start/stop.

#20 Updated by tonyyuan 6 months ago

Test plan:

SSSD includes PAM and NSS modules to allow remote users to login and be recognized as valid users, including group membership. It also can also cache this information to allow offline authentication.

Below are test cases:

  1. nss_sss test: look up user information with getent/id.

  2. pam_sss test: ssh login localhost as remote user.

  3. offline authentication test: shutdown ldap server, ssh login localhost as remote user.

  4. write permission test: change remote user password with passwd

  5. sssd-sudo test: define a sudoers rule entries in ldap server which allow a remote user to open a file owned by another remote user (using /usr/bin/cat). The testing file can be created in testcase 2 above and only the remote owner has read permission.

The domain server can be either 389-ds or openldap, or both of them.

The major difference between this test and the sssd which is currently running in openQA is:

The test will cover more sssd functionalities and less connections with remote servers.

#21 Updated by szarate 6 months ago

Thanks for clarifying Tony!

Please also consider: #13168 for your test plan

#22 Updated by szarate 6 months ago

  • Related to action #13168: [qe-core][opensuse][sle][functional] Sudo and LDAP/SSSD for sudoers added

#23 Updated by tonyyuan 6 months ago

szarate wrote:

Thanks for clarifying Tony!

Please also consider: #13168 for your test plan

Ok, I will add sudo/sudoers testcases in this test plan.

#24 Updated by tonyyuan 6 months ago

How to automate these test cases:
The testcases will be running in a openQA job
Two options, both are doable:
Option 1. Setting up openldap/389-ds server in container and sssd in host. The testcases above are executed in host.
advantage: the test is self sufficient with no dependency

Option 2. Using/setting up dedicated openldap/389-ds server outside openQA. The test won't have to set up for itself.
advantage: make the test automation considerably easy
disadvantage: having dependency on dedicated external server

#25 Updated by tonyyuan 6 months ago

test cases and server configuration files are all placed in testopia
sssd openldap: https://bugzilla.suse.com/tr_show_case.cgi?case_id=1768711
sssd 389-ds: https://bugzilla.suse.com/tr_show_case.cgi?case_id=1768710

#26 Updated by szarate 6 months ago

I'll look to setting up the instance in cloud.suse.de today

#27 Updated by tonyyuan 5 months ago

sssd automation plan:

  1. As openldap will be discarded from sles. sssd automation test will be split into two test modules like sssd_openldap and sssd_389ds. Executing the two tests in one single openQA job. When openldap being discarded we just simply unscheduled sssd_openldap. We will not have to change code.

  2. As there are already some sssd tests developed by some squads running in OSD, we could name the two modules sssd_openldap_functional, sssd_389ds_functional.

  3. Setting up openldap/389-ds server in container and running the tests in single machine. It makes the automation development and maintenance much easier then multi-machines.

Below are verification run:
http://10.67.17.201/tests/1516
http://10.67.17.201/tests/1513
Note: The two modules require sshpass in Packagehub repo.

#28 Updated by szarate 5 months ago

Hi Tony, can you use expect instead of sshpass? you can check tests/console/sshd.pm to see how it is being used there, as it doesn't depend on PackageHub.

For the rest, it's looking really good!

#30 Updated by tonyyuan 5 months ago

  • % Done changed from 0 to 90

#31 Updated by coolgw 5 months ago

Before close this ticket.
I have some questions on the requirement of this ticket since we will base current implementation to integrate migration process.
1) what's the support status for following build version?
11sp4
12sp3
12sp4
2) what's support status for following platform?
s390
aarch
power
3) This ticket include sssd + Windows domain server scenario or not?(Or maybe we need another ticket?) Since i saw https://progress.opensuse.org/issues/34576 mention this scenario.

#32 Updated by tonyyuan 5 months ago

coolgw wrote:

Before close this ticket.
I have some questions on the requirement of this ticket since we will base current implementation to integrate migration process.
1) what's the support status for following build version?
11sp4
12sp3
12sp4

All versions newer than 12sp2 work.
I can not even find iso and repo for 11sp4.

2) what's support status for following platform?
s390
aarch
power
I don't have HW test. If you have you can try it. Just check out the latest code and schedule it as I told you. It will be done with 15 mins. It should work.

3) This ticket include sssd + Windows domain server scenario or not?(Or maybe we need another ticket?) Since i saw https://progress.opensuse.org/issues/34576 mention this scenario.

sssd + Windows is another ticket.

Again, the sssd test is designed for functional and maint update test, not for migration. It might not help as you expect. I suggest you no using it, but it's up to you.

#33 Updated by tjyrinki_suse 5 months ago

The PR is now merged. I've created #93210 as a follow-up.

Tony, can you work on scheduling the test?

See https://gitlab.suse.de/openqa/all-yaml and do PRs against qam-openqa-yaml and from qa-sle-functional-userspace.

For Tumbleweed job group at https://openqa.opensuse.org/group_overview/1 can be edited directly, or you can paste eg here what to add there and on which line.


The Samba AD sssd work was actually started a long time ago, and that ticket is 72169.

#34 Updated by tjyrinki_suse 5 months ago

  • Related to action #93210: [migration][qe-core] sssd openldap/389-ds basic testing, modify to be suitable for migration testing added

#36 Updated by tjyrinki_suse 5 months ago

The PR was also tested on tumbleweed, and at least 389-ds testing should continue to be enabled for 15-SP4 and Tumbleweed, so I'd remove the sysauth/sssd.pm from functional/extra_tests_textmode_phub.yaml schedule and add sssd_389ds_functional to it (because using sshpass from packagehub) and to add also sssd_389ds_functional to functional/extra_tests_textmode.yaml under tumbleweed_tests.

#37 Updated by tonyyuan 5 months ago

Will openldap be removed off sle15sp4?
If openldap remain in sle15sp4 we can run both openldap_sssd and 389_ds sssd in a job.

A new PR: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/12638

#38 Updated by tjyrinki_suse 5 months ago

openLDAP server is already in Legacy module only in 15-SP3, but now I don't find authorative answer.

It's likely best to keep the test for now in 15-SP4 as well.

#40 Updated by tonyyuan 4 months ago

Can we close this ticket?

#41 Updated by tjyrinki_suse 4 months ago

  • Status changed from In Progress to Resolved

Yes! Thank you a lot for the big work, Tony.

There are no new SP3/SP4 builds in sight soon now, so new tickets will be opened if by the time of those builds start there are new problems.

#42 Updated by okurz 4 months ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: extra_tests_textmode_phub
https://openqa.suse.de/tests/5989271

To prevent further reminder comments one of the following options should be followed:

  1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
  2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
  3. The label in the openQA scenario is removed

#43 Updated by openqa_review 3 months ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: extra_tests_textmode_phub
https://openqa.suse.de/tests/5989271

To prevent further reminder comments one of the following options should be followed:

  1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
  2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
  3. The label in the openQA scenario is removed

#44 Updated by okurz 3 months ago

  • Status changed from Resolved to Feedback

^

#45 Updated by tonyyuan 3 months ago

The failed sssd test was already removed off job schedule a while ago. The two new sssd functional tests take it's place.

cat schedule/functional/extra_tests_textmode_phub.yaml

name: extra_tests_in_textmode_phub
description: >
Maintainer: slindomansilla.
Extra tests about software in package hub
schedule:
- installation/bootloader_start
- boot/boot_to_desktop
- console/add_phub_extension
- console/python_scientific
- '{{wpa_supplicant}}'
- console/vmstat
- console/sssd_389ds_functional
- console/sssd_openldap_functional
- console/coredump_collect
conditional_schedule:
wpa_supplicant:
ARCH:
x86_64:
- console/wpa_supplicant

#46 Updated by szarate 3 months ago

  • Status changed from Feedback to Resolved

okurz wrote:

^

If you are going to reopen a ticket, please state why, ^ to denote see previous test, when actual previous test corresponds to a product already released that hasn't had any changes is not one.

#47 Updated by okurz 3 months ago

szarate wrote:

okurz wrote:

^

If you are going to reopen a ticket, please state why, ^ to denote see previous test, when actual previous test corresponds to a product already released that hasn't had any changes is not one.

Sure but the reminder comment states what can be done in these cases. In the meantime after a longer time new builds for 15SP4 have been triggered so the problem is solved already.

Also available in: Atom PDF