Project

General

Profile

Actions

action #124101

closed

[qe-core] Analyze coverage of glibc

Added by szarate over 1 year ago. Updated about 1 year ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Spike/Research
Target version:
Start date:
2023-02-08
Due date:
% Done:

80%

Estimated time:
Difficulty:
Sprint:
QE-Core: March Sprint (Mar 08 - Apr 05)

Description

Tasks

  • Define a timebox during the planning
  • Check for established tests that exist already (Like LTP for Glibc)
  • Check if the glibc package has tests enabled on OBS

Acceptance Criteria

  1. Document the research on this ticket
  2. Propose a test plan for Glibc at a functional level

Notes

  • Don't think about running a Fuzzer :)
  • Verify interoperability (in case of ISV building their own thing)
  • Test core features of GLibc, i.e., running torture tests.
  • Consider building glibc during the test
  • Consider using single and multiple core configurations
tests/console/gdb.pm
tests/console/glibc_sanity.pm
tests/console/glibc_tunables.pm
tests/jeos/glibc_locale.pm
tests/kernel/ulp_openposix.pm
tests/kernel/ulp_threads.pm

Files

LTSS-updates-count.txt (3.72 KB) LTSS-updates-count.txt hrommel1, 2023-02-08 14:13

Related issues 1 (0 open1 closed)

Copied to openQA Tests - action #125606: [qe-core] Test glibc on 11SP4Resolveddzedro2023-02-08

Actions
Actions #1

Updated by szarate over 1 year ago

  • Project changed from 175 to openQA Tests
Actions #2

Updated by szarate over 1 year ago

  • Sprint set to QE-Core: February Sprint (Feb 08 - Mar 08)
  • Tags set to qe-core-february-sprint
  • Category set to Spike/Research
  • Target version set to QE-Core: Ready
Actions #3

Updated by hrommel1 over 1 year ago

I have attached the list of updates that had been either approved or rejected by Maintenance QA in the life time of SLE 11 SP4 LTSS.
The first column contains the number of occurrences, the second column contains the names of the (source) rpms.
Though the Extreme Core offering focuses on kernel, openssl and glibc only, this might give you an idea of the most important/busy packages during LTSS.

Actions #4

Updated by szarate over 1 year ago

  • Description updated (diff)
Actions #5

Updated by szarate over 1 year ago

  • Subject changed from [qe-core] Analize coverage of glibc to [qe-core] Analyze coverage of glibc
  • Description updated (diff)
  • Status changed from New to Workable
Actions #6

Updated by szarate over 1 year ago

  • Description updated (diff)
Actions #7

Updated by szarate over 1 year ago

  • Description updated (diff)
Actions #8

Updated by ph03nix over 1 year ago

  • Status changed from Workable to In Progress
  • Assignee set to ph03nix
Actions #9

Updated by ph03nix over 1 year ago

Build time tests are present for all SLES version

All SLES versions down to 11-SP4 have a build time test in the %check section of the above referenced spec file.

A remaining question is how can we (QA) rely on those tests which are outside of our control?

Actions #10

Updated by ph03nix over 1 year ago

tests/console/glibc_sanity.pm checks the sanity of the glibc file on the system under test. It runs the file and therefore checks for overall sanity of the file.

This is nice and sufficient as a sanity test.

Actions #11

Updated by ph03nix over 1 year ago

tests/jeos/glibc_locale.pm tests the locale settings but not so much glibc itself and might not be too useful.

Actions #12

Updated by ph03nix over 1 year ago

A Google search didn't resulted in any meaningful results for a comprehensive glibc test suite

We're having the openposix test suite already: https://openqa.suse.de/tests/10455037

Actions #13

Updated by ph03nix over 1 year ago

So the only part we are missing is a simple integration test to check, if the glibc integrates well into the system. This is something that we can do in reasonable time (1 developer day) ourselves by writing a small C program, compile it on the machine and run it.

Running in addition a couple of tools which are shipped should be sufficient for a simple integration test.

Actions #14

Updated by ph03nix over 1 year ago

In addition there is tests/console/glibc_tunables.pm, which could act already as a small integration test, but I would recommend to write a dedicated one which is doing a bit more. Perhaps pick 3-4 library functions from glibc which are commonly used and test those as well.

Actions #15

Updated by ph03nix over 1 year ago

  • Status changed from In Progress to Feedback

Summary

  1. There are build time tests present for all SLES versions, we only need to ensure that those will always be executed (they are outside of QA control)
  2. Some simple tests exists already and can be re-used: glibc_sanity.pm (required) and glibc_tunabled.pm (IMHO optional and might not even work on 11-SP4)
  3. The ltp_openposix tests from the Kernel group runs a vast number of POSIX compliance tests that should test the glibc intensively - This might be the most comprehensive test suite we already have.
  4. We might want to write ourselves a simple integration test, that checks some pre-existing tools if they work (just running --version on a handful is enough)
  5. (actually 4.1) Write a simple compile test of a predefined C program, which uses a handful of library functions

A reasonable coverage of glibc against major regressions can be easily achieved by using the above stated tests and by extending those with a simple compile+run test of a simple C program.

Test plan

  1. Ensure the build tests in OBS are running
  2. Schedule our existing test runs
  1. Write a simple test which compiles a predefined C program, which uses a handful of library functions. We should use gcc or clang, so that it indirectly would also test the glibc on the system itself.
Actions #16

Updated by ph03nix over 1 year ago

  • Status changed from Feedback to In Progress
Actions #17

Updated by ph03nix over 1 year ago

ulp_openposix might be already a comprehensive test suite: https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/tests/kernel/ulp_openposix.pm

ltp_openposix tests glibc and can be scheduled via tests/kernel/boot_ltp. See e.g. https://openqa.suse.de/tests/10455037.

Actions #18

Updated by ph03nix over 1 year ago

  • Status changed from In Progress to Feedback
Actions #19

Updated by ph03nix over 1 year ago

  • % Done changed from 0 to 80
Actions #20

Updated by MDoucha over 1 year ago

ph03nix wrote:

ulp_openposix might be already a comprehensive test suite: https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/tests/kernel/ulp_openposix.pm

See e.g. https://openqa.suse.de/tests/10455037.

ltp_openposix and ulp_openposix are two different things. ltp_openposix tests glibc. ulp_openposix tests libpulp0 and glibc-livepatches.

In addition, there's also ltp_syscalls and ltp_syscalls_ipc which test lots of kernel syscalls both directly and through glibc wrapper functions.

Actions #21

Updated by ph03nix over 1 year ago

MDoucha wrote:

ltp_openposix and ulp_openposix are two different things. ltp_openposix tests glibc. ulp_openposix tests libpulp0 and glibc-livepatches.

In addition, there's also ltp_syscalls and ltp_syscalls_ipc which test lots of kernel syscalls both directly and through glibc wrapper functions.

Thank you, I updated the previous statements and recommended to use ltp_openposix for regression tests on glibc

Actions #22

Updated by szarate over 1 year ago

One question is, will we need an LTP image to run the ltp_openposix tests? I want to avoid testing the same thing twice.

Actions #23

Updated by MDoucha over 1 year ago

szarate wrote:

One question is, will we need an LTP image to run the ltp_openposix tests? I want to avoid testing the same thing twice.

Yes, the openposix tests are bundled in the LTP package in QA:Head and the jobs boot from standard LTP disk image.

Actions #25

Updated by szarate over 1 year ago

  • Sprint changed from QE-Core: February Sprint (Feb 08 - Mar 08) to QE-Core: March Sprint (Mar 08 - Apr 05)
Actions #26

Updated by szarate over 1 year ago

Actions #27

Updated by szarate over 1 year ago

we can follow up on #125606

Actions #28

Updated by jstehlik about 1 year ago

  • Status changed from Feedback to Resolved
Actions

Also available in: Atom PDF