action #124101
closed[qe-core] Analyze coverage of glibc
Added by szarate about 1 year ago. Updated about 1 year ago.
80%
Description
Tasks¶
- Define a timebox during the planning
- Check for established tests that exist already (Like LTP for Glibc)
- Check if the glibc package has tests enabled on OBS
Acceptance Criteria¶
- Document the research on this ticket
- Propose a test plan for Glibc at a functional level
Notes¶
- Don't think about running a Fuzzer :)
- Verify interoperability (in case of ISV building their own thing)
- Test core features of GLibc, i.e., running torture tests.
- Consider building glibc during the test
- Consider using single and multiple core configurations
tests/console/gdb.pm
tests/console/glibc_sanity.pm
tests/console/glibc_tunables.pm
tests/jeos/glibc_locale.pm
tests/kernel/ulp_openposix.pm
tests/kernel/ulp_threads.pm
Files
| LTSS-updates-count.txt (3.72 KB) LTSS-updates-count.txt | hrommel1, 2023-02-08 14:13 |
Updated by szarate about 1 year ago
- Sprint set to QE-Core: February Sprint (Feb 08 - Mar 08)
- Tags set to qe-core-february-sprint
- Category set to Spike/Research
- Target version set to QE-Core: Ready
Updated by hrommel1 about 1 year ago
- File LTSS-updates-count.txt LTSS-updates-count.txt added
I have attached the list of updates that had been either approved or rejected by Maintenance QA in the life time of SLE 11 SP4 LTSS.
The first column contains the number of occurrences, the second column contains the names of the (source) rpms.
Though the Extreme Core offering focuses on kernel, openssl and glibc only, this might give you an idea of the most important/busy packages during LTSS.
Updated by szarate about 1 year ago
- Subject changed from [qe-core] Analize coverage of glibc to [qe-core] Analyze coverage of glibc
- Description updated (diff)
- Status changed from New to Workable
Updated by ph03nix about 1 year ago
- Status changed from Workable to In Progress
- Assignee set to ph03nix
Updated by ph03nix about 1 year ago
Build time tests are present for all SLES version
All SLES versions down to 11-SP4 have a build time test in the %check section of the above referenced spec file.
A remaining question is how can we (QA) rely on those tests which are outside of our control?
Updated by ph03nix about 1 year ago
tests/console/glibc_sanity.pm checks the sanity of the glibc file on the system under test. It runs the file and therefore checks for overall sanity of the file.
This is nice and sufficient as a sanity test.
Updated by ph03nix about 1 year ago
tests/jeos/glibc_locale.pm tests the locale settings but not so much glibc itself and might not be too useful.
Updated by ph03nix about 1 year ago
A Google search didn't resulted in any meaningful results for a comprehensive glibc test suite
We're having the openposix test suite already: https://openqa.suse.de/tests/10455037
Updated by ph03nix about 1 year ago
So the only part we are missing is a simple integration test to check, if the glibc integrates well into the system. This is something that we can do in reasonable time (1 developer day) ourselves by writing a small C program, compile it on the machine and run it.
Running in addition a couple of tools which are shipped should be sufficient for a simple integration test.
Updated by ph03nix about 1 year ago
In addition there is tests/console/glibc_tunables.pm, which could act already as a small integration test, but I would recommend to write a dedicated one which is doing a bit more. Perhaps pick 3-4 library functions from glibc which are commonly used and test those as well.
Updated by ph03nix about 1 year ago
- Status changed from In Progress to Feedback
Summary¶
- There are build time tests present for all SLES versions, we only need to ensure that those will always be executed (they are outside of QA control)
- Some simple tests exists already and can be re-used:
glibc_sanity.pm(required) andglibc_tunabled.pm(IMHO optional and might not even work on 11-SP4) - The
ltp_openposixtests from the Kernel group runs a vast number of POSIX compliance tests that should test the glibc intensively - This might be the most comprehensive test suite we already have. - We might want to write ourselves a simple integration test, that checks some pre-existing tools if they work (just running
--versionon a handful is enough) - (actually 4.1) Write a simple compile test of a predefined C program, which uses a handful of library functions
A reasonable coverage of glibc against major regressions can be easily achieved by using the above stated tests and by extending those with a simple compile+run test of a simple C program.
Test plan¶
- Ensure the build tests in OBS are running
- Schedule our existing test runs
tests/console/glibc_sanity.pmltp_openposix(viatests/kernel/boot_ltp) (See e.g. https://openqa.suse.de/tests/10455037)
- Write a simple test which compiles a predefined C program, which uses a handful of library functions. We should use
gccorclang, so that it indirectly would also test the glibc on the system itself.
Updated by ph03nix about 1 year ago
- Status changed from Feedback to In Progress
Updated by ph03nix about 1 year ago
ulp_openposix might be already a comprehensive test suite: https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/tests/kernel/ulp_openposix.pm
ltp_openposix tests glibc and can be scheduled via tests/kernel/boot_ltp. See e.g. https://openqa.suse.de/tests/10455037.
Updated by ph03nix about 1 year ago
- Status changed from In Progress to Feedback
Updated by MDoucha about 1 year ago
ph03nix wrote:
ulp_openposixmight be already a comprehensive test suite: https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/tests/kernel/ulp_openposix.pmSee e.g. https://openqa.suse.de/tests/10455037.
ltp_openposix and ulp_openposix are two different things. ltp_openposix tests glibc. ulp_openposix tests libpulp0 and glibc-livepatches.
In addition, there's also ltp_syscalls and ltp_syscalls_ipc which test lots of kernel syscalls both directly and through glibc wrapper functions.
Updated by ph03nix about 1 year ago
MDoucha wrote:
ltp_openposix and ulp_openposix are two different things. ltp_openposix tests
glibc. ulp_openposix testslibpulp0andglibc-livepatches.In addition, there's also ltp_syscalls and ltp_syscalls_ipc which test lots of kernel syscalls both directly and through glibc wrapper functions.
Thank you, I updated the previous statements and recommended to use ltp_openposix for regression tests on glibc
Updated by szarate about 1 year ago
One question is, will we need an LTP image to run the ltp_openposix tests? I want to avoid testing the same thing twice.
Updated by MDoucha about 1 year ago
szarate wrote:
One question is, will we need an LTP image to run the
ltp_openposixtests? I want to avoid testing the same thing twice.
Yes, the openposix tests are bundled in the LTP package in QA:Head and the jobs boot from standard LTP disk image.
Updated by szarate about 1 year ago
- Sprint changed from QE-Core: February Sprint (Feb 08 - Mar 08) to QE-Core: March Sprint (Mar 08 - Apr 05)
Updated by szarate about 1 year ago
- Copied to action #125606: [qe-core] Test glibc on 11SP4 added