action #18558
closed[sle][functional][u][hard][investigation] Snapper tests run too long
0%
Description
Snapper cleanup runs for 54 minutes.
Snapper create runs for 18 minutes.
Combined they take too long time and cause job timeouts after 2 hours.
Acceptance criteria¶
- AC1: snapper cleanup + create runs together on ppc64le take less time than ~50m
Reproducible¶
openQA test in scenario:
- sle-12-SP3-Server-DVD-ppc64le-extra_tests_in_textmode_ppc@ppc64le SP3 is GM. We are in SP4.
- https://openqa.suse.de/tests/876434/modules/snapper_cleanup/steps/390 The test module is no more scheduled for this test suite.
It failed for SP3 on Build 0304
But now the test module is only scheduled for test suite: osd#extra_tests_filesystem#latest
Further details¶
- Latest for SP3: osd#extra_tests_in_textmode_ppc#latest
- Latest job with scheduled snapper_cleanup: osd#941999 (2017-05-18)
Updated by dgutu about 7 years ago
I do consider to increase MAX_JOB_TIME. Right now I don't see any possible way to changes the test logic.
If this fails on the ppc64le and not on other platform than maybe its not the test case a problem.
Opinions are welcome :)
Updated by mkravec about 7 years ago
Snapper cleanup runs about 40 minutes on x86_64 - maybe using virtio_terminal it could be finished much faster.
Updated by dgutu about 7 years ago
mkravec wrote:
Snapper cleanup runs about 40 minutes on x86_64 - maybe using virtio_terminal it could be finished much faster.
Sorry, no idea what is virtio_terminal. Paste some more info please, thx.
Updated by okurz@suse.de about 7 years ago
I do consider to increase MAX_JOB_TIME. Right now I don't see any possible
way to changes the test logic. If this fails on the ppc64le and not on
other platform than maybe its not the test case a problem.Opinions are welcome :)
short-term: make tests work again -> increase timeout
mid-term: simplify tests as one module for such a long time is certainly not a
good idea
Updated by okurz almost 7 years ago
In the the meantime we split extra tests into "extra_tests_filesystem" and "extra_tests_in_textmode"
Updated by okurz almost 7 years ago
- Category changed from Bugs in existing tests to Enhancement to existing tests
- Status changed from New to In Progress
- Assignee deleted (
okurz)
next steps: simplify tests as one module for such a long time is certainly not a
good idea
Updated by okurz almost 7 years ago
- Subject changed from Snapper tests run too long to [sle][functional]Snapper tests run too long
Updated by SLindoMansilla almost 7 years ago
Two months ago the module snapper_cleanup was not included anymore on this test:
Updated by riafarov over 6 years ago
- Status changed from In Progress to Workable
Updated by okurz almost 6 years ago
- Subject changed from [sle][functional]Snapper tests run too long to [sle][functional][u] Snapper tests run too long
- Description updated (diff)
- Due date set to 2018-07-03
- Target version changed from Milestone 16 to Milestone 17
Updated by okurz almost 6 years ago
- Target version changed from Milestone 17 to Milestone 17
Updated by riafarov almost 6 years ago
- Subject changed from [sle][functional][u] Snapper tests run too long to [sle][functional][u][hard][investigation] Snapper tests run too long
Updated by SLindoMansilla almost 6 years ago
- Related to action #31351: [functional][u][medium] force_cron_run does not actually run any crons (occasionally) added
Updated by SLindoMansilla almost 6 years ago
- Description updated (diff)
- Status changed from Workable to In Progress
Updated by SLindoMansilla almost 6 years ago
- Status changed from In Progress to Resolved
The test module is only scheduled for test suites
- extra_tests_filesystem: osd#extra_tests_filesystem#latest+previous
- mau-filesystem: osd#mau-filesystem#latest+previous
- jeos-filesystem: osd#jeos-filesystem#latest+previous
The issue didn't happen since at least 7 months:
We have also improvements in the old "force_cron_run" (see poo#31351)
which is going to be renamed (see poo#38093)
And this kind of performance issues will be handle on system performance scenarios (see poo#38090)
Updated by okurz almost 6 years ago
- Status changed from Resolved to In Progress
So how long do the test modules take? What performance issue do you mean?
Updated by SLindoMansilla almost 6 years ago
With performance issue I refer each time some background tasks affect the current test.
- [2018-07-02T19:47:33.0300 CEST] [debug] ||| starting snapper_cleanup
- [2018-07-02T19:56:44.0716 CEST] [debug] ||| finished snapper_cleanup
~9m11s
- [2018-07-02T20:01:07.0820 CEST] [debug] ||| starting snapper_create
- [2018-07-02T20:04:12.0381 CEST] [debug] ||| finished snapper_create
~3m5s
Wait, I realized I am checking for x86_64, but AC mentions ppc64le and that scenario is not active. Activated scenario on Test Development: SLE_12_SP4 and tracking.
Updated by okurz almost 6 years ago
good. With https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/5222 snapper_cleanup might have changed its behaviour a lot so it might be that actually the goal is fulfilled already without further code changes but you are right to crosscheck this. Before closing this ticket please make sure we do not have "lost" snapper_cleanup on our relevant products. Could be that we have other tickets for that. I just want to make sure we don't forget something :)