action #81811
closed
[sle][security][sle15sp3] test fails in manually_add_profile about type_string missing and Needles mismatch
Added by bchou over 3 years ago.
Updated about 3 years ago.
Category:
Bugs in existing tests
Description
Observation¶
openQA test in scenario sle-15-SP3-Online-aarch64-yast2_apparmor@aarch64 fails in
manually_add_profile
Test suite description¶
Maintainer: llzhao@suse.com
Test "yast2 apparmor" with an existing disk image.
Reproducible¶
Fails since (at least) Build 120.1 (current job)
Expected result¶
Last good: 118.3 (or more recent)
Further details¶
Always latest result in this scenario: latest
- Status changed from New to Feedback
- % Done changed from 0 to 50
It might be a performance issue.
Try "VNC_TYPING_LIMIT=9" and let's see.
- Status changed from Feedback to Workable
- % Done changed from 50 to 90
- Estimated time changed from 4.00 h to 8.00 h
From "https://openqa.suse.de/tests/5341465/file/autoinst-log.txt" we can see the time cunsuming for each match.
the first needle match it takes about 300-198 = 102s.
[0m[37m[2021-01-26T07:10:39.818 CET] [debug] no change: 198.8s
[0m[32m[2021-01-26T07:10:40.928 CET] [debug] >>> testapi::_handle_found_needle: found settings_disable_enable_apparmor-AppArmor-Configuration-Settings-20200514, similarity 1.00 @ 130/247
the second: 90-87=3
[0m[37m[2021-01-26T07:10:54.931 CET] [debug] no change: 87.9s
[0m[32m[2021-01-26T07:10:56.042 CET] [debug] >>> testapi::_handle_found_needle: found settings_disable_enable_apparmor-AppArmor-Configuration-Settings-20200514, similarity 1.00 @ 130/247
the third: 30-25=5s
[0m[37m[2021-01-26T07:15:34.727 CET] [debug] no change: 26.9s
[0m[37m[2021-01-26T07:15:35.769 CET] [debug] no match: 25.9s, best candidate: scan_audit_logs-AppArmor-Scan-Audit-logs-20200628 (0.00)
the fouth 30-27=3s
[0m[37m[2021-01-26T07:22:49.785 CET] [debug] no match: 27.9s, best candidate: manually_add_profile-AppArmor-Chose-a-program-to-generate-a-profile-20200827 (0.68)
- Status changed from Workable to Feedback
- % Done changed from 90 to 100
PR merged, let's check next run.
Please understand that VNC_TYPING_LIMIT can be used for debugging and workaround low-performance workers however in production we should aim for tests that can cope with arbitrary slow-downs during execution. So regardless of your results in most cases the test code needs adaptions. And I think you did just the right thing with https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/11842, appreciated :)
okurz wrote:
Please understand that VNC_TYPING_LIMIT can be used for debugging and workaround low-performance workers however in production we should aim for tests that can cope with arbitrary slow-downs during execution. So regardless of your results in most cases the test code needs adaptions. And I think you did just the right thing with https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/11842, appreciated :)
Got it, thank you Oliver.
- Status changed from Feedback to Resolved
This is an autogenerated message for openQA integration by the openqa_review script:
This bug is still referenced in a failing openQA test: yast2_apparmor
https://openqa.suse.de/tests/5460986
To prevent further reminder comments one of the following options should be followed:
- The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
- The openQA job group is moved to "Released"
- The label in the openQA scenario is removed
Also available in: Atom
PDF