Project

General

Profile

Actions
Copy link

action #81811

closed

[sle][security][sle15sp3] test fails in manually_add_profile about type_string missing and Needles mismatch

Added by bchou over 3 years ago. Updated about 3 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
llzhao
Category:
Bugs in existing tests
Target version:
-
Start date:
2021-01-06
Due date:
% Done:

100%

Estimated time:
8.00 h
Difficulty:

Description

Observation

openQA test in scenario sle-15-SP3-Online-aarch64-yast2_apparmor@aarch64 fails in
manually_add_profile

Test suite description

Maintainer: llzhao@suse.com
Test "yast2 apparmor" with an existing disk image.

Reproducible

Fails since (at least) Build 120.1 (current job)

Expected result

Last good: 118.3 (or more recent)

Further details

Always latest result in this scenario: latest

Actions
Copy link
 #1

Updated by llzhao about 3 years ago

  • Status changed from New to Feedback
  • % Done changed from 0 to 50

It might be a performance issue.
Try "VNC_TYPING_LIMIT=9" and let's see.

Actions
Copy link
 #3

Updated by llzhao about 3 years ago

  • Status changed from Feedback to Workable
Actions
Copy link
 #4

Updated by llzhao about 3 years ago

  • % Done changed from 50 to 90
  • Estimated time changed from 4.00 h to 8.00 h
Actions
Copy link
 #5

Updated by llzhao about 3 years ago

From "https://openqa.suse.de/tests/5341465/file/autoinst-log.txt" we can see the time cunsuming for each match.

  1. the first needle match it takes about 300-198 = 102s.
    [2021-01-26T07:10:39.818 CET] [debug] no change: 198.8s
    [2021-01-26T07:10:40.928 CET] [debug] >>> testapi::_handle_found_needle: found settings_disable_enable_apparmor-AppArmor-Configuration-Settings-20200514, similarity 1.00 @ 130/247

  2. the second: 90-87=3
    [2021-01-26T07:10:54.931 CET] [debug] no change: 87.9s
    [2021-01-26T07:10:56.042 CET] [debug] >>> testapi::_handle_found_needle: found settings_disable_enable_apparmor-AppArmor-Configuration-Settings-20200514, similarity 1.00 @ 130/247

  3. the third: 30-25=5s
    [2021-01-26T07:15:34.727 CET] [debug] no change: 26.9s
    [2021-01-26T07:15:35.769 CET] [debug] no match: 25.9s, best candidate: scan_audit_logs-AppArmor-Scan-Audit-logs-20200628 (0.00)

  4. the fouth 30-27=3s
    [2021-01-26T07:22:49.785 CET] [debug] no match: 27.9s, best candidate: manually_add_profile-AppArmor-Chose-a-program-to-generate-a-profile-20200827 (0.68)

Actions
Copy link
 #6

Updated by llzhao about 3 years ago

  • Status changed from Workable to Feedback
  • % Done changed from 90 to 100

PR merged, let's check next run.

Actions
Copy link
 #7

Updated by okurz about 3 years ago

Please understand that VNC_TYPING_LIMIT can be used for debugging and workaround low-performance workers however in production we should aim for tests that can cope with arbitrary slow-downs during execution. So regardless of your results in most cases the test code needs adaptions. And I think you did just the right thing with https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/11842, appreciated :)

Actions
Copy link
 #8

Updated by llzhao about 3 years ago

okurz wrote:

Please understand that VNC_TYPING_LIMIT can be used for debugging and workaround low-performance workers however in production we should aim for tests that can cope with arbitrary slow-downs during execution. So regardless of your results in most cases the test code needs adaptions. And I think you did just the right thing with https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/11842, appreciated :)

Got it, thank you Oliver.

Actions
Copy link
 #9

Updated by llzhao about 3 years ago

  • Status changed from Feedback to Resolved
Actions
Copy link
 #10

Updated by okurz about 3 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: yast2_apparmor
https://openqa.suse.de/tests/5460986

To prevent further reminder comments one of the following options should be followed:

  1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
  2. The openQA job group is moved to "Released"
  3. The label in the openQA scenario is removed
Actions
Copy link

Also available in: Atom PDF