action #114532: [qe-core][sles15sp4]test fails in tomcat with most failures at “WebSocketsTest" - openQA Tests - openSUSE Project Management Tool

Custom queries

All open Feature tests
openQA Infrastructure Project
openqa-review - Closed tickets last updated by openqa-review, last 30 days
QA roadmap long-term
QA SLE functional
QA SLE Functional - closed in last 14 days
QA SLE Functional - High, need to be refined
QA SLE Functional - over cycle time median
QA SLE u
QA SLE y
QA tools (tag not necessary in openQA and subprojects)
QA tools tag (tag not necessary in openQA and subprojects; excluding tickets in "Ready" version as they are already on the backlog)
QAC - Backlog
QAM
QE tools team - backlog (ready issues)
QE tools team - backlog (w/o infra)
QE tools team - backlog SLA high
QE tools team - backlog SLA immediate
QE tools team - backlog SLA no immediate/urgent in feedback/blocked
QE tools team - backlog SLA normal
QE tools team - backlog SLA urgent
QE tools team - backlog SLO high
QE tools team - backlog SLO normal
QE tools team - backlog SLO urgent
QE tools team - backlog, high-level view (epics and higher)
QE tools team - backlog, non-reactive work, needs parent
QE tools team - backlog, top-level view (all sagas)
QE tools team - closed within last 14 days
QE tools team - closed within last 60 days
QE tools team - closed yesterday
QE Tools Team - Collaborative Session
QE tools team - due date forecast
QE tools team - exceeding due-date
QE tools team - infrastructure backlog
QE tools team - next - sorted by update time
QE tools team - next issues
QE tools team - non-estimated (unblocked) issues (w/o infra)
QE tools team - ready issues - Workable
QE tools team - ready, not assigned/blocked/low
QE tools team - update forecast
QE tools team - updated by priority
QE tools team - what members of the team are working on - Feedback (not-low)
QE Tools Team Backlog By Assignee
SLE15 Migration Open Tickets
SLE15 SP1 Migration Open Tickets
SLE15SP3 Migration open ticket
SLE15SP3 Security open ticket
Tools Team Retrospective
Tools Team Retrospective (not estimated or assigned)

Actions

Copy link

action #114532

closed

[qe-core][sles15sp4]test fails in tomcat with most failures at “WebSocketsTest"

Added by rfan1 almost 2 years ago. Updated over 1 year ago.

Status:

Resolved

Priority:

Normal

Assignee:

rfan1

Category:

Bugs in existing tests

Target version:

QA - QE-Core: Ready

Start date:

2022-07-22

Due date:

% Done:

100%

Estimated time:

4.00 h

Difficulty:

Sprint:

QE-Core: July Sprint (Jul 06 - Aug 03)

Tags:

bugbusters, qe-core-coverage

Description

Observation¶

openQA test in scenario sle-15-SP4-Server-DVD-Updates-x86_64-qam-regression-tomcat@64bit fails in
tomcat

Test suite description¶

Testsuite maintained at https://gitlab.suse.de/qa-maintenance/qam-openqa-yml.

Reproducible¶

Fails since (at least) Build 20220721-1

Expected result¶

Last good: 20220720-1 (or more recent)

Further details¶

Always latest result in this scenario: latest

History
Notes
Property changes

Actions

Copy link

Updated by rfan1 almost 2 years ago

This ticket failed multiple times in the past several builds.
And seems most of the failure occurred with "WebSocketsTest" function.

I am wondering there are some performance problems there, but can't understand why sp4 hits the issue more often.

Actions

Copy link

Updated by rfan1 almost 2 years ago

Subject changed from [qe-core][sles15sp4]test fails in tomcat with test at “WebSocketsTest" to [qe-core][sles15sp4]test fails in tomcat with most failures at “WebSocketsTest"

Actions

Copy link

Updated by rfan1 almost 2 years ago

Re-run with 4GB memory
http://openqa.suse.de/t9197005

Actions

Copy link

Updated by rfan1 almost 2 years ago

Tags set to bugbusters
Status changed from New to In Progress
Assignee set to rfan1
Target version set to QE-Core: Ready
Estimated time set to 4.00 h

More test runs with 4g mem:

#for i in 1 2 3 4 5; do openqa-clone-job  --from http://openqa.suse.de --host http://openqa.suse.de 9207213 _GROUP_ID=0 -skip-download --skip-chained-deps QEMURAM=4096 BUILD=tomcat;sleep 1;done
Created job #9207333: sle-15-SP4-Server-DVD-Updates-x86_64-Build20220723-1-qam-regression-tomcat@64bit -> http://openqa.suse.de/t9207333
Created job #9207334: sle-15-SP4-Server-DVD-Updates-x86_64-Build20220723-1-qam-regression-tomcat@64bit -> http://openqa.suse.de/t9207334
Created job #9207335: sle-15-SP4-Server-DVD-Updates-x86_64-Build20220723-1-qam-regression-tomcat@64bit -> http://openqa.suse.de/t9207335
Created job #9207336: sle-15-SP4-Server-DVD-Updates-x86_64-Build20220723-1-qam-regression-tomcat@64bit -> http://openqa.suse.de/t9207336
Created job #9207337: sle-15-SP4-Server-DVD-Updates-x86_64-Build20220723-1-qam-regression-tomcat@64bit -> http://openqa.suse.de/t9207337

Actions

Copy link

Updated by rfan1 almost 2 years ago

% Done changed from 0 to 20

Increasing mem size and adjusting some timeout value can help much!
now the failure ratio reduces to ~10%

Actions

Copy link

Updated by geor almost 2 years ago

In my opinion, the multiplayer drawboard example is prone to needle mismatch, because it requires the use of sending page downs and page ups, and needle matching will depend on screen size parameters among other factors.
I believe, in the sake of stability, that we can safely delete the function that tests the drawboard example.
We already cover the rest of Tomcat's WebSocket examples, and, in reality, this drawboard example is not really tested with multiple connections as it would ideally be the case.
So as far as coverage is concerned we should be ok.

Actions

Copy link

Updated by rfan1 almost 2 years ago

geor wrote:

In my opinion, the multiplayer drawboard example is prone to needle mismatch, because it requires the use of sending page downs and page ups, and needle matching will depend on screen size parameters among other factors.
I believe, in the sake of stability, that we can safely delete the function that tests the drawboard example.
We already cover the rest of Tomcat's WebSocket examples, and, in reality, this drawboard example is not really tested with multiple connections as it would ideally be the case.
So as far as coverage is concerned we should be ok.

Thanks much!
That can explain my I can't fix the issue by increasing the mem size and timeout value. It is not a performance issue.
Let me try to delete this function and see.

Actions

Copy link

Updated by rfan1 almost 2 years ago

#openqa-clone-job  --from http://openqa.suse.de --host http://openqa.suse.de 9207213 _GROUP_ID=0 -skip-download --skip-chained-deps BUILD=tomcat_drawboard_4g _timeout QEMURAM=4096 CASEDIR=https://github.com/rfan1/os-autoinst-distri-opensuse.git#tomcat_multiplayer_drawboard

http://openqa.suse.de/tests/9208390#next_previous
Good news: In past 15 Runs, only failed once and seems caused by page refreshing issue.

Actions

Copy link