action #96296: [qem] test fails in disk_activation - openQA Tests - openSUSE Project Management Tool

Custom queries

All open Feature tests
openQA Infrastructure Project
openqa-review - Closed tickets last updated by openqa-review, last 30 days
QA roadmap long-term
QA SLE functional
QA SLE Functional - closed in last 14 days
QA SLE Functional - High, need to be refined
QA SLE Functional - over cycle time median
QA SLE u
QA SLE y
QA tools (tag not necessary in openQA and subprojects)
QA tools tag (tag not necessary in openQA and subprojects; excluding tickets in "Ready" version as they are already on the backlog)
QAC - Backlog
QAM
QE tools team - backlog (dev)
QE tools team - backlog (ready issues)
QE tools team - backlog SLA high
QE tools team - backlog SLA immediate
QE tools team - backlog SLA no immediate/urgent in feedback/blocked
QE tools team - backlog SLA normal
QE tools team - backlog SLA urgent
QE tools team - backlog SLO high
QE tools team - backlog SLO normal
QE tools team - backlog SLO urgent
QE tools team - backlog, high-level view (epics and higher)
QE tools team - backlog, non-reactive work, needs parent
QE tools team - backlog, top-level view (all sagas)
QE tools team - closed within last 14 days
QE tools team - closed within last 60 days
QE tools team - closed yesterday
QE Tools Team - Collaborative Session
QE tools team - due date forecast
QE tools team - exceeding due-date
QE tools team - infrastructure backlog
QE tools team - next - sorted by update time
QE tools team - next issues
QE tools team - non-estimated (unblocked) issues (dev)
QE tools team - non-estimated (unblocked) issues (infra)
QE tools team - ready issues - Workable
QE tools team - ready, not assigned/blocked/low
QE tools team - SLO high forecast
QE tools team - update forecast
QE tools team - updated by priority
QE tools team - what members of the team are working on - Feedback (not-low)
QE Tools Team Backlog By Assignee
SLE15 Migration Open Tickets
SLE15 SP1 Migration Open Tickets
SLE15SP3 Migration open ticket
SLE15SP3 Security open ticket
Tools Team Retrospective
Tools Team Retrospective (not estimated or assigned)

Actions

Copy link

action #96296

closed

coordination #96302: [qe-core][QU] Quarterly update failures

[qem] test fails in disk_activation

Added by hurhaj over 3 years ago. Updated about 3 years ago.

Status:

Resolved

Priority:

Normal

Assignee:

geor

Category:

Bugs in existing tests

Target version:

QA - QE-Core: Ready

Start date:

2021-07-29

Due date:

% Done:

100%

Estimated time:

Difficulty:

Description

Observation¶

no devices are in the list

openQA test in scenario sle-15-SP3-Online-QR-s390x-zfcp@s390x-zfcp fails in
disk_activation

Test suite description¶

Maintainer: QE Yast, mgriessmeier

Installation-only test configuring an s390x ZFCP storage.

Reproducible¶

Fails since (at least) Build 188.11

Expected result¶

Last good: (unknown) (or more recent)

Further details¶

Always latest result in this scenario: latest

Related issues 1 (0 open — 1 closed)

Related to openQA Tests - action #96680: [qe-core] s390qa102.qa.suse.de zfcp issue

Resolved

geor

2021-08-09

Actions

Issue # Delay: days Cancel

History
Notes
Property changes

Actions

Copy link

Updated by szarate over 3 years ago

Parent task set to #96302

Setting to parent task to have a common tracker

Actions

Copy link

Updated by oorlov over 3 years ago

This failure does not relate to any test code/schedule files. No changes were introduced.

Looks like a product bug or infrastructure issue. Matthias suggested to set up the machine manually and check the issue.

I'm not an expert in s390x setup, so this ticket is better to address to whom it may concern.

Actions

Copy link

Updated by geor over 3 years ago

Assignee set to geor

Actions

Copy link

Updated by geor over 3 years ago

Status changed from New to In Progress
% Done changed from 0 to 20

Actions

Copy link

Updated by geor over 3 years ago

% Done changed from 20 to 90

It seems like s390qa101.qa.suse.de has no issues with zfcp, whereas s390qa102.qa.suse.de does.
Whenever the job ends up running in the second worker instance (s390qa102.qa.suse.de) disk_activation module will fail.

Bringing the FCP device online with chccwdev -e 0.0.fa00 on s390qa101 results in the respective LUNs getting attached:

# lsluns
Scanning for LUNs on adapter 0.0.fa00
        at port 0x500507630703d3b3:
                0x4001405000000000
        at port 0x500507630708d3b3:
                0x4001405000000000

Bringing the FCP device online with chccwdev -e 0.0.fa00 on s390qa102 results in the following:

# lsluns
lsluns: No port on any adapter exists

# lszfcp -PHD
0.0.fa00 host0
Error: No fcp devices found.

# cat /proc/scsi/scsi  (shows no SCSI attached devices)
Attached devices:
#

Also in s390qa101 the output of lszdev does show the two SCSI devices (with both names)

zfcp-lun     0.0.fa00:0x500507630703d3b3:0x4001405000000000  yes  no    sdb sg1
zfcp-lun     0.0.fa00:0x500507630708d3b3:0x4001405000000000  yes  no    sda sg0

whereas, in s390qa102 it does not

After consulting with Matthias on this, it turns out that there could be a probable mapping discrepancy, where the 0.0.fa00 points to different devices between the two s390 instances, resulting in this inconsistency. Further investigation will need to be done in order to pinpoint the exact root of the problem. For now a workaround will be to only schedule the zfcp testsuites on worker instance 1.

Actions

Copy link