action #36754
closed[qe-core][functional][systemd][medium] test fails in systemd_testsuite - needs further investigation
Added by nicksinger over 6 years ago. Updated almost 4 years ago.
0%
Description
Observation¶
Could not access KVM kernel module: No such device
Reproducible¶
- aarch64: at the moment osd#1749227#step/systemd_testsuite/49 shows bsc#1097440
- s390x: osd#1749598#step/systemd_testsuite/43
Acceptance criteria¶
AC1 Test suite suse_patches-systemd_testsuite doesn't fail on missing KVM kernel module
Tasks¶
- Enable KVM virtualization on openQA-worker hosts.
- If necessary, install SLE15 for aarch64 openQA-workers hosts (tblume is not sure if we can do it on SLE12)
Expected result¶
Last good build 563.1:
Updated by SLindoMansilla over 6 years ago
- Related to coordination #34996: [qe-core][functional][opensuse][epic] test fails in systemd_testsuite - TEST-16-EXTEND-TIMEOUT works only when executed against systemd built in the same specfile added
Updated by okurz over 6 years ago
- Subject changed from [functional][systemd][u] test fails in systemd_testsuite on aarch64 - needs further investigation to [functional][systemd][u][fast] test fails in systemd_testsuite on aarch64 - needs further investigation
- Due date set to 2018-06-19
- Priority changed from Normal to Urgent
- Target version set to Milestone 17
happening in SLE15 GMC.
Updated by mgriessmeier over 6 years ago
- Subject changed from [functional][systemd][u][fast] test fails in systemd_testsuite on aarch64 - needs further investigation to [functional][systemd][u][fast][medium] test fails in systemd_testsuite - needs further investigation
- Description updated (diff)
- Status changed from New to Workable
Updated by SLindoMansilla over 6 years ago
- Status changed from Workable to In Progress
- Assignee set to SLindoMansilla
Updated by SLindoMansilla over 6 years ago
First SR to reorganize SUSE patches accepted: https://build.opensuse.org/request/show/615165
Updated by SLindoMansilla over 6 years ago
aarch64 issue is a bug and is handle in https://bugzilla.suse.com/show_bug.cgi?id=1097440
Updated by SLindoMansilla over 6 years ago
- Description updated (diff)
SR to patch upstream bug: https://build.opensuse.org/request/show/616259
SR to patch openSUSE bug: https://build.opensuse.org/request/show/616468
Investigating s390x issue.
Updated by SLindoMansilla over 6 years ago
- Status changed from In Progress to Blocked
- Priority changed from Urgent to High
At the moment SLE15 is already on GMC phase. Adjusting prio.
I cannot investigate further due to a malfunction of the s390x-susekvm worker: https://progress.opensuse.org/issues/33202
Updated by SLindoMansilla over 6 years ago
- Blocked by action #33202: [sle][functional][s390x][zkvm][u][hard] test fails in boot_to_desktop - still insufficient error reporting, black screen with mouse cursor - we all hate it (was: I hate it) added
Updated by SLindoMansilla over 6 years ago
- Status changed from Blocked to In Progress
I forgot to change status after s390x worker was self-healed
Updated by SLindoMansilla over 6 years ago
Follow up for "ID_LIKE" patch to be fully back ported into SLE15: https://github.com/openSUSE/systemd/blob/SLE15/test/test-functions
Updated by okurz over 6 years ago
- Target version changed from Milestone 17 to Milestone 17
Updated by SLindoMansilla over 6 years ago
- Description updated (diff)
- Priority changed from High to Normal
The problem is identified.
SLE15 is GM, so lowering priority.
Updated by SLindoMansilla over 6 years ago
- Description updated (diff)
- Assignee changed from SLindoMansilla to tsaupe
Updated by SLindoMansilla over 6 years ago
- Blocked by action #25248: [tools][functional][systemd][u] openQA-workers for systemd department added
Updated by mgriessmeier over 6 years ago
- Due date changed from 2018-06-19 to 2018-07-03
Updated by okurz over 6 years ago
- Subject changed from [functional][systemd][u][fast][medium] test fails in systemd_testsuite - needs further investigation to [functional][systemd][u][medium] test fails in systemd_testsuite - needs further investigation
- Due date deleted (
2018-07-03) - Target version changed from Milestone 17 to Milestone 20
So I discussed in detail with slindomansilla related to #34996 as well. AFAIU this ticket is for SLE in particular. Currently we focus on SLE12SP4 and for SLE15 the tests had been failing for a long time. This is why I delay the ticket plan to M20 as we should get openSUSE Tumbleweed in a stable state first and then revisit the situation for SLE tests. Whenever we see tests failing consistently and for a long time we should move the test scenario to the according "test development" job groups on the openQA server(s). Let's focus into #34996 . @tsaupe of course the plan is only according to our planning as QSF (QA SLE functional) to look into this issue again later, milestone 20, which is 2018-11, but you are free to continue on this ticket as you like :)
Updated by okurz over 6 years ago
This is an autogenerated message for openQA integration by the openqa_review script:
This bug is still referenced in a failing openQA test: suse_patches-systemd_testsuite
https://openqa.suse.de/tests/1772093
Updated by okurz over 6 years ago
This is an autogenerated message for openQA integration by the openqa_review script:
This bug is still referenced in a failing openQA test: suse_patches-systemd_testsuite
https://openqa.suse.de/tests/1772093
Updated by okurz over 6 years ago
This is an autogenerated message for openQA integration by the openqa_review script:
This bug is still referenced in a failing openQA test: suse_patches-systemd_testsuite
https://openqa.suse.de/tests/1772093
Updated by okurz about 6 years ago
- Status changed from In Progress to Workable
- Assignee deleted (
tsaupe) - Priority changed from Normal to High
Seems like no real progress here. By now I think we should take a look into SLE15 tests, at least remove the red blob from the validation tests. https://bugzilla.suse.com/show_bug.cgi?id=1105101 is currently used as label in these tests but also there nothing is moving forward, multiple unattended openqa-review reminder comments.
Updated by nicksinger about 6 years ago
- Related to action #32563: [functional][u] fix salt for power added
Updated by SLindoMansilla about 6 years ago
- Status changed from Workable to Feedback
- Assignee set to SLindoMansilla
A SR provided by tsaupe was accepted yesterday: https://build.suse.de/request/show/175568
I will verify the fix on OSD.
Waiting for:
Updated by SLindoMansilla about 6 years ago
- Status changed from Feedback to In Progress
The job times out. I will try to reproduce how much time it needs to adapt the test.
Updated by SLindoMansilla about 6 years ago
- Status changed from In Progress to Feedback
PR to increase timeout: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/6200
Updated by SLindoMansilla about 6 years ago
Waiting for verification run on OSD:
Updated by SLindoMansilla about 6 years ago
- Status changed from Feedback to In Progress
Both fail. 1 hour timeout is not enough. Investigating...
Updated by SLindoMansilla about 6 years ago
tblume has prepared a SR to solve this issue that makes the test hang in the qemu VM: https://build.suse.de/request/show/177535
Updated by SLindoMansilla about 6 years ago
SR is accepted, waiting for the package to build on QA:SLE15: https://build.suse.de/package/show/QA:SLE15/systemd-v234-testsuite
Updated by SLindoMansilla about 6 years ago
- Status changed from In Progress to Feedback
Waiting for verification on OSD:
Updated by SLindoMansilla about 6 years ago
It is still hanging on second test. Waiting for feedback from tblume.
Updated by SLindoMansilla about 6 years ago
- Has duplicate action #43826: [qam] [SLE15] test fails in systemd_testsuite added
Updated by SLindoMansilla about 6 years ago
Last SR broke also x86_64 tests: https://openqa.suse.de/tests/2259623#step/systemd_testsuite/13
Reverting last SR
Updated by SLindoMansilla about 6 years ago
tblume already provided a new SR with the fix: https://build.suse.de/request/show/177653
Not needed to revert changes.
Updated by SLindoMansilla about 6 years ago
Waiting for verification on OSD:
- aarch64: https://openqa.suse.de/tests/2261449 (fail: it still hangs in TEST-02)
- ppc64le: https://openqa.suse.de/tests/2261448 (pass)
- s390x: https://openqa.suse.de/tests/2261447 (scheduled)
- x86_64: https://openqa.suse.de/tests/2261446 (pass)
Updated by SLindoMansilla about 6 years ago
- Status changed from Feedback to In Progress
Results for build 96.7:
- aarch64: https://openqa.suse.de/tests/2261889#step/systemd_testsuite/16 (fails, it still hangs on TEST-02)
- ppc64le: https://openqa.suse.de/tests/2261978#step/systemd_testsuite/53 (it fails now in TEST-10)
- s390x: https://openqa.suse.de/tests/2263572
- x86_64: https://openqa.suse.de/tests/2262698 (passes)
Investigating fail on TEST-10:
- Try to reproduce in the previous build (95.1): https://openqa.suse.de/tests/2263603
- Try to reproduce in the current build (96.7): https://openqa.suse.de/tests/2263604
Updated by SLindoMansilla about 6 years ago
Fail reproduced:
Updated by SLindoMansilla about 6 years ago
- Status changed from In Progress to Feedback
New SR from tblume to fix aarch64 problem: https://build.suse.de/request/show/177815
Waiting for verification: https://openqa.suse.de/tests/2263888
Updated by SLindoMansilla about 6 years ago
TEST-02 passes, but it fails on other tests later.
Verifying that the SR is not breaking s390x and x86_64 scenarios:
- s3390x: https://openqa.suse.de/tests/2263926
- x86_64: https://openqa.suse.de/tests/2263913 (passes)
Updated by okurz about 6 years ago
- Related to action #44150: [functional][u] test fails in systemd_testsuite, Test died: Could not retrieve required variable QA_HEAD_REPO at /var/lib/openqa/cache/openqa.suse.de/tests/sle/tests/console/systemd_testsuite.pm line 21. added
Updated by okurz about 6 years ago
- Priority changed from High to Urgent
- Target version changed from Milestone 20 to Milestone 21
M20 is over, as we already shifted a lot I am not sure how this will go on. As we still have jobs failing labeled with this ticket, e.g. https://openqa.suse.de/tests/2271738#step/systemd_testsuite/17, I see it as urgent now. The urgency could be handled by removing the scenario again. But I leave it to you to decide which way to go, just … urgently please :)
Updated by SLindoMansilla about 6 years ago
- Status changed from Feedback to In Progress
At the moment there is no s390x worker to verify.
OSD ones are getting freeze.
QSF shared workers are not working because the s390x zkvm backend was never adapted to work on workers with enabled cache.
Fixing s390x backend first and moving aarch64 and ppc64le scenarios to development job group: https://openqa.suse.de/admin/job_templates/96
- A fix was applied for XEN: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/6191
- That needs to be applied for s390x: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/6253
Updated by SLindoMansilla about 6 years ago
- Priority changed from Urgent to High
Urgency removed after moving aarch64 and ppc64le scenarios to development job group.
@okurz, feel free to set normal prio.
Updated by SLindoMansilla about 6 years ago
- Status changed from In Progress to Blocked
Updated by SLindoMansilla about 6 years ago
- Status changed from Blocked to In Progress
Worker is working again: http://slindomansilla-vm.qa.suse.de/tests/959
Updated by SLindoMansilla about 6 years ago
Generating the missing qcow2 image: http://slindomansilla-vm.qa.suse.de/tests/962
Updated by SLindoMansilla about 6 years ago
- Related to action #44468: [qe-core] Proper handling of assets for svirt workers added
Updated by SLindoMansilla about 6 years ago
- Related to deleted (action #44468: [qe-core] Proper handling of assets for svirt workers)
Updated by SLindoMansilla about 6 years ago
- Blocked by action #44468: [qe-core] Proper handling of assets for svirt workers added
Updated by SLindoMansilla about 6 years ago
- Blocked by deleted (action #25248: [tools][functional][systemd][u] openQA-workers for systemd department)
Updated by SLindoMansilla about 6 years ago
- Related to action #25248: [tools][functional][systemd][u] openQA-workers for systemd department added
Updated by SLindoMansilla about 6 years ago
- Status changed from In Progress to Feedback
s390x verification failed: https://openqa.suse.de/tests/2288353
Waiting for tsaupe's feedback on https://build.suse.de/request/show/177956
Updated by SLindoMansilla about 6 years ago
- Status changed from Feedback to Workable
Not able to verify on s390x until there is an available qcow2 image of SLE15-SP1 textmode for s390x-kvm-sle12
Updated by SLindoMansilla about 6 years ago
- Related to action #45158: [systemd] Implement systemd testsuite as openQA perl module added
Updated by SLindoMansilla about 6 years ago
- Status changed from Workable to Blocked
Blocked by #45158
Updated by SLindoMansilla about 6 years ago
- Related to deleted (action #45158: [systemd] Implement systemd testsuite as openQA perl module)
Updated by SLindoMansilla about 6 years ago
- Blocked by action #45158: [systemd] Implement systemd testsuite as openQA perl module added
Updated by okurz almost 6 years ago
This is an autogenerated message for openQA integration by the openqa_review script:
This bug is still referenced in a failing openQA test: suse_patches-systemd_testsuite
https://openqa.suse.de/tests/2343909
Updated by okurz almost 6 years ago
- Target version changed from Milestone 21 to future
due to blocker
Updated by okurz almost 6 years ago
This is an autogenerated message for openQA integration by the openqa_review script:
This bug is still referenced in a failing openQA test: suse_patches-systemd_testsuite
https://openqa.suse.de/tests/2429581
Updated by okurz almost 6 years ago
This is an autogenerated message for openQA integration by the openqa_review script:
This bug is still referenced in a failing openQA test: suse_patches-systemd_testsuite
https://openqa.suse.de/tests/2452703
Updated by SLindoMansilla almost 6 years ago
The test suite was moved to the development job group due to the bug in existing tests that are taking long time to be fixed: https://openqa.suse.de/admin/job_templates/96
Updated by SLindoMansilla over 5 years ago
PR https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/6253
still blocked by #44468
Updated by SLindoMansilla over 5 years ago
- Blocked by deleted (action #45158: [systemd] Implement systemd testsuite as openQA perl module)
Updated by SLindoMansilla over 5 years ago
- Related to action #45158: [systemd] Implement systemd testsuite as openQA perl module added
Updated by tjyrinki_suse about 4 years ago
- Subject changed from [functional][systemd][u][medium] test fails in systemd_testsuite - needs further investigation to [qe-core][functional][systemd][medium] test fails in systemd_testsuite - needs further investigation
Updated by SLindoMansilla almost 4 years ago
- Blocked by deleted (action #44468: [qe-core] Proper handling of assets for svirt workers)
Updated by SLindoMansilla almost 4 years ago
- Related to action #44468: [qe-core] Proper handling of assets for svirt workers added
Updated by SLindoMansilla almost 4 years ago
- Status changed from Blocked to Resolved
Issue not reproducible anymore