Project

General

Profile

action #60935

[SLE][Migration][SLE15SP2][Regression]test fails in salt - salt-run timeout

Added by tinawang123 6 months ago. Updated 2 months ago.

Status:
Resolved
Priority:
Urgent
Assignee:
Category:
Bugs in existing tests
Target version:
-
Start date:
2019-12-12
Due date:
% Done:

0%

Estimated time:
12.00 h
Difficulty:
Duration:

Description

Observation

openQA test in scenario sle-15-SP2-Regression-on-Migration-from-SLE15-SPX-to-SLE15-SP2-ppc64le-online_sles15sp1_pscc_lp-basesys-srv-desk-dev-contm-lgm-py2-tsm-wsm_all_full@ppc64le fails in
salt

Test suite description

Reproducible

Fails since (at least) Build 104.1 (current job)

Further details

Always latest result in this scenario: latest


Related issues

Related to openQA Tests - action #60662: [SLE][Migration][SLE15SP2][Regression] Some regression tests fails on ppc64le - /dev/hvc0: permission deniedResolved2019-12-05

History

#1 Updated by coolgw 5 months ago

  • Assignee set to coolgw

#3 Updated by coolgw 5 months ago

should be an product issue, 5mins no event show?
[2019-12-10T20:38:48.201 CET] [debug] /var/lib/openqa/cache/openqa.suse.de/tests/sle/tests/console/salt.pm:57 called testapi::assert_script_run
[2019-12-10T20:38:48.201 CET] [debug] <<< testapi::assert_script_run(cmd='salt-run state.event tagmatch="salt/auth" quiet=True count=1', timeout=300, fail_message='', quiet=undef)
[2019-12-10T20:38:48.201 CET] [debug] /var/lib/openqa/cache/openqa.suse.de/tests/sle/tests/console/salt.pm:57 called testapi::assert_script_run
[2019-12-10T20:38:48.201 CET] [debug] <<< testapi::type_string(string='salt-run state.event tagmatch="salt/auth" quiet=True count=1', max_interval=250, wait_screen_changes=0, wait_still_screen=0, timeout=30, similarity_level=47)
[2019-12-10T20:38:54.389 CET] [debug] /var/lib/openqa/cache/openqa.suse.de/tests/sle/tests/console/salt.pm:57 called testapi::assert_script_run
[2019-12-10T20:38:54.390 CET] [debug] <<< testapi::type_string(string='; echo NOnZ1-$?- > /dev/hvc0
', max_interval=250, wait_screen_changes=0, wait_still_screen=0, timeout=30, similarity_level=47)
[2019-12-10T20:38:57.602 CET] [debug] /var/lib/openqa/cache/openqa.suse.de/tests/sle/tests/console/salt.pm:57 called testapi::assert_script_run
[2019-12-10T20:38:57.602 CET] [debug] <<< testapi::wait_serial(expect_not_found=0, record_output=undef, buffer_size=undef, no_regex=0, timeout=300, quiet=undef, regexp=qr/NOnZ1-\d+-/)
[2019-12-10T20:43:59.022 CET] [debug] >>> testapi::wait_serial: (?:NOnZ1-\d+-): fail

#4 Updated by coolgw 5 months ago

linux-4360:/tmp/test # /usr/share/openqa/script/clone_job.pl --from http://openqa.suse.de --host http://openqa.suse.de 3688179 --skip-download GROUP=0 EXCLUDE_MODULES=install_service,supportutils,zypper_lifecycle,force_scheduled_tasks,yast2_lan,keymap_or_locale,check_upgraded_service,orphaned_packages_check,

Created job #3720696: sle-15-SP2-Regression-on-Migration-from-SLE15-SPX-to-SLE15-SP2-ppc64le-Build104.1-online_sles15sp1_pscc_lp-basesys-srv-desk-dev-contm-lgm-py2-tsm-wsm_all_full@ppc64le -> http://openqa.suse.de/t3720696

#5 Updated by coolgw 5 months ago

coolgw wrote:

linux-4360:/tmp/test # /usr/share/openqa/script/clone_job.pl --from http://openqa.suse.de --host http://openqa.suse.de 3688179 --skip-download GROUP=0 EXCLUDE_MODULES=install_service,supportutils,zypper_lifecycle,force_scheduled_tasks,yast2_lan,keymap_or_locale,check_upgraded_service,orphaned_packages_check,

Created job #3720696: sle-15-SP2-Regression-on-Migration-from-SLE15-SPX-to-SLE15-SP2-ppc64le-Build104.1-online_sles15sp1_pscc_lp-basesys-srv-desk-dev-contm-lgm-py2-tsm-wsm_all_full@ppc64le -> http://openqa.suse.de/t3720696

salt issue self fixed by just exclude some modules?

#6 Updated by coolgw 5 months ago

i try to run with install_service/check_upgrade_service again
linux-4360:/tmp/Linux-0.12 # /usr/share/openqa/script/clone_job.pl --from http://openqa.suse.de --host http://openqa.suse.de 3688179 --skip-download GROUP=0 EXCLUDE_MODULES=supportutils,zypper_lifecycle,force_scheduled_tasks,yast2_lan,keymap_or_locale,orphaned_packages_check,
Created job #3724728: sle-15-SP2-Regression-on-Migration-from-SLE15-SPX-to-SLE15-SP2-ppc64le-Build104.1-online_sles15sp1_pscc_lp-basesys-srv-desk-dev-contm-lgm-py2-tsm-wsm_all_full@ppc64le -> http://openqa.suse.de/t3724728

#7 Updated by coolgw 5 months ago

above run failed
linux-4360:~/openqa-distr # /usr/share/openqa/script/clone_job.pl --from http://openqa.suse.de --host http://openqa.suse.de 3688179 --skip-download GROUP=0 EXCLUDE_MODULES=supportutils,zypper_lifecycle,force_scheduled_tasks,yast2_lan,keymap_or_locale,orphaned_packages_check,
Created job #3724901: sle-15-SP2-Regression-on-Migration-from-SLE15-SPX-to-SLE15-SP2-ppc64le-Build104.1-online_sles15sp1_pscc_lp-basesys-srv-desk-dev-contm-lgm-py2-tsm-wsm_all_full@ppc64le -> http://openqa.suse.de/t3724901

#8 Updated by leli 5 months ago

  • Estimated time set to 12.00 h

#9 Updated by coolgw 5 months ago

try run since above run failed on prepare_test_data
linux-4360:~/openqa-distr # /usr/share/openqa/script/clone_job.pl --from http://openqa.suse.de --host http://openqa.suse.de 3688179 --skip-download GROUP=0 EXCLUDE_MODULES=supportutils,zypper_lifecycle,force_scheduled_tasks,yast2_lan,keymap_or_locale,orphaned_packages_check,prepare_test_data

#10 Updated by coolgw 5 months ago

in order narrow down the issue, rerun test case only keep service check related module before salt
linux-4360:~ # /usr/share/openqa/script/clone_job.pl --from http://openqa.suse.de --host http://openqa.suse.de 3721117 --skip-download GROUP=0 EXCLUDE_MODULES=supportutils,force_scheduled_tasks,textinfo,hostname,upgrade_snapshots,x_vt,zypper_lr,check_system_info,zypper_ref,ncurses,yast2_lan,curl_https
Created job #3738601: sle-15-SP2-Regression-on-Migration-from-SLE15-SPX-to-SLE15-SP2-ppc64le-Build108.1-online_sles15sp1_pscc_lp-basesys-srv-desk-dev-contm-lgm-py2-tsm-wsm_all_full@ppc64le -> http://openqa.suse.de/t3738601

#11 Updated by coolgw 5 months ago

  • Status changed from New to Blocked

SCC register failed base above result

#12 Updated by coolgw 5 months ago

rerun it:
linux-4360:~ # /usr/share/openqa/script/clone_job.pl --from http://openqa.suse.de --host http://openqa.suse.de 3721117 --skip-download GROUP=0 EXCLUDE_MODULES=supportutils,force_scheduled_tasks,textinfo,hostname,upgrade_snapshots,x_vt,zypper_lr,check_system_info,zypper_ref,ncurses,yast2_lan,curl_https
Created job #3762554: sle-15-SP2-Regression-on-Migration-from-SLE15-SPX-to-SLE15-SP2-ppc64le-Build108.1-online_sles15sp1_pscc_lp-basesys-srv-desk-dev-contm-lgm-py2-tsm-wsm_all_full@ppc64le -> http://openqa.suse.de/t3762554
linux-4360:~ #

#13 Updated by coolgw 4 months ago

#15 Updated by leli 3 months ago

  • Priority changed from Normal to Urgent

#17 Updated by coolgw 2 months ago

  • Status changed from Blocked to In Progress

i suspect also related with hvc0 permission deny issue

#18 Updated by coolgw 2 months ago

  • Related to action #60662: [SLE][Migration][SLE15SP2][Regression] Some regression tests fails on ppc64le - /dev/hvc0: permission denied added

#19 Updated by coolgw 2 months ago

Base following result, seems salt is start work, and permission deny issue still happen, so seems not related with 60662.

https://openqa.suse.de/tests/3996096#

#20 Updated by coolgw 2 months ago

  • Status changed from In Progress to Resolved

Trigger new bug for this after check the detail log.
Bug 1167566 - [Build 163.1] openQA test fails in salt - crash happen after start salt(only found in ppc64) (edit)

Also available in: Atom PDF