Project

General

Profile

action #29940

[opensuse][functional][hard]test fails in gnuhealth_install -> are ttyS0 permissions reset by systemd automatically spawned getty?

Added by okurz almost 4 years ago. Updated almost 4 years ago.

Status:
Resolved
Priority:
Urgent
Assignee:
Category:
Bugs in existing tests
Start date:
2018-01-03
Due date:
2018-01-16
% Done:

0%

Estimated time:
Difficulty:

Description

Observation

openQA test in scenario opensuse-Tumbleweed-DVD-x86_64-gnuhealth@64bit fails in
gnuhealth_install

Reproducible

Fails since about two weeks now in most but not all cases.

Expected result

https://openqa.opensuse.org/tests/572779 is an example that worked but possibly only by chance

Problem

  • H1: systemd service getty@.service automatically spawns on ttyS0 as we specify the kernel command line parameter console=ttyS0 which is starting a getty on ttyS0 which is periodically resetting permissions on /dev/ttyS0 while the installation of packages might take a bit longer or shorter sometimes causing a race condition
  • H2: only reproduced on faster o3 infrastructure
  • H3: different o3 network structure
  • H4: different qemu or os-autoinst version
  • H5: REJECT: when booting the disk image different (no SKIPTO use) -> see #29940#note-3

Suggestions

  • Reproduce locally with statistics (e.g. 10 times) -> could not be reproduced (yet) see #29940#note-2 through #29940#note-4
  • Refactor all cases of 'chown.*serialdev'
  • Disable service if running with systemctl mask getty@$serialdev
  • Crosscheck other potential uses of getty on different architectures and backends, e.g. ipmi, s390x z/VM, etc.

Further details

Always latest result in this scenario: latest


Related issues

Related to openQA Tests - action #30033: [qam] test fails in libreoffice_mainmenu_components - ttyS0 permission deniedResolved2018-01-08

History

#1 Updated by okurz almost 4 years ago

  • Due date set to 2018-01-16
  • Priority changed from Normal to Urgent
  • Target version set to Milestone 13

#2 Updated by okurz almost 4 years ago

Triggered test to repeatedly test the faulty test module "gnuhealth-gnuhealth_install" with

for i in {1..20}; do sudo -u _openqa-worker perl /local/openQA/script/clone_job.pl --skip-chained-deps --from https://openqa.opensuse.org 572771 WORKER_CLASS=qemu_x86_64_tw MAKETESTSNAPSHOTS=1 SKIPTO=gnuhealth-gnuhealth_install ; done

http://lord.arch/tests?match=gnuhealth shows the results. So far in 5/5 runs I could not reproduce the tty permission problem in gnuhealth test locally. Maybe 1) only reproduced on faster o3 infrastructure or 2) different o3 network structure or 3) different qemu or os-autoinst version or 4) when booting the disk image different (no SKIPTO use)

#3 Updated by okurz almost 4 years ago

In 20/20 runs problem not reproduced. Trying again with booting each time

for i in {1..20}; do sudo -u _openqa-worker perl /local/openQA/script/clone_job.pl --skip-chained-deps --from https://openqa.opensuse.org 572771 WORKER_CLASS=qemu_x86_64_tw TEST=poo29940_gnuhealth_tty_perm_$i ; done

-> http://lord.arch/tests/overview?distri=opensuse&version=Tumbleweed&build=20180101

#4 Updated by okurz almost 4 years ago

  • Description updated (diff)

20/20 passed in http://lord.arch/tests/overview?build=20180101&distri=opensuse&version=Tumbleweed so the issue is not reproducible locally for me.

Still, to try to clean up a bit: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/4156

#5 Updated by okurz almost 4 years ago

  • Subject changed from [opensuse][functional]test fails in gnuhealth_install -> are ttyS0 permissions reset by systemd automatically spawned getty? to [opensuse][functional][hard]test fails in gnuhealth_install -> are ttyS0 permissions reset by systemd automatically spawned getty?

merged.

https://openqa.opensuse.org/tests/575735# is the validation job but I would like to confirm that with more tests on production:

for i in {1..20}; do openqa_clone_job_o3 --skip-download --skip-chained-deps 575735 TEST=okurz_check_gnuhealth_tty_perms_poo29940_$i _GROUP="Development Tumbleweed" ; done

-> https://openqa.opensuse.org/tests/overview?version=Tumbleweed&groupid=38&build=20180104&distri=opensuse

#6 Updated by okurz almost 4 years ago

  • Status changed from In Progress to Resolved

20/20 passed on openSUSE TW

#7 Updated by pcervinka almost 4 years ago

  • Related to action #30033: [qam] test fails in libreoffice_mainmenu_components - ttyS0 permission denied added

Also available in: Atom PDF