Project

General

Profile

action #32926

[sle][functional][y][hyperv][medium] avoid typing username before switched tty (was: test fails in yast2_i - (mising needles?, rather too low timeout for hyperv) for Installation Report succesful)

Added by JERiveraMoya about 2 years ago. Updated about 2 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Bugs in existing tests
Start date:
2018-03-08
Due date:
2018-05-22
% Done:

0%

Estimated time:
Difficulty:
Duration: 54

Description

Observation

openQA test in scenario sle-15-Installer-DVD-x86_64-minimal+base@svirt-hyperv fails in
yast2_i

Missing needles for Installation Report as required for other architectures (see last good) or the timeout is not enough.

Reproducible

Fails since (at least) Build 486.4

Expected result

Last good: https://openqa.suse.de/tests/1526061#step/yast2_i/16

Further details

Always latest result in this scenario: latest


Related issues

Related to openQA Tests - action #30616: [sle][functional][svirt-hyperv][u] test fails in rescuesystem - empty X window or stuck in installer loadingResolved2018-01-222018-04-24

Related to openQA Tests - action #33064: [functional][u][hyperv][hard] svirt-hyperv tests loose key presses. Related to FreeRDP update?Resolved2018-03-122018-04-24

Related to openQA Infrastructure - action #18608: [tools][sle][u][functional][research][medium] Implement proper clean up for images on jump hosts, e.g. s390pb, hyperv host, svirt and a proper monitoringWorkable

Related to openQA Tests - action #33190: [sle][functional][u][svirt-hyperv][medium] test fails in apache_nss with timeout on apache service, timing issue on typing?Rejected2018-03-132018-07-03

Related to openQA Tests - action #33091: [functional][hyperv][u] test fails in accept_license on slow systemResolved2018-03-122018-05-22

Related to openQA Tests - action #32608: [sle][functional][sporadic][hard][svirt-hyperv]test fails in consoletest_setup in sle-15-Installer-DVD-x86_64-Build489.1-minimal+base@svirt-hypervResolved2018-03-012018-05-22

Related to openQA Tests - action #33022: [sle][functional][u][hyperv][sporadic] test fails in zypper_inRejected2018-03-092018-07-03

Related to openQA Tests - action #10206: [tools]libvirt tests (Xen, Hyper-V, VMware)Resolved2016-01-13

Related to openQA Tests - action #34471: [functional][opensuse][u][medium] too early matching in too generic needle text-login-20160812New2018-04-08

Has duplicate openQA Tests - action #33322: [functional][hyperv] test fails in yast2_i; longer timeout neededRejected2018-03-15

Has duplicate openQA Tests - action #33583: [sle][functional] test fails in yast2_i - Installing packages is taking to long in Xen PVRejected2018-03-212018-04-24

History

#1 Updated by JERiveraMoya about 2 years ago

  • Subject changed from test fails in yast2_i - mising needles for Installation Report succesful to [sle][functional] test fails in yast2_i - mising needles for Installation Report succesful

#2 Updated by okurz about 2 years ago

  • Related to action #30616: [sle][functional][svirt-hyperv][u] test fails in rescuesystem - empty X window or stuck in installer loading added

#3 Updated by okurz about 2 years ago

  • Related to action #33064: [functional][u][hyperv][hard] svirt-hyperv tests loose key presses. Related to FreeRDP update? added

#4 Updated by okurz about 2 years ago

  • Related to action #18608: [tools][sle][u][functional][research][medium] Implement proper clean up for images on jump hosts, e.g. s390pb, hyperv host, svirt and a proper monitoring added

#5 Updated by okurz about 2 years ago

  • Related to action #33190: [sle][functional][u][svirt-hyperv][medium] test fails in apache_nss with timeout on apache service, timing issue on typing? added

#6 Updated by okurz about 2 years ago

  • Related to action #33091: [functional][hyperv][u] test fails in accept_license on slow system added

#7 Updated by okurz about 2 years ago

  • Related to action #32608: [sle][functional][sporadic][hard][svirt-hyperv]test fails in consoletest_setup in sle-15-Installer-DVD-x86_64-Build489.1-minimal+base@svirt-hyperv added

#8 Updated by okurz about 2 years ago

  • Related to action #33022: [sle][functional][u][hyperv][sporadic] test fails in zypper_in added

#9 Updated by okurz about 2 years ago

  • Related to action #10206: [tools]libvirt tests (Xen, Hyper-V, VMware) added

#10 Updated by okurz about 2 years ago

  • Subject changed from [sle][functional] test fails in yast2_i - mising needles for Installation Report succesful to [sle][functional][u][hyperv] test fails in yast2_i - mising needles for Installation Report succesful
  • Target version set to Milestone 18

#11 Updated by okurz about 2 years ago

  • Subject changed from [sle][functional][u][hyperv] test fails in yast2_i - mising needles for Installation Report succesful to [sle][functional][u][hyperv][yast] test fails in yast2_i - (mising needles?, rather too low timeout for hyperv) for Installation Report succesful
  • Status changed from New to Feedback
  • Assignee set to okurz

https://openqa.suse.de/tests/1525973/modules/yast2_i/steps/20 looks like pretty obvious too timeout.

It seems our hyperv virtualization is notoriously slow but we have not set a higher timeout_scale on the machine yet so I set this now with

TIMEOUT_SCALE=3
_COMMENT="set TIMEOUT_SCALE as hyperv virtualization is notoriously slow, see https://progress.opensuse.org/issues/32926"

or are there any objections?

#12 Updated by okurz about 2 years ago

  • Has duplicate action #33322: [functional][hyperv] test fails in yast2_i; longer timeout needed added

#13 Updated by michalnowak about 2 years ago

Good idea.

#14 Updated by okurz about 2 years ago

  • Has duplicate action #33583: [sle][functional] test fails in yast2_i - Installing packages is taking to long in Xen PV added

#15 Updated by okurz about 2 years ago

as reported in #33583 we still see errors which seem to be hyperv-timeout related.

@michalnowak what would you say, increase TIMEOUT_SCALE even more or is it something else or yast2_i specific timeout increase?

#16 Updated by michalnowak about 2 years ago

okurz wrote:

as reported in #33583 we still see errors which seem to be hyperv-timeout related.

@michalnowak what would you say, increase TIMEOUT_SCALE even more or is it something else or yast2_i specific timeout increase?

Actually the subject of that ticket was wrong to blame Hyper-V, it was Xen PV. It should not happen there at all, will investigate it as it happened for the second time in two days.

#17 Updated by okurz about 2 years ago

  • Status changed from Feedback to Workable
  • Assignee changed from okurz to riafarov

riafarov, I invested some effort to prevent generic "text-login" needles but you created one text-login-20180315 reintroducing the same problem we saw before, e.g. in https://openqa.suse.de/tests/1607827#step/consoletest_setup/5 where the wrong tty-prompt is matched prematurely. But apparently you created that needle for sle-15-Installer-DVD-x86_64-Build503.1-autoyast_bug-887126_autoinst@64bit so maybe it's needed so let's please crosscheck together shortly what the right approach would be.

#18 Updated by michalnowak about 2 years ago

text-login-20180315 is prematurely matching on TTY1 while we are waiting for the console to be switched to TTY6. This is where the switch is still in progress but openQA start typing login name: https://openqa.suse.de/tests/1607827#step/consoletest_setup/3.

In situations where Hyper-V is faster, we are luckier and both text-login-20180315 and tty6-selected are matched: https://openqa.suse.de/tests/1605086#step/consoletest_setup/2 But only the latter one should have been.

#19 Updated by riafarov about 2 years ago

  • Subject changed from [sle][functional][u][hyperv][yast] test fails in yast2_i - (mising needles?, rather too low timeout for hyperv) for Installation Report succesful to [sle][functional][u][hyperv][yast][fast] test fails in yast2_i - (mising needles?, rather too low timeout for hyperv) for Installation Report succesful
  • Due date set to 2018-04-24

I will pick it up as next item, if no-one else will do so. Adding it to the sprint. It doesn't look like easy ticket, and I don't have overview anyway (I don't remember all the needles I've created and reasons why).

#20 Updated by riafarov about 2 years ago

  • Subject changed from [sle][functional][u][hyperv][yast][fast] test fails in yast2_i - (mising needles?, rather too low timeout for hyperv) for Installation Report succesful to [sle][functional][y][hyperv][yast][fast] test fails in yast2_i - (mising needles?, rather too low timeout for hyperv) for Installation Report succesful

#21 Updated by riafarov about 2 years ago

  • Status changed from Workable to In Progress

#22 Updated by riafarov about 2 years ago

https://gitlab.suse.de/openqa/os-autoinst-needles-sles/merge_requests/809 to remove needle. Will propose option to match "tty$nr-selected", currently we have

if (match_has_tag("tty$nr-selected") or match_has_tag("text-login")) {
     type_string "$user\n";
     handle_password_prompt;
}

#23 Updated by riafarov about 2 years ago

  • Status changed from In Progress to Feedback

https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/4855

Removing needle I've created didn't help to Anton's case: http://kimball.arch.suse.de/tests/1124#step/login/6 so we definitely need better solution, e.g. one I've proposed.

#24 Updated by riafarov about 2 years ago

  • Subject changed from [sle][functional][y][hyperv][yast][fast] test fails in yast2_i - (mising needles?, rather too low timeout for hyperv) for Installation Report succesful to [sle][functional][y][hyperv][yast][fast] avoid typing username before switched tty (was: test fails in yast2_i - (mising needles?, rather too low timeout for hyperv) for Installation Report succesful)

#26 Updated by riafarov about 2 years ago

wait_still_screen 3 didn't help here: https://openqa.opensuse.org/tests/659550#step/zypper_add_repos/1
Proposing: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/4907 to identify all problematic cases and then decide how to proceed.

#27 Updated by riafarov about 2 years ago

So, remaining part is ti get stats when we get soft-failure, create required needles and after that assert tty by default.

#28 Updated by mgriessmeier about 2 years ago

  • Due date changed from 2018-04-24 to 2018-05-08

as agreed with riafarov, moving to Sprint 16

#29 Updated by okurz about 2 years ago

  • Subject changed from [sle][functional][y][hyperv][yast][fast] avoid typing username before switched tty (was: test fails in yast2_i - (mising needles?, rather too low timeout for hyperv) for Installation Report succesful) to [sle][functional][y][hyperv][yast] avoid typing username before switched tty (was: test fails in yast2_i - (mising needles?, rather too low timeout for hyperv) for Installation Report succesful)
  • Status changed from Feedback to Workable
  • Assignee deleted (riafarov)
  • Target version changed from Milestone 18 to Milestone 16

unassigning from riarafov due to temporary absence.

Next steps:

#30 Updated by okurz about 2 years ago

  • Subject changed from [sle][functional][y][hyperv][yast] avoid typing username before switched tty (was: test fails in yast2_i - (mising needles?, rather too low timeout for hyperv) for Installation Report succesful) to [sle][functional][y][hyperv] avoid typing username before switched tty (was: test fails in yast2_i - (mising needles?, rather too low timeout for hyperv) for Installation Report succesful)

#31 Updated by okurz about 2 years ago

  • Related to action #35742: [functional][u] test fails in consoletest_setup - premature user name typing added

#32 Updated by okurz about 2 years ago

  • Related to deleted (action #35742: [functional][u] test fails in consoletest_setup - premature user name typing)

#33 Updated by SLindoMansilla about 2 years ago

  • Related to action #34471: [functional][opensuse][u][medium] too early matching in too generic needle text-login-20160812 added

#34 Updated by JERiveraMoya about 2 years ago

  • Status changed from Workable to In Progress
  • Assignee set to JERiveraMoya

#35 Updated by JERiveraMoya about 2 years ago

Taking the interval 11 days, since the pull request creating the soft-failure, searching poo#32926 for SLE we can found the soft-failure only in test mau-qa_kernel_kexec:

  • sle 15 & sle-12-SP3 for aarch64 & x86_64 all soft-failing since 14 days ago, except for 1 passed out 10 for sle 15 x86_64. Searching for openSUSE there are more test suites affected:
  • Only for aarch64 in openSUSE Tumbleweed AArch64. Available statistics 1 out 1, soft-failing in create_hdd_gnome, create_hdd_textmode, minimalx, textmode. All needles seems have been already created in both side.

#36 Updated by riafarov about 2 years ago

  • Due date changed from 2018-05-08 to 2018-05-22

Next step would be to make new behavior to be default and verify that tests identified by JERivera still work.

#37 Updated by riafarov about 2 years ago

  • Subject changed from [sle][functional][y][hyperv] avoid typing username before switched tty (was: test fails in yast2_i - (mising needles?, rather too low timeout for hyperv) for Installation Report succesful) to [sle][functional][y][hyperv][medium] avoid typing username before switched tty (was: test fails in yast2_i - (mising needles?, rather too low timeout for hyperv) for Installation Report succesful)

#38 Updated by okurz about 2 years ago

still found references of the soft-fail, e.g. https://openqa.opensuse.org/tests/674191?#step/NM_wpa2_enterprise/31 where the soft-fail refers to a needle needed for the post_fail_hook. Albeit, if this test case would fail early because the tty5-selected needle would not exist it is not a big problem, one would create the needle and rerun the test to execute the post_fail_hook properly. I created a tty5-selected needle and retriggered

#39 Updated by JERiveraMoya about 2 years ago

The area was missing, once I created the area for tty5 and restarted is not displaying the soft-failure anymore: https://openqa.opensuse.org/tests/674288#

#40 Updated by okurz about 2 years ago

  • Assignee changed from JERiveraMoya to okurz

As discussed in the daily meeting let's proceed ensuring that the tty$nr-needles cover three areas, "Welcome", "tty$nr" and "login:". I looked over all "tt$nr-selected" needles on https://openqa.suse.de/admin/needles , deleted old ones that did not match for quite some time, created new needles which explicitly cover the three above mentioned sections and deleted the corresponding old needles covering not all three areas. I will monitor some retriggered jobs and then conduct the same approach for o3.

#41 Updated by riafarov about 2 years ago

Any progress here? I would at least merge PR from Joaquin and then you can improve solution with your proposal. WDYT?

#42 Updated by okurz about 2 years ago

So I did the corresponding steps for o3 as well.

https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/5023 was the PR from jrivera which we already closed.

https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/5026 is my PR

Looking at

https://w3.nue.suse.com/~okurz/openqa_opensuse_org_status.html I can find no references to poo#32926, however on https://w3.nue.suse.com/~okurz/openqa_suse_de_status.html I can still find references to "kernel_kexec" that look for "linux-login" and make it wrong as the prompt is looked for before actual reboot which I think we need to handle first.

#44 Updated by okurz about 2 years ago

  • Status changed from In Progress to Feedback

both merged, waiting to see if we missed anything.

#46 Updated by riafarov about 2 years ago

  • Status changed from Feedback to Resolved

No issues identified, resolving.

Also available in: Atom PDF