action #36280

[sle][functional][y][medium] test fails in snapper_undochange - console screen is not clear and it leads to problem with needle match

Added by zluo almost 2 years ago. Updated about 1 year ago.

Status:RejectedStart date:16/05/2018
Priority:HighDue date:
Assignee:riafarov% Done:

0%

Category:Bugs in existing tests
Target version:QA - future
Difficulty:medium
Duration:

Description

compare with previous successful test run, I found needle snapper-emergency-shell cannot be matched because on top of console still remains some lines.

https://openqa.suse.de/tests/1679964#step/snapper_undochange/21 shows correct match.

Observation

openQA test in scenario sle-15-Installer-DVD-x86_64-extra_tests_filesystem@64bit fails in
snapper_undochange

Reproducible

Fails since (at least) Build 617.5

Expected result

Last good: 611.1 (or more recent)

Further details

Always latest result in this scenario: latest

Suggestion

Investigate enabling of audit.d and see if we can disable it.
Instead of asserting the needle, look for the string in the serial output as pattern.


Related issues

Related to openQA Tests - action #34342: [functional][u][medium][sporadic] test fails in suseconne... Resolved 05/04/2018 09/10/2018

History

#1 Updated by JERiveraMoya almost 2 years ago

I was checking with @mloviska that the audit is enable so the logs appears before and makes the needle not match.

#2 Updated by okurz almost 2 years ago

  • Due date set to 19/06/2018
  • Target version set to Milestone 17

#4 Updated by okurz over 1 year ago

  • Priority changed from Normal to High

Seems to happen in current tests

#5 Updated by JERiveraMoya over 1 year ago

  • Subject changed from [sle][functional][u] test fails in snapper_undochange - console screen is not clear and it leads to problem with needle match to [sle][functional][u][medium] test fails in snapper_undochange - console screen is not clear and it leads to problem with needle match
  • Description updated (diff)
  • Difficulty set to medium

#6 Updated by JERiveraMoya over 1 year ago

  • Status changed from New to Workable

#7 Updated by JERiveraMoya over 1 year ago

  • Related to action #34342: [functional][u][medium][sporadic] test fails in suseconnect_scc - Extra console output makes asserts fail added

#8 Updated by clanig over 1 year ago

  • Status changed from Workable to In Progress
  • Assignee set to clanig

#9 Updated by clanig over 1 year ago

My progress so far:
1. The issue is located at the btrfs_test.pm which is used by several test cases.
2. I have been able to shift the check whether the correct shell spawns to the virtual serial console on the test machine. But testing continues on this serial console so I can't just switch back somewhere and everything is fine.
3. That causes another issue that the color tag isn't interpreted anymore so it's just passed through and waiting for the console, it can't match with "# " anymore so I had to add an alternative matching pattern in distribution.pm of the os-autoinst project on the test machine. When it turns out that this is a general issue with the serial console there should be all cases of patterns covered that can happen. I will investigate this later.
4. Switching to the serial console also broke some other checks. I am going to address these issues firstly.

Generally I hugely prefer matching patterns on the serial output instead of using needles whenever possible. Even for those needles that are currently working acceptably.

#10 Updated by okurz over 1 year ago

clanig wrote:

Generally I hugely prefer matching patterns on the serial output instead of using needles whenever possible. Even for those needles that are currently working acceptably.

Yes, definitely the way to go, good choice :)

#11 Updated by clanig over 1 year ago

I have created a pull request and it would be nice to get some feedback. For the automated serial console login there is already covered that the PS1-String is printed with ascii formatting chars. But this standard subroutine didn't work with the rescue.target shell so I target this only in the btrfs_test.pm. In case more tests need to switch to this shell it would make sense to make the standard login procedure capable to login there as well. That's just my opinion on it.
https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/5241

#12 Updated by okurz over 1 year ago

  • Target version changed from Milestone 17 to Milestone 17

#13 Updated by mgriessmeier over 1 year ago

  • Due date deleted (19/06/2018)

#14 Updated by okurz over 1 year ago

  • Target version changed from Milestone 17 to Milestone 18

#15 Updated by okurz over 1 year ago

  • Target version changed from Milestone 18 to future

#16 Updated by okurz over 1 year ago

  • Status changed from In Progress to New
  • Assignee deleted (clanig)

@clanig I assume you do not plan to work on this anymore? If you do I would be happy and you can reassign :)

#17 Updated by clanig over 1 year ago

@okurz:
Yes, I have to agree with you. It looks like I took too long on this beside the other things. I'm very sorry that I haven't finished it in a reasonable time.

#18 Updated by okurz about 1 year ago

  • Subject changed from [sle][functional][u][medium] test fails in snapper_undochange - console screen is not clear and it leads to problem with needle match to [sle][functional][y][medium] test fails in snapper_undochange - console screen is not clear and it leads to problem with needle match

I think "snapper" is more fitting for QSF-y

#19 Updated by riafarov about 1 year ago

  • Status changed from New to Rejected
  • Assignee set to riafarov

Seems that this got solved by some other changes: https://openqa.suse.de/tests/2419634#step/snapper_undochange/1
Looks pretty stable.

Also available in: Atom PDF