Project

General

Profile

Actions

action #36448

closed

[functional][y] Fix serial exception catching feature from openQA to be able to use for job labels

Added by okurz over 6 years ago. Updated over 6 years ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
Bugs in existing tests
Target version:
SUSE QA (private) - Milestone 18
Start date:
2018-05-23
Due date:
2018-08-14
% Done:

0%

Estimated time:
13.00 h
Difficulty:

Description

Observation

openQA test in scenario sle-12-SP4-Server-DVD-Staging:B-x86_64-ext4@uefi-staging fails in
welcome
on https://bugzilla.suse.com/show_bug.cgi?id=1093659

Acceptance criteria

  • AC1: All jobs hitting this error will be able to record a soft failure with bsc#1093659

Suggestions

Further details

Always latest result in this scenario: latest


Related issues 1 (0 open1 closed)

Blocks openQA Tests (public) - action #38621: [functional][y] test fails in welcome - "Module is not signed with expected PKCS#7 message" (bsc#1093659) - Use serial exception catching feature from openQA to make sure the jobs reference the bug, e.g. as labelResolvedriafarov2018-05-23

Actions
Actions #1

Updated by okurz over 6 years ago

  • Subject changed from [functional][y] test fails in welcome - "Module is not signed with expected PKCS#7 message" (bsc#1093659) - Use serial exception catching feature from openQA to catch it to [functional][y][fast] test fails in welcome - "Module is not signed with expected PKCS#7 message" (bsc#1093659) - Use serial exception catching feature from openQA to catch it
  • Due date set to 2018-06-19
  • Priority changed from Normal to High
  • Target version set to Milestone 17

I assume it is way easier for us to develop this as long as the bug is there so I am adding it to the fast-lane.

Actions #2

Updated by riafarov over 6 years ago

Mentioned feature doesn't allow to set custom message, so requires changes in openQA framework. Additionally, it didn't work for me in other scenario. @okurz, please re-evaluate priority considering mentioned points.

Actions #3

Updated by okurz over 6 years ago

If this feature continues to be useless for us and is not used anywhere in os-autoinst-distri-opensuse you might at least be able to propose deletion again ;) I still see the prio as high because right now it's easier to look into this than later when the bug disappears. Of course, we can try to preserve the older build but then maybe it won't work anymore because the installer changed behaviour and we need to adjust test backwards to be able to test this. Better to take a look now.

Actions #4

Updated by JERiveraMoya over 6 years ago

  • Assignee set to JERiveraMoya
Actions #5

Updated by JERiveraMoya over 6 years ago

Not found any iso in OSD to at least see how it work the mentioned feature.

Actions #7

Updated by okurz over 6 years ago

You can ask dasantiago who implemented it. Be quick about that to use his limited availability ;)

Actions #8

Updated by riafarov over 6 years ago

  • Status changed from New to Workable
Actions #9

Updated by JERiveraMoya over 6 years ago

  • Status changed from Workable to In Progress
Actions #10

Updated by JERiveraMoya over 6 years ago

After debugging with Santiago, feature is working, we realized that we were calling the wrong function, see comment.
local test: http://dhcp254.suse.cz/tests/1640#step/welcome/9

Actions #11

Updated by JERiveraMoya over 6 years ago

  • Status changed from In Progress to Feedback
Actions #12

Updated by okurz over 6 years ago

  • Target version changed from Milestone 17 to Milestone 17
Actions #13

Updated by riafarov over 6 years ago

  • Due date changed from 2018-06-19 to 2018-07-03
Actions #14

Updated by okurz over 6 years ago

we still need test adaptions for this, not only os-autoinst, right? I would be interested in how the feature in action looks like from the test point of view.

Actions #15

Updated by JERiveraMoya over 6 years ago

  • Estimated time set to 13.00 h
Actions #16

Updated by JERiveraMoya over 6 years ago

PR in os-autoinst repo still not merged.

Actions #17

Updated by riafarov over 6 years ago

  • Due date changed from 2018-07-03 to 2018-07-31
Actions #19

Updated by okurz over 6 years ago

  • Subject changed from [functional][y][fast] test fails in welcome - "Module is not signed with expected PKCS#7 message" (bsc#1093659) - Use serial exception catching feature from openQA to catch it to [functional][y] test fails in welcome - "Module is not signed with expected PKCS#7 message" (bsc#1093659) - Use serial exception catching feature from openQA to catch it and make sure the jobs reference the bug, e.g. as label
  • Description updated (diff)
  • Target version changed from Milestone 17 to Milestone 18

improved description based on our discussion in the sprint planning meeting

Actions #20

Updated by JERiveraMoya over 6 years ago

  • Description updated (diff)
Actions #21

Updated by JERiveraMoya over 6 years ago

@riafarov will create a follow-up ticket about the automatic label.

Actions #22

Updated by JERiveraMoya over 6 years ago

  • Description updated (diff)
Actions #23

Updated by okurz over 6 years ago

As explained earlier: I am fine when you want to create another ticket but please make sure to do so because I want to prevent that we consider ourselves done even though the jobs are still making as much review work as in before. When you create a new ticket please also add it to the current sprint as well. I consider this story more important than other tickets as well which we could then postpone.

Actions #24

Updated by JERiveraMoya over 6 years ago

Yes, thanks for understanding, new ticket has to fall in this sprint, of course, I understand the importance of this ticket to remove tedious work in daily reviews and as a main and final work, it is just a way to account for shippable steps for the team. I will check it with @riafarov (forgot to add it as a watcher) to ensure is in the sprint.

Actions #25

Updated by okurz over 6 years ago

  • Copied to action #38621: [functional][y] test fails in welcome - "Module is not signed with expected PKCS#7 message" (bsc#1093659) - Use serial exception catching feature from openQA to make sure the jobs reference the bug, e.g. as label added
Actions #26

Updated by okurz over 6 years ago

  • Copied to deleted (action #38621: [functional][y] test fails in welcome - "Module is not signed with expected PKCS#7 message" (bsc#1093659) - Use serial exception catching feature from openQA to make sure the jobs reference the bug, e.g. as label)
Actions #27

Updated by okurz over 6 years ago

  • Related to action #38621: [functional][y] test fails in welcome - "Module is not signed with expected PKCS#7 message" (bsc#1093659) - Use serial exception catching feature from openQA to make sure the jobs reference the bug, e.g. as label added
Actions #28

Updated by okurz over 6 years ago

  • Related to deleted (action #38621: [functional][y] test fails in welcome - "Module is not signed with expected PKCS#7 message" (bsc#1093659) - Use serial exception catching feature from openQA to make sure the jobs reference the bug, e.g. as label)
Actions #29

Updated by okurz over 6 years ago

  • Blocks action #38621: [functional][y] test fails in welcome - "Module is not signed with expected PKCS#7 message" (bsc#1093659) - Use serial exception catching feature from openQA to make sure the jobs reference the bug, e.g. as label added
Actions #30

Updated by okurz over 6 years ago

  • Subject changed from [functional][y] test fails in welcome - "Module is not signed with expected PKCS#7 message" (bsc#1093659) - Use serial exception catching feature from openQA to catch it and make sure the jobs reference the bug, e.g. as label to [functional][y] Fix serial exception catching feature from openQA to be able to use for job labels
  • Description updated (diff)

ok, sure. I talked to riafarov and we hope we could make it more clear what the requirements are for this ticket and a followup which we pulled out into #38621 . So let's hope we can close this ticket soon whenever the tools team merged your PR in os-autoinst.

Actions #31

Updated by JERiveraMoya over 6 years ago

  • Status changed from Feedback to In Progress
Actions #32

Updated by JERiveraMoya over 6 years ago

PR merged :)

Actions #33

Updated by szarate over 6 years ago

This needs to be tested better, please add a unit test for this:

https://github.com/os-autoinst/os-autoinst/blob/fdcb65d22c47e2cdd0501532955a120a4b338e97/autotest.pm#L329

take a look at: https://github.com/os-autoinst/os-autoinst/pull/994 for details on how it fails.

Actions #34

Updated by riafarov over 6 years ago

  • Assignee changed from JERiveraMoya to riafarov

Taking over to fix initial PR

Actions #35

Updated by riafarov over 6 years ago

  • Status changed from In Progress to Feedback

https://github.com/os-autoinst/os-autoinst/pull/998
Bug in the code fixed + added unit test. So now can implement to put comment in case soft-failure/hard-failure in openQA repo, this part is for os-autoinst only.

Actions #36

Updated by riafarov over 6 years ago

@okurz: I've evaluated your suggestion and it's not possible to do that using os-autoinst only, as openQA sets the overall status of the job. I tried to return and stop gracefully, but seems there are some checks on front end side, which fail the job anyway, so will go further with idea of automated comment, or checking if we can soft-fail job with skipped modules.

Actions #37

Updated by okurz over 6 years ago

Ok, makes sense. We could add something to ttm as it already handles commenting on openQA. openqa-review does not

Actions #38

Updated by riafarov over 6 years ago

  • Due date changed from 2018-07-31 to 2018-08-14
Actions #39

Updated by riafarov over 6 years ago

  • Status changed from Feedback to Resolved

Changes are merged and deployed to osd.

Actions

Also available in: Atom PDF