action #32977
[functional][yast][y]Release notes content is not really checked
0%
Description
Release notes on sle15 have content from sles12-sp3 and test is passing.
Modify test to check for proper content, for example using assert_screen("release-notes-sles-".get_var('VERSION'));
Example of invalid test: http://openqa-apac1.suse.de/tests/372/modules/releasenotes/steps/5
History
#1
Updated by mkravec over 5 years ago
Updated by - Description updated (diff)
#2
Updated by okurz about 5 years ago
Updated by - Subject changed from Release notes content is not really checked to [functional]Release notes content is not really checked
- Due date set to 2018-03-27
- Target version set to Milestone 15
I recommend to not make ourselves rely on version specific needles. They are a lot of maintenance work and lead to other people distrusting openQA (or QA). However, what about simply creating workaround needles on top with a bugref?
#3
Updated by okurz about 5 years ago
- Due date deleted (
2018-03-27) - Assignee deleted (
mloviska) - Target version changed from Milestone 15 to Milestone 17
sprint13 too full, just as well as M15 and M16
#4
Updated by okurz about 5 years ago
- Subject changed from [functional]Release notes content is not really checked to [functional][yast][y]Release notes content is not really checked
#5
Updated by okurz about 5 years ago
- Checklist item changed from [ ] Remove needles matching non-specific content, [ ] Add needles with specific SLES version in the tag, [ ] Modify release-notes test module to
- Due date set to 2018-03-27
- Status changed from New to Workable
- Assignee set to okurz
- Target version changed from Milestone 17 to Milestone 15
Actually I want to check if I can not just generate a error-detection needle with workaround property.
#6
Updated by okurz about 5 years ago
- Status changed from Workable to Resolved
https://openqa.suse.de/tests/1551946#step/releasenotes/4 is showing the current, correct release notes. If there would be wrong release notes I would create an additional needle with workaround property to track it as soft-failure. This will also be monitored by openqa-review. I do not see the benefit in letting an additional scenario fail with a needle mismatch. We have enough of these and they don't help to build trust.