action #40475

[functional][y][saga] Establish YaST team split

Added by riafarov over 1 year ago. Updated 9 days ago.

Status:BlockedStart date:31/08/2018
Priority:HighDue date:24/03/2020
Assignee:riafarov% Done:

91%

Category:Enhancement to existing testsEstimated time:76.00 hours
Target version:QA - future
Difficulty:
Duration: 408

Description

Motivation

We agreed that we should focus on YaST more as it's developed internally and close feedback loop.
Basic idea is to do QA with backlog which is synced with development. It includes integration testing, as we currently do, but more focused on YaST and being a precondition for other tests if installation doesn't work.
It will allow us to test features during the development and provide feedback, so it's cheaper and easier to change things. Additionally, it will prevent broken build from reaching integration testing when multiple QA teams will be affected

Acceptance Criteria

  • AC1: separate job group for QA SLE YaST team; this should concern o.s.d
  • AC2: We know how the openSUSE community, including release managers of Leap and Tumbleweed, would like to benefit from this (and if they see this as a benefit)
  • AC3: We have a clear list of test scenarios which the QA SLE YaST team owns
  • AC4: The purpose of each test scenario owned by the Y team is described

Tasks

Short-term: split job group, take responsibility for the review, set it as a precondition for other functional tests. If installation is broken, there is not much sense in testing other components.
Establish process to judge overall product quality.
Long-term: sub-team joins SysMgmt team for the QA and helping to establish earlier testing

Recommendation: Take care of better communication around this change (as discussed in f2f conversation - perhaps even a simple set of slides. We hate them. But we need to get the message across)

Further points

  • reduce scope for the reviewer (better focus), hence better understanding of the tests
  • will be easier to involve YaST team and collaborate with them
  • not much benefits for U-subteam, but definitely no harm
  • potentially will affect collaboration between subteams, learn HPC/HA experience

Subtasks

action #40478: [functional][y] support multiple job groups for ttmWorkable

action #42191: [functional][y][epic] Have separate job group for YaST su...Resolvedriafarov

action #42227: [functional][y] Move relevant scenarios to YaST job group...Resolvedriafarov

action #42236: [functional][y] Identify all scenarios to be split into t...Resolvedriafarov

action #42503: [functional][y] Define test matrix for the YaST job groupResolvedriafarov

action #42578: [functional][y][spike] Rethink what is the purpose of cre...ResolvedJERiveraMoya

action #42914: [functional][y][u] Update review workflow for split domainsResolvedoorlov

action #43796: [functional][y] Implement ext4 specific test for autoyast...ResolvedJERiveraMoya

action #42854: [functional][y][epic] Introduce relevant installation val...New

action #42848: [sle][functional][y] Do not trigger not relevant user spa...Resolvedriafarov

action #42917: [sle][functional][y] Introduce only relevant tests module...ResolvedJERiveraMoya

action #52418: [functional][y] Validate setup in lvm+RAID1 test suiteResolvedybonatakis

action #57794: [functional][y] Validate crypt_no_lvm test suiteResolvedJERiveraMoya

action #58909: [functional][y] Validate ext4 installationResolvedsyrianidou_sofia

action #58912: [functional][y] Validate gpt installationResolvedJRivrain

action #63328: [functional][y] Enable validation for cryptlvm+activate_e...New

action #42494: [epic][functional][y] Split/adapt scenarios to move insta...Resolvedriafarov

action #42941: [sle][functional][y] implement validation for lvm-full-en...Resolvedybonatakis

action #48173: [sle][functional][y] implement validation for encrypt lvm...Resolvedmloviska

action #48401: [functional][y] Implement test module to validate iscsi_i...Resolvedmloviska

action #48176: [sle][functional][y] implement validation for autoyast_gn...Resolvedybonatakis

action #52508: [functional][y] Validate skip_registration+all-packages-i...Rejectedriafarov

action #53258: [functional][y] Validate installation with btrfsResolvedoorlov

action #54779: [sle][functional][y] fix validation tests for encrypted l...Resolvedoorlov

action #55421: [functional][y] change btrfs test suite as per #53258Resolvedoorlov

action #56990: [functional][y] Validate minimal+base scenario with modul...Closedybonatakis


Related issues

Related to openQA Tests - action #41852: [functional][y][sle] add more staging tests Resolved 01/10/2018 20/11/2018
Related to openQA Project - action #42851: [functional][y][tools] Implement view for multiple job gr... Resolved 24/10/2018 04/12/2018

History

#1 Updated by okurz over 1 year ago

  • Category set to Enhancement to existing tests

#2 Updated by okurz over 1 year ago

  • Target version set to Milestone 26

#3 Updated by riafarov over 1 year ago

  • Due date set to 09/10/2018

due to changes in a related task

#4 Updated by okurz over 1 year ago

  • Due date changed from 09/10/2018 to 31/08/2018

due to changes in a related task

#5 Updated by riafarov over 1 year ago

  • Due date set to 31/08/2018

due to changes in a related task

#6 Updated by sebchlad over 1 year ago

Thanks Rodion for writing down the idea of having separated QA SLE YaST team. It is a very clear statement and I see it fully aligned with what we've discussed over last several months.
I will add Alex C. as a watcher as his team members participate in the QA SLE YaST team.

I wonder why we consider finishing this by Milestone 26. This seems like we tackle here a very complex problem, but to me, I must admit, it doesn't look like that.
Of course this is a long term, perhaps more of a vision than a task: "Long-term: sub-team joins SysMgmt team for the QA and helping to establish earlier testing".

Looking at the project schedule - SP4 is about to be finished (from our pov).
SP1 will be entering quite important phases in 2-3 months from now.

I would consider more aggressive schedule for finishing this saga.

@okurz: @riafarov: what do you think?

I will propose some changes to the description of this ticket to better reflect "what should be done" in down-to-earth terms.

#7 Updated by okurz over 1 year ago

  • Related to action #41852: [functional][y][sle] add more staging tests added

#8 Updated by riafarov over 1 year ago

As a result of our discussion, I will change relations to have only mandatory tickets which have to be resolved and other, which aim improvements on established process.

#9 Updated by sebchlad over 1 year ago

  • Subject changed from [functional][y][saga] Proceed with YaST sub-team split to [functional][y][saga] Establish YaST sub-team split
  • Description updated (diff)

#10 Updated by sebchlad over 1 year ago

  • Description updated (diff)

#11 Updated by sebchlad over 1 year ago

riafarov wrote:

As a result of our discussion, I will change relations to have only mandatory tickets which have to be resolved and other, which aim improvements on established process.

+2. Definitely a good idea.

If you don't mind I will propose some specific mandatory tasks, which we should get done in order to fulfill the exit criteria

#12 Updated by okurz over 1 year ago

  • Description updated (diff)

#13 Updated by riafarov over 1 year ago

  • Due date changed from 06/11/2018 to 04/12/2018

due to changes in a related task

#14 Updated by riafarov over 1 year ago

First draft of the slides to explain motivation, benefits and plan can be found in the attachment section of the ticket.

#15 Updated by okurz over 1 year ago

  • Related to action #42851: [functional][y][tools] Implement view for multiple job groups added

#16 Updated by okurz about 1 year ago

  • Status changed from New to Blocked
  • Assignee set to okurz

Blocked by subtasks

#17 Updated by riafarov about 1 year ago

  • Due date changed from 12/02/2019 to 26/02/2019

due to changes in a related task

#18 Updated by okurz 11 months ago

  • Due date changed from 26/03/2019 to 18/06/2019

due to changes in a related task

#19 Updated by riafarov 11 months ago

  • Subject changed from [functional][y][saga] Establish YaST sub-team split to [functional][y][saga] Establish YaST team split

#20 Updated by okurz 9 months ago

  • Assignee changed from okurz to riafarov

Move to new QSF-y PO after I moved to the "tools"-team. I mainly checked the subject line so in individual instances you might not agree to take it over completely into QSF-y. Feel free to reassign to me or someone else in this case. Thanks.

#21 Updated by riafarov 9 months ago

  • Due date changed from 18/06/2019 to 09/07/2019

due to changes in a related task

#22 Updated by riafarov 7 months ago

  • Target version changed from Milestone 26 to Milestone 27

#23 Updated by mgriessmeier 5 months ago

  • Target version changed from Milestone 27 to Milestone 28

#24 Updated by riafarov 5 months ago

  • Due date changed from 08/10/2019 to 22/10/2019

due to changes in a related task

#25 Updated by riafarov 4 months ago

  • Target version changed from Milestone 28 to future

#26 Updated by riafarov 4 months ago

  • Due date changed from 05/11/2019 to 19/11/2019

due to changes in a related task

#27 Updated by riafarov 9 days ago

  • Due date changed from 11/02/2020 to 24/03/2020

due to changes in a related task

Also available in: Atom PDF