action #40988

action #40979: [sle][functional][y][epic] New test strategy for autoyast profiles from ay-tests repo

[sle][functional][y][timeboxed:16h] Establish pipeline for autoyast testing

Added by riafarov over 1 year ago. Updated 5 months ago.

Status:WorkableStart date:13/09/2018
Priority:NormalDue date:
Assignee:-% Done:

0%

Category:Spike/Research
Target version:QA - future
Difficulty:
Duration:

Description

Motivation

As QA engineer I would like to test product early too. After discussion with YaST we have agreed that we need to re-enable tests for autoyast before these reach staging.
For quite some time they were not executed, and now our proposal would be to reuse same approach we have in openQA.

Acceptance criteria

  1. There is a PoC pipeline for testing YaST package(s) during development->staging->build

History

#1 Updated by okurz over 1 year ago

  • Category set to Enhancement to existing tests
  • Target version set to future

#3 Updated by okurz over 1 year ago

If I understand correctly this ticket is what is also mentioned as an example in #42218 . If it is true that the YaST development team abandoned the autoyast integration tests then I am not convinced we should care either. They might have good reasons but I do not see it would be useful if they do the development part and we cover "their" testing. How can we clarify this?

#4 Updated by riafarov over 1 year ago

  • Due date set to 04/12/2018

Setting due date to reset it for the parent task. We need to clarify our motivation.

#5 Updated by riafarov over 1 year ago

  • Subject changed from [sle][functional][y] Move SLE 15 profiles from aytests repo to os-autoinst-distri-opensuse to [sle][functional][y] Establish pipeline for autoyast testing
  • Description updated (diff)
  • Status changed from New to Workable

#6 Updated by riafarov over 1 year ago

  • Subject changed from [sle][functional][y] Establish pipeline for autoyast testing to [sle][functional][y][timeboxed:8h] Establish pipeline for autoyast testing

#7 Updated by okurz over 1 year ago

  • Subject changed from [sle][functional][y][timeboxed:8h] Establish pipeline for autoyast testing to [sle][functional][y] Establish pipeline for autoyast testing

I suggest to call isotovideo from os-autoinst. Keep in mind that isotovideo also supports to be called without a vars.json file by specifying all necessary parameters as command line parameters, e.g. isotovideo -d productdir=$(pwd) version=foo distri=bar

#8 Updated by okurz over 1 year ago

  • Subject changed from [sle][functional][y] Establish pipeline for autoyast testing to [sle][functional][y][timeboxed:8h] Establish pipeline for autoyast testing

#9 Updated by okurz over 1 year ago

  • Target version changed from future to Milestone 21

#10 Updated by riafarov about 1 year ago

  • Due date changed from 04/12/2018 to 18/12/2018
  • Assignee set to riafarov

riafarov will update with the ideas we have discussed with schubi and okurz.

#11 Updated by riafarov about 1 year ago

  • Assignee deleted (riafarov)

I was not able to fully work on it, so un-assign myself for now and here is the current summary:
1. Packages do not have cycle dependencies, so we can run CI rebuilding single package + YaST:Head used as an update repo.
2. Conclusion from the point above is that we can run CI per PR
3. For AY we can use openQA as a good fitting tool.
4. For other packages we should wait till we get YaST UI testing framework.
5. Unit tests is still must have and we should cover as much as possible to get fast and easrly feedback.
6. To improve investigation we should always the iso with which we are sure that CI works if executed without recent changes.
7. There are concerns as jenkins nodes we have currently won't be capable to run heavy jobs, we have to try.

Here are more details about infrastructure:
These machines are:

  • vm-autoyast-ci-worker1.suse.cz
  • vm-autoyast-ci-worker2.suse.cz
  • vm-autoyast-ci-worker3.suse.cz

The root password is listed in the YaST/Jenkins[1] wiki page. However, it
would be a good thing to have your SSH public keys added to the
authorized_keys just in case we change the password or the authentication
method in the future (you can send me your public keys and I will take care).

Finally, it would be great if you can document all the steps so we can add
them to our Salt configuration[2]. Actually, if you plan to set more than one
machine, we could use Salt to configure the rest.

[1] https://wiki.microfocus.net/index.php/YaST/jenkins#SUSE_.28internal.29
[2] https://gitlab.suse.de/yast/infra/

https://ci.suse.de/computer/
https://wiki.microfocus.net/index.php?title=YaST/jenkins
https://gitlab.suse.de/yast/infra
https://gitlab.suse.de/yast/infra/blob/master/srv/salt/yast-jenkins/files/authorized_keys
https://ci.suse.de/view/YaST/job/yast-autoyast-profiles-test/
https://mailman.suse.de/mailman/listinfo/yast-internal
https://github.com/yast/autoyast-integration-test

https://build.opensuse.org/project/show/openSUSE:Factory:Staging:M
https://devopscube.com/jenkins-build-trigger-github-pull-request/

#12 Updated by riafarov about 1 year ago

  • Due date changed from 18/12/2018 to 15/01/2019
  • Target version changed from Milestone 21 to Milestone 22

More issues identified, postponing. riafarov will put more details.

#13 Updated by riafarov about 1 year ago

  • Due date changed from 15/01/2019 to 29/01/2019

#14 Updated by riafarov about 1 year ago

  • Due date changed from 29/01/2019 to 26/02/2019

#15 Updated by riafarov about 1 year ago

  • Due date deleted (26/02/2019)

Putting it to the product backlog, no motivation right now and we should get UI framework to at least consider using it there.

#16 Updated by riafarov about 1 year ago

  • Target version changed from Milestone 22 to Milestone 24

#17 Updated by okurz about 1 year ago

Well, Milestone 24 is probably not too late but I would really favor this ticket. Is there something I could help with?

#18 Updated by riafarov about 1 year ago

In general, I also would like to make it happen, but as mentioned in my last comment, it would be good to have UI framework in the state that we can try it out in the CI. So it's mainly about priority and not that we won't do that.

#19 Updated by okurz about 1 year ago

alright, I will trust you on this then :)

#20 Updated by riafarov 10 months ago

  • Target version changed from Milestone 24 to Milestone 25

#21 Updated by riafarov 9 months ago

  • Due date set to 13/08/2019
  • Target version changed from Milestone 25 to Milestone 27

#22 Updated by riafarov 7 months ago

  • Subject changed from [sle][functional][y][timeboxed:8h] Establish pipeline for autoyast testing to [sle][functional][y][timeboxed:16h] Establish pipeline for autoyast testing
  • Category changed from Enhancement to existing tests to Spike/Research

#23 Updated by JERiveraMoya 7 months ago

  • Due date changed from 13/08/2019 to 27/08/2019

#24 Updated by riafarov 7 months ago

  • Due date changed from 27/08/2019 to 22/10/2019
  • Target version changed from Milestone 27 to Milestone 30+

#25 Updated by riafarov 5 months ago

  • Due date deleted (22/10/2019)
  • Target version changed from Milestone 30+ to future

We are still not there to address this one.

Also available in: Atom PDF