Project

General

Profile

action #62525

coordination #36712: [saga] Use YaST specific framework for GUI testing

coordination #50672: [funtional][y][epic] Use libyui-rest-api for YaST modules testing

[functional][y] Propose development environment for the rspec test development with libyui

Added by riafarov almost 2 years ago. Updated about 1 year ago.

Status:
Resolved
Priority:
Normal
Assignee:
Target version:
SUSE QA - Milestone 32
Start date:
2020-01-22
Due date:
2020-02-11
% Done:

0%

Estimated time:
8.00 h

Description

See motivation in the parent ticket.

Story

As a QA engineer I would like to have scalable, easy to recover environment to develop tests with libyui-restapi and execute them.

Suggestions

As we want to run X11 tests, containers are not the best option, but TW based VM with shared path to the code would be an option.
ruby part should be included in the setup, but C++ libraries we can leave out of scope.

Wrapping tests in openQA is out of scope of this ticket.

https://github.com/jknphy/libyui_client and https://github.com/libyui/libyui-rest-api/

History

#1 Updated by riafarov almost 2 years ago

  • Description updated (diff)
  • Status changed from New to Workable
  • Estimated time set to 8.00 h

#2 Updated by ybonatakis almost 2 years ago

  • Status changed from Workable to In Progress
  • Assignee set to ybonatakis

i assigned this to myself to take also a first idea of the whole thing we try to do but feel free to do your own research if you want

#3 Updated by JERiveraMoya almost 2 years ago

Tool picked up by @Ybonatakis with rbenv worked for me fine in a VM with Leap 15.2. Update of C/C++ compiler (several packages) was needed plus installation of three additional packages:
zypper in gcc
zypper in libopenssl-devel readline-devel zlib-devel

#4 Updated by ybonatakis almost 2 years ago

  • Status changed from In Progress to Feedback

For this sprint i tried to focus in two particular objectives:

  • to provide solution for a common testing environment
  • to run a rspec against a virtual machine

Testing env

between rvm and rbenv(most common tools out there), i picked the later.
with rbenv i was able to run the bundler and install the gems under every environment i tried.
I have created a script that it helps to start off. The script needs additional work but its purpose
is to install and configure the rbenv and finish with the installation of the gems, such as rspec can run tests
straight away at the end.

Depending from the distribution of choice, different dependencies might be requested. For example, on Leap you need to install readline-devel, where it is not required in TW. Both of them will ask for libopenssl-devel, if it is not installed on the system.

With all the above moves, we create an environment with which will use the same ruby version to be used for the required gems that list in the Gemfile and Gemfile.lock. TW does not satisfy the minimum requirement for those gem versions on TW. The ruby version i used is 2.6.5. This is because it was compatible with the bundler 2.0.2 it is used. The use of ruby 2.7.0 was broken for my case as it doesnt support vagrant at the moment.

Rspec against virtual box

The scenario i tried to exercise was:

  • Test should start a virtual machine
  • run rspecs against that vm

To run the above scenario the system should have installed vagrant(2.6) and libvirt(libvirt-5.1.0-lp151.7.6.1.x86_64). In additional it requires the vagrant-libvirt plugin. The vagrant-libvirt which adds a Libvirt provider to Vagrant. Vagrant and vagrant-libvirt can be found in the repositories of TW but the second is not available for Leap. For the later you need to run

$ vagrant plugin install vagrant-libvirt

Between the package version and the plugin one I encountered some issues with another module(nokogiri) that comes as dependency of the vagrant-libvirt, which was coming to a conflict with some other version from the installed gems.

OBS publishes some vagrant boxes in download.suse.de(where i get the one that i used to implement the testing) but there are official boxes on https://app.vagrantup.com/opensuse

The one that i downloaded was the Tumbleweed.x86_64-libvirt.box which is semi-official. This comes without yast so ihad to installed all the necessary components. I search how we could create vagrant boxes from the isos and i found a way with kiwi. Or we can ask Dan Čermák which seems to suport the vagrant images in IBS (https://confluence.suse.com/pages/viewpage.action?spaceKey=applications&title=Vagrant+and\
+Vagrant+Boxes+for+openSUSE+and+SLE)

What i accomplish here is to run a separate rake task before the rspec test, to call vagrant up and then execute commands to this machine to start the yast app. from this point and after the rest-api should take over to do whatever it should do

More about Vagrant

Vagrant is a tool to create a virtualized environment which supports many hypervisors with one of them libvirt (default is virtualbox). The only thing that we need is a Vagrantfile somewhere in the filesystem to start a VM. Downside is that it might be slow when first initialize the VM.

Vagrant provides configuration and basic network to access the vm from outside. however it is configurable and possible for other extented configuration. We can run ssh commands through the vagrant cli, or get the public configuration and ssh directly if needed

Proposals

  • The test runner should take a configuration file to pass settings to the test(ex vagrant box to test, etc)
  • Implement Logger, and Reporter
  • Add cleanup

relative code in the env branch

#5 Updated by riafarov almost 2 years ago

  • Status changed from Feedback to Resolved

Also available in: Atom PDF