Project

General

Profile

action #120022

coordination #121879: [epic] Improvements in the test automation framework with libyui-rest-api

Implement Step in framework at the level of Test Layer with and without decoration

Added by coolgw 3 months ago. Updated about 1 month ago.

Status:
Resolved
Priority:
Low
Assignee:
Target version:
Start date:
2022-11-07
Due date:
% Done:

0%

Estimated time:

Description

Motivation

Once Yam squad is starting progressively to rewrite some YaST modules test cases with libyui-rest-api and apply a good design to them, like in this PR: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/15496
we still need to improve how each step is visualized, we can see that the screenshot we provide is not in sync with the action:
https://openqa.opensuse.org/tests/2857887#step/yast2_firewall_stop_service/5
Wrapping the method, for example like this wait_screen_change(sub { $testapi::distri->get_firewall()->start_firewall() } );
See Draft PR: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/15844/files#diff-8fc5ca29998de81bd0c818e44236ad9c51cc838bafc7429a1f30d83e43acbc3dR23
we can achieve: https://openqa.opensuse.org/tests/2857895#step/yast2_firewall_stop_service/8

The goal is to find a solution, perhaps decorator pattern in Perl or some sort of wrapping which is flexible and decoupled from the other layers.
Therefore in the example above we still run $testapi::distri->get_firewall()->start_firewall() but behind the scene the other actions will take place. But those actions should be assigned from outside, so Controller layer can have some variable to store and execute it, but doesn't know anything about testapi.pm.

Acceptance criteria

AC1: Research about potential solution to wrap functions in Perl
AC2: Define a wrapper/decorator for some methods of the Object
AC3: Being able to have other Object methods not decorated
AC4: We can switch off the mechanism in single place without issues.

Additional information

We could use this snippet as starting point:

#!/usr/bin/perl
use strict;
use warnings;

use experimental 'signatures';
use Sub::Util 'subname';
use feature qw(say);

# This utility function accepts a func ref and a code ref and wraps them
sub wrap ($func, $code) {
    no strict;
    no warnings 'redefine';
    *{ subname($func) } = $code->($func);
}

# This will be the wrapper. Like a Python decorator, it's accepts the wrapped
# function as the first argument, and returns a new function
sub debug($func) {
    my $func_name = subname($func);
    return sub (@args) {
        say {*STDERR} "DEBUG: Calling $func_name(@args)";
        $func->(@args);
    }
}

# You can wrap the function anywhere, but doing it before the function
# definition it easier to notice, and is also similar to Python decorators.
wrap(\&double, \&debug);
sub double($n) {
    return $n * 2;
  • 'debug' might be our step_decorator method where we can apply those additional functions, but in theory we should have step_around, step_before and step_after and for our case in principle we would add step_around in combination with step_after. To make them different decorator we'll give us more flexibility and we should be able to apply more than one as in our case.
  • Once the decorator is created the second task is to know where to put it? The simple way would be to put in the Test layer, completely independent of the framework, that is the most decoupled solution probably, but in fact we already have some knowledge of the methods for a Controller, meaning that some methods are action that change the screen, so for those step_around and step_after decorator can be assign in the code implementation of the Controller, and it could also be assign on the code implementation of the Page, if we will remove the Controllers in the future.
  • Instead of assigning specific implementation of decorators in the Object (Controller or Page) we will initialize this objects applying those wrap operations with values of those decorator, which will not be assigned directly in the Object, but in the setup of the libyui, in what we call the App. In that way at the beginning of these tests where we are testing YaST module we initially setup the specific implementation for this decorators. Then when the Object (Controller for now) is instantiated it will already contains by code the type of decorator but not the exact implementation, as it would be passed on initialization. If this suggestion doesn't work, we should discuss further where to do this wrap operations.

History

#1 Updated by JERiveraMoya 3 months ago

  • Tags set to qe-yast-refinement
  • Subject changed from Improve visualization on libyui (add decorator for libyui control level api) to Implement Step in framework at the level of Test Layer with and without decoration
  • Description updated (diff)

#2 Updated by JERiveraMoya 3 months ago

  • Description updated (diff)

#3 Updated by JERiveraMoya 3 months ago

  • Description updated (diff)

#4 Updated by JERiveraMoya 3 months ago

  • Description updated (diff)
  • Target version set to Current

#5 Updated by JERiveraMoya 3 months ago

  • Description updated (diff)

#6 Updated by JERiveraMoya 3 months ago

  • Priority changed from Normal to Low

#7 Updated by JERiveraMoya 3 months ago

  • Tags deleted (qe-yast-refinement)
  • Description updated (diff)
  • Status changed from New to Workable

#8 Updated by coolgw about 2 months ago

  • Assignee set to coolgw

#9 Updated by coolgw about 2 months ago

  • Status changed from Workable to In Progress

#10 Updated by JERiveraMoya about 2 months ago

  • Parent task set to #121879

#12 Updated by JERiveraMoya about 1 month ago

I think we did a nice investigation here, just try out things without big cost is interesting and even fun.
I would consider this as a PoC, could you please save this code in confluence?
The reason is that I had second thoughts about this studying other web automation frameworks. In those there are capability to take snapshots/screenshots for each step with a before and after. Of course that is not an option for us, we don't have such an evolved framework.
There is a simple solution for us using Page Model Object, which is to create a more fancy library for assertions.
Check comments of the creator of POM mentioned here: https://martinfowler.com/bliki/PageObject.html when talking about assertions.
Therefore at the cost of checking the value on the screen of the step we want to visualize, we can then call our library which will take the screenshot.
Of course cons is that we need to write more code to check something that afik because we use introspection with libyui rest api is always ok, therefore in the test we just have a call that wait for the value to be the one expected and when that happen the library takes the screenshot, advantage is that is doesn't have the complexity of that pattern and doesn't use other backend openqa function other than save_screenshot. Also those wrappers that we were trying here need to be set in each object which doesn't convince me much. wdyt about this? I will file a ticket for this second path.

#13 Updated by JERiveraMoya about 1 month ago

  • Status changed from In Progress to Resolved

Also available in: Atom PDF