QA » openQA Project » openQA Tests

Note for the benefit of drive-by readers: This is based on previous notes, and will need some refinement. I intend to identify some actionable points from this.

dzedro wrote:

send_key is often used with wait_still_screen or wait_screen_change, despite wait_screen_change is part of send_key as parameter to wait after key press. I guess most of people don't know that there is parameter in send_key for wait_screen_change, which is still questionable if it will be sufficient wait function.

Btw I was thinking that the parameter is not working since it was using wait_idle, which became deprecated. In one tests was created send_key_and_wait [1], use of wait_still_screen could be parameter of send_key, replace or be second parameter with wait_screen_change?

[1] https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/tests/yast2_gui/yast2_instserver.pm#L27

So the current version of send_key_and_wait is basically send_key $key, $stilltime, $timeout//=5; wait_still_screen $stilltime, $timeout;. On average it's called with a $stilltime of 2 seconds... so I see two attractive options:

1. Extending send_key with a new $wait_still_screen option, like type_string/password woud make sense for consistency/simplicity
2. Adding send_key_still_screen which defaults to 2 seconds stilltime and 5 seconds timeout.

To further pout this to the test, we could merge all calls of the send_key_and_wait function and see if that works well.

@dzedro should we move this to "openQA project" or is your intention to discuss this within the scope of "test maintainers" mainly?

I think you are raising many important points and it is good to see that you care about general performance and stability which I only see in a very limited scope among users of openQA.

In general I think we have a lot of potential for improvement as mainly we are using a system from around 2013-2015 and there had been a lot of improvements in the domain of mainly virtualized environments since then, e.g. improvement using newer qemu features, virtio, qcow tune parameters, filesystems from which machines are started, etc.

dzedro wrote:

Maybe decreasing IO load on workers, snapshots on qemu backend are done basically everywhere even where it doesn't make much sense, small jobs(one, few) tests, multimachine and installations.

I add QEMU_DISABLE_SNAPSHOTS=1 into most of QAM suites, there are installations with following tests, but fail is mostly some random timeout, typing, etc. issue. The test is restarted anyway... It could make sense to disable snapshots by default on suites where snapshot is waste of IO, exceptions could overwrite it with QEMU_DISABLE_SNAPSHOTS=0.

Maybe the setting should be moved to a different layer? The job group, not the test, for instance.

okurz wrote:

@dzedro should we move this to "openQA project" or is your intention to discuss this within the scope of "test maintainers" mainly?

I think you are raising many important points and it is good to see that you care about general performance and stability which I only see in a very limited scope among users of openQA.

In general I think we have a lot of potential for improvement as mainly we are using a system from around 2013-2015 and there had been a lot of improvements in the domain of mainly virtualized environments since then, e.g. improvement using newer qemu features, virtio, qcow tune parameters, filesystems from which machines are started, etc.

Anybody is welcome, to discuss such change.

@cdywan I would vote for option 1. extend existing send_key.
I was disabling it in specific tests where snapshots are pointless, when you have big group and you don't know exactly where you want to have snapshots then per test.

• Default timeout based on (over)load
  • Query host load data (not VM guest/ worker)
  • (e.g. load average: 2,48, 1,90, 1,91)
  • 8 CPUs, 16 tasks loaded => timeout of 8 seconds
  • timeout += (load/cpu) > 1 ? (load / cpu) * seconds(=1) : 0
  • Swappiness (free, Perl API?)
  • timeout += swap ? (seconds=1) : 0
  • I/O load (iostat, Perl API?)
  • timeout += ??
  • Networking
  • useful? may not be relevant to our setups
• Wait on ... instead of seconds
  • cpu idle cycles (context switches?)
  • disk sync
• Conceive a test with artificially busy processes
  • HUNGRY_RAM=n (malloc(1024*1024*1024);)
  • HUNGRY_PROCESSES=n (yes >/dev/null)
  • HUNGRY_DISKS=n
  • artificially hungry node app

These tickets are not on high prio

This ticket was set to Normal priority but was not updated within the SLO period. Please consider picking up this ticket or just set the ticket to the next lower priority.