Project

General

Profile

Actions

action #21052

closed

[qam] test fails in shotwell_export - failure on first launch

Added by pcervinka over 6 years ago. Updated over 6 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Bugs in existing tests
Target version:
-
Start date:
2017-08-03
Due date:
% Done:

100%

Estimated time:
Difficulty:

Description

Observation

openQA test in scenario sle-12-SP2-Desktop-DVD-Updates-x86_64-qam-regression-other@64bit fails in
shotwell_export

Reproducible

Fails since (at least) Build 20170801-1

Expected result

Last good: 20170731-1 (or more recent)

Further details

Always latest result in this scenario: latest

If there is a failure in previous shotwell_* test openQA will revert to last successful
milestone "boot_to_desktop". It means that shotwell will launch with "first launch" dialog.
Tests shotwell_edit and shotwell_export expect "shotwell-launched", it ends with failure.

Tests shotwell_edit and shotwell_export should be more robust and detect "first launch".

Actions #1

Updated by dasantiago over 6 years ago

This test is missing an:

assert_screen 'shotwell-first-launch';
wait_screen_change { send_key "ret"; };

This is happening because the build will run 3 tests in order:

  • shotwell_import
  • shotwell_edit
  • shotwell_export

On shotwell_import it's ok, but on shotwell_edit the test is invoking the function clean_shotwell, that will remove the folder "/home/$username/.local/share/shotwell", making again necessary to assert the initial screen.

So either we remove the invocation from shotwell_edit, or we assert the screen in shotwell_export.

Actions #2

Updated by pluskalm over 6 years ago

  • Assignee set to vsvecova
Actions #3

Updated by vsvecova over 6 years ago

  • Status changed from New to In Progress
  • % Done changed from 0 to 80

I agree with Petr's original comment. Upon opening the application, the first shotwell module (shotwell_import) handles the first-launch window and closes it, while the next two modules (shotwell_edit and shotwell_export) do not expect the welcome window any more and merely assert that the application is launched.

When everything goes well, the welcome screen does not appear any more, even though every shotwell* module calls clean_shotwell() at the end, because the clean function only removes shotwell library files and does not reset shotwell completely.

However, if any of the previous modules fail, the SUT state is restored to the last anchored snapshot, which in this case is right after boot_to_desktop, therefore the next shotwell* module after the failure behaves again as first launch and the welcome window appears again.

I opted for removing the code to start shotwell out of the individual shotwell* tests and instead added a start_shotwell() function into the x11regression library, in accordance with the DRY principle. The function starts the application and subsequently checks whether there is a welcome window pop up, in order to prevent failures when the state is restored from the last good snapshot.

PR: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/3530

Verification runs:
SLED12 SP2: http://dreamyhamster.suse.cz/tests/746
SLED12 SP3: http://dreamyhamster.suse.cz/tests/744
Note that in both runs, the shotwell_edit module fails due to another issue (21048), however, the following module, shotwell_export, detects the welcome screen and runs fine.

Needles MR: Not applicable - no new needles were needed.

Actions #4

Updated by vsvecova over 6 years ago

  • Status changed from In Progress to Resolved
  • % Done changed from 80 to 100

The shotwell tests are now working as expected in production:

SLED12 SP2: https://openqa.suse.de/tests/1148427#step/shotwell_export/4
SLED12 SP3: https://openqa.suse.de/tests/1148275#step/shotwell_export/4

The failure in shotwell_edit is unrelated to this issue and is owing to poo#21048.

Actions

Also available in: Atom PDF