Project

General

Profile

Actions

action #1272

closed

Write some test cases for webUI

Added by ancorgs over 10 years ago. Updated almost 6 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
Feature requests
Target version:
Start date:
2014-01-27
Due date:
% Done:

60%

Estimated time:
30.00 h

Description

This should probably not be a separate task, since should be part of #1271. In any case, after the initial porting some unit tests and integration tests (I'm using Rails terminology here, that can be different to the Mojo one) will be needed in order to be able to verify that the port works as expected and to be able to add new features.

That means learning to write tests in Mojolicious, of course.


Checklist

  • Interface test: results of a finished test
  • Interface test: live view (running test)
  • Interface test: list of tests
  • Interface test: buildview
  • Responses from running controller: modlist, status, livelog and streaming
  • Uploadlog
  • Interface test: needle editor
  • Interface test: audioview
  • Interface test: needle diff
  • Interface test: view test source code
  • Responses from file controller: images/x, file/x, logfile/x, diskimages/x, needles/x/y
  • API test: jobs
  • API tests: workers
  • API tests: isos

Related issues 1 (0 open1 closed)

Has duplicate openQA Project - action #1715: write api openqa testsRejected2014-02-25

Actions
Actions #1

Updated by alarrosa over 10 years ago

  • Target version set to Sprint 01
Actions #2

Updated by alarrosa over 10 years ago

  • Assignee set to cwh
Actions #3

Updated by alarrosa over 10 years ago

  • Target version changed from Sprint 01 to Sprint 02
Actions #4

Updated by cwh over 10 years ago

  • % Done changed from 0 to 10
Actions #5

Updated by ancorgs over 10 years ago

Today we discussed several things needed in order to have proper tests.

  1. Fixtures: that is, example data in both the database and the testresults directory, leaving the production database and testresults untouched, of course.
  2. Setup and teardown mechanisms: that is, tests needs to ensure the dataset is "clean" before running and after finishing its execution.
  3. Some tests have a time-dependency on the fixtures.

Some useful stuff:

Actions #6

Updated by ancorgs over 10 years ago

Playing with https://github.com/tempire/mojoexample and having a look at the implementation of its tests suite. It's exactly what I had in mind for ours, so we can grab some ideas and code.

Actions #7

Updated by ancorgs over 10 years ago

I will describe here the ideas I want to copy/adapt from https://github.com/tempire/MojoExample

  • First of all, they have a facility Test::Database package providing the following functionality:
   # Creates an sqlite3 test.db database from DBIC Schema AND load the fixtures
   my $schema = Test::Database->new->create(Schema => 'test.db');
   # Creates an in-memory sqlite3 database from DBIC Schema AND load the fixtures
   my $schema = Test::Database->new->create(Schema => ':memory:');
  • They define a attribute for the application called 'schema' and a helper called 'db' for accessing it from the controllers. So they can:
  # At any point
  $theApp->schema # The first time it's called without having been explicitly assigned, it will default to the production db

  # In a controller
  $self->db->resultset('Photoset')->search

  # At the beginning of every test that is independent of Mojo (model tests)
  my $schema = Test::Database->new->create(Schema => ':memory:')
  $schema->test_whatever

  # At the beginning of Mojo tests
  my $schema = Test::Database->new->create(Schema => 'test.db');
  my $t = Test::Mojo->new('NameOfTheApp');
  $t->app->schema($schema);

In this way, they have a single point to access the database in the application, an easy way to create a temporary database with all the test fixtures (that are stored in 't/fixtures') loaded and a convenient way to change the database in use during execution time (handy for the tests).

Fixtures and volatile test databases. Almost everything we need.

Actions #8

Updated by ancorgs over 10 years ago

  • Status changed from New to In Progress
  • % Done changed from 10 to 20

After some pair programming (Christopher and myself), we already have a mechanism to run tests in an automatically created test database with some fixtures loaded from files. Let's write tons of tests.

Actions #9

Updated by ancorgs over 10 years ago

Old tests fixed to use the new temporary database mechanism. 'make test' is not needed anymore, the good old 'script/openqa test' manages everything now.

Actions #10

Updated by ancorgs over 10 years ago

  • % Done changed from 20 to 30

Test framework improved to also use temporary example testresults. Listing test adapted to use the new example data. Still not committed because I'm improving the example testresults.

Actions #11

Updated by ancorgs over 10 years ago

  • % Done changed from 30 to 50

Now we have a test for '/tests' which use example fixtures and example testresults (including succeeded and failed tests) in a safe way with time-dependent information (to test the time filter). Will all that in place, writing more tests should be almost peanuts (except those needing Javascript).

Actions #12

Updated by ancorgs over 10 years ago

  • Checklist item changed from to [ ] Interface test: results of a finished test, [ ] Interface test: live view (running test), [x] Interface test: list of tests, [ ] Interface test: buildview, [ ] Responses from running controller: modlist, status, livelog and streaming, [ ] Uploadlog, [ ] Interface test: needle editor, [ ] Interface test: audioview, [ ] Interface test: needle diff, [ ] Interface test: view test source code, [ ] Responses from file controller: images/x, file/x, logfile/x, diskimages/x, needles/x/y, [ ] API test: jobs, [ ] API tests: workers, [ ] API tests: isos

I created a list of tests we should implement based on the already existing infrastructure

Actions #13

Updated by ancorgs over 10 years ago

Added support for tests in subdirectories. Took me a while. I had a problem, I tried to solve it with Perl and regular expressions... you know the rest.

Actions #14

Updated by alarrosa over 10 years ago

  • Target version changed from Sprint 02 to Sprint 03
Actions #15

Updated by ancorgs about 10 years ago

Test::Mojo works nicely 'emulating' a web browser as long as there is no Javascript involved. The problem is that half of the openQA UI relies on Javascript. We have two options for testing the javascript

  1. Keep the current approach for testing server responses and a different test suite just for testing the Javascript using Mocha, Jasmine or something similar.
  2. Add javascript support for our perl-based test suite so we can simulate a more full-featured browser.

I have been researching about the second idea (that is, looking for a perl equivalent to Ruby's Capybara) and apparently we have three main options (apart from more custom-made solutions):

  • Test::WWW:Mechanize::Mojo, a library that merges Test::Mojo and Test::WWW::Mechanize.
  • Brownie, a browser integration framework which supports two drivers (backends to really drive the interaction with the web server): WWW::Mechanize and Selenium RemoteWebDriver
  • Wight, a browser integration framework with PhantomJS as backend.

PhantomJS is awesome and we already have packages in OBS. It would be my first option. Unfortunately Wight looks way less mature than Brownie.

Mechanize does not support javascript, unless we can use WWW::Scripter::Plugin::Javascript, Gtk2::WebKit::Mechanize or WWW-Mechanize-Firefox. All of them are supposed to be implementations of Mechanize. If they really work as drop-in replacements, Test::WWW:Mechanize::Mojo and Brownie+Mechanize would become nice options for our purposes.

Brownie+Selenium RemoteWebDriver would mean needing a Selenium server to run the tests (a small Java monster). Or not? The good news are that PhantomJS implements the WebDriver Wire protocol, which means that can act as a Selenium server replacement, so in theory you could use Selenium::Remote::Driver with PhantomJS, with no real Selenium at all. On the other hand, we could also try WWW::WebKit which is supposed to be a drop-in replacement for WWW::Selenium. If it works, we could use Brownie+RemoteWebDriver without needing a server running at all, none Selenium or PhantomJS.

In my opinion, we should really try to use Test::WWW:Mechanize::Mojo with a javascript enabled Mechanize replacement as a first option.

Actions #16

Updated by ancorgs about 10 years ago

  • Checklist item changed from [ ] Interface test: results of a finished test, [ ] Interface test: live view (running test), [x] Interface test: list of tests, [ ] Interface test: buildview, [ ] Responses from running controller: modlist, status, livelog and streaming, [ ] Uploadlog, [ ] Interface test: needle editor, [ ] Interface test: audioview, [ ] Interface test: needle diff, [ ] Interface test: view test source code, [ ] Responses from file controller: images/x, file/x, logfile/x, diskimages/x, needles/x/y, [ ] API test: jobs, [ ] API tests: workers, [ ] API tests: isos to [ ] Interface test: results of a finished test, [ ] Interface test: live view (running test), [x] Interface test: list of tests, [ ] Interface test: buildview, [ ] Responses from running controller: modlist, status, livelog and streaming, [ ] Uploadlog, [ ] Interface test: needle editor, [ ] Interface test: audioview, [ ] Interface test: needle diff, [ ] Interface test: view test source code, [x] Responses from file controller: images/x, file/x, logfile/x, diskimages/x, needles/x/y, [ ] API test: jobs, [ ] API tests: workers, [ ] API tests: isos

Tests for file controller added. Two bugs killed in the process.

Actions #17

Updated by alarrosa about 10 years ago

  • Target version changed from Sprint 03 to Sprint 04
Actions #18

Updated by alarrosa about 10 years ago

  • Target version changed from Sprint 04 to Sprint 05
Actions #19

Updated by alarrosa about 10 years ago

  • Target version changed from Sprint 05 to Sprint 06
Actions #20

Updated by alarrosa about 10 years ago

  • Target version deleted (Sprint 06)
Actions #21

Updated by ancorgs about 10 years ago

  • Checklist item changed from [ ] Interface test: results of a finished test, [ ] Interface test: live view (running test), [x] Interface test: list of tests, [ ] Interface test: buildview, [ ] Responses from running controller: modlist, status, livelog and streaming, [ ] Uploadlog, [ ] Interface test: needle editor, [ ] Interface test: audioview, [ ] Interface test: needle diff, [ ] Interface test: view test source code, [x] Responses from file controller: images/x, file/x, logfile/x, diskimages/x, needles/x/y, [ ] API test: jobs, [ ] API tests: workers, [ ] API tests: isos to [ ] Interface test: results of a finished test, [ ] Interface test: live view (running test), [x] Interface test: list of tests, [x] Interface test: buildview, [ ] Responses from running controller: modlist, status, livelog and streaming, [ ] Uploadlog, [ ] Interface test: needle editor, [ ] Interface test: audioview, [ ] Interface test: needle diff, [ ] Interface test: view test source code, [x] Responses from file controller: images/x, file/x, logfile/x, diskimages/x, needles/x/y, [ ] API test: jobs, [ ] API tests: workers, [ ] API tests: isos

Tests for the new buildview included in #1385. One strike more in the checklist.

Actions #22

Updated by ancorgs almost 10 years ago

Fixing a bug in massive job restarting and adding tests for API:V1:Jobs in the process https://github.com/os-autoinst/openQA/pull/87

Actions #23

Updated by coolo over 9 years ago

  • Status changed from In Progress to New
  • Assignee deleted (cwh)
  • Priority changed from Immediate to Normal
Actions #24

Updated by coolo over 9 years ago

  • Category set to 124
Actions #25

Updated by coolo over 9 years ago

https://github.com/os-autoinst/openQA/pull/177 is a PoC for using phantomjs together with Selenium::Remote::Driver

Actions #26

Updated by okurz over 8 years ago

  • Checklist item changed from [ ] Interface test: results of a finished test, [ ] Interface test: live view (running test), [x] Interface test: list of tests, [x] Interface test: buildview, [ ] Responses from running controller: modlist, status, livelog and streaming, [ ] Uploadlog, [ ] Interface test: needle editor, [ ] Interface test: audioview, [ ] Interface test: needle diff, [ ] Interface test: view test source code, [x] Responses from file controller: images/x, file/x, logfile/x, diskimages/x, needles/x/y, [ ] API test: jobs, [ ] API tests: workers, [ ] API tests: isos to [ ] Interface test: results of a finished test, [ ] Interface test: live view (running test), [x] Interface test: list of tests, [x] Interface test: buildview, [ ] Responses from running controller: modlist, status, livelog and streaming, [ ] Uploadlog, [x] Interface test: needle editor, [ ] Interface test: audioview, [ ] Interface test: needle diff, [ ] Interface test: view test source code, [x] Responses from file controller: images/x, file/x, logfile/x, diskimages/x, needles/x/y, [ ] API test: jobs, [ ] API tests: workers, [ ] API tests: isos

needle editor test fixed with

commit d3a6e159bc39da8ac9740d49961080dac7f3e9a9
Author: Max Lin mlin@suse.com
Date: Wed Mar 18 17:53:25 2015 +0800

Improve needle editor test coverage, action#6562

Testing needle editor based on Selenium::Remote:Driver and phantomjs.
Actions #27

Updated by okurz over 8 years ago

  • Checklist item changed from to [x] API tests: isos
Actions #28

Updated by okurz over 8 years ago

isos tests ("assets") considered done with
commit 981dde70fda4e27e0850d5091be32e76b777a668
Author: Ondrej Holecek oholecek@suse.com
Date: Thu Apr 23 10:47:07 2015 +0200

add asset tests

- check proper asset assignements
- check job is cloned if asset creator is cloned
Actions #29

Updated by okurz over 8 years ago

  • Assignee set to dheidler

@dheidler as you currently have only one task assigned you might want to keep this as a "fallback" task :-) Please could you check on the current status of tests and update this ticket

Actions #30

Updated by dheidler over 8 years ago

  • Checklist item changed from to [x] API tests: workers
Actions #31

Updated by dheidler over 8 years ago

  • Checklist item changed from to [ ] API tests: workers
Actions #32

Updated by dheidler over 8 years ago

  • Checklist item changed from [ ] Interface test: results of a finished test, [ ] Interface test: live view (running test), [x] Interface test: list of tests, [x] Interface test: buildview, [ ] Responses from running controller: modlist, status, livelog and streaming, [ ] Uploadlog, [x] Interface test: needle editor, [ ] Interface test: audioview, [ ] Interface test: needle diff, [ ] Interface test: view test source code, [x] Responses from file controller: images/x, file/x, logfile/x, diskimages/x, needles/x/y, [ ] API test: jobs, [ ] API tests: workers, [x] API tests: isos to [ ] Interface test: results of a finished test, [ ] Interface test: live view (running test), [x] Interface test: list of tests, [x] Interface test: buildview, [ ] Responses from running controller: modlist, status, livelog and streaming, [ ] Uploadlog, [x] Interface test: needle editor, [ ] Interface test: audioview, [ ] Interface test: needle diff, [ ] Interface test: view test source code, [x] Responses from file controller: images/x, file/x, logfile/x, diskimages/x, needles/x/y, [x] API test: jobs, [x] API tests: workers, [x] API tests: isos
  • % Done changed from 50 to 60
./t/api/01-workers.t ...................... ok
...
./t/api/04-jobs.t ......................... ok
Actions #33

Updated by dheidler over 8 years ago

  • Checklist item changed from [ ] Interface test: results of a finished test, [ ] Interface test: live view (running test), [x] Interface test: list of tests, [x] Interface test: buildview, [ ] Responses from running controller: modlist, status, livelog and streaming, [ ] Uploadlog, [x] Interface test: needle editor, [ ] Interface test: audioview, [ ] Interface test: needle diff, [ ] Interface test: view test source code, [x] Responses from file controller: images/x, file/x, logfile/x, diskimages/x, needles/x/y, [x] API test: jobs, [x] API tests: workers, [x] API tests: isos to [ ] Interface test: results of a finished test, [ ] Interface test: live view (running test), [x] Interface test: list of tests, [x] Interface test: buildview, [ ] Responses from running controller: modlist, status, livelog and streaming, [ ] Uploadlog, [x] Interface test: needle editor, [ ] Interface test: audioview, [ ] Interface test: needle diff, [x] Interface test: view test source code, [x] Responses from file controller: images/x, file/x, logfile/x, diskimages/x, needles/x/y, [x] API test: jobs, [x] API tests: workers, [x] API tests: isos

t/ui/03-source.t

Actions #34

Updated by dheidler over 7 years ago

  • Assignee changed from dheidler to okurz
Actions #35

Updated by okurz over 7 years ago

  • Checklist item changed from [ ] Interface test: results of a finished test, [ ] Interface test: live view (running test), [x] Interface test: list of tests, [x] Interface test: buildview, [ ] Responses from running controller: modlist, status, livelog and streaming, [ ] Uploadlog, [x] Interface test: needle editor, [ ] Interface test: audioview, [ ] Interface test: needle diff, [x] Interface test: view test source code, [x] Responses from file controller: images/x, file/x, logfile/x, diskimages/x, needles/x/y, [x] API test: jobs, [x] API tests: workers, [x] API tests: isos to [x] Interface test: results of a finished test, [ ] Interface test: live view (running test), [x] Interface test: list of tests, [x] Interface test: buildview, [ ] Responses from running controller: modlist, status, livelog and streaming, [ ] Uploadlog, [x] Interface test: needle editor, [ ] Interface test: audioview, [ ] Interface test: needle diff, [x] Interface test: view test source code, [x] Responses from file controller: images/x, file/x, logfile/x, diskimages/x, needles/x/y, [x] API test: jobs, [x] API tests: workers, [x] API tests: isos
  • Assignee deleted (okurz)

See t/ui/18-tests-details.t since

commit 7fac90a
Author: Oliver Kurz okurz@suse.de
Date: Fri Jun 10 14:57:37 2016 +0200

Add proper test for wait_serial output on web UI

New fake test results are added to one existing job resembling a bit more how
a real "sshfs" test looks like including a wait_serial output.
'tests-src' is renamed to 'tests-details', testing both src window as well as
wait_serial output. It can be extended to test the custom test fails which
also have sparingly, e.g. when the backend dies.
Actions #36

Updated by mkittler over 7 years ago

  • Checklist item changed from to [x] Interface test: needle diff
Actions #37

Updated by mkittler over 7 years ago

  • Checklist item changed from to [ ] Interface test: needle diff
Actions #38

Updated by okurz over 7 years ago

  • Target version set to future
Actions #39

Updated by EDiGiacinto about 7 years ago

  • Checklist item changed from to [x] Interface test: live view (running test)
Actions #40

Updated by EDiGiacinto about 7 years ago

  • Checklist item changed from to [ ] Interface test: live view (running test)
Actions #41

Updated by coolo over 6 years ago

  • Status changed from New to Resolved

Closing this epic - we are pretty good already with the full tests and with new stuff added.

Actions #42

Updated by okurz almost 6 years ago

  • Target version changed from future to future
Actions

Also available in: Atom PDF