QA » openQA Project

The main benefit is the way it stores artefacts (i.e. logs from tests) and that you can ssh into running test containers to debug within the environment. This can make it much faster to turn around a problem

General Considerations¶

• circleci provides list of commands which can be used in config. E.g. checkout will check out current branch into project in current environment (e.g. in container) .
• circleci provides local tool circleci : -- check syntax using command circleci config process .circleci/config.yml -- execute job from .circleci/config.yml: check syntax using command circleci local execute --job testui
• choose between machine and docker executor. machine allows to manage containers with docker commands, while docker executor allows command only inside container. Officially machine executor is not supported for commands of local circleci tool. E.g. for me checkout command didn't work locally
• circleci Dashboard allows to restart jobs "with SSH" and easily login into executor using command printed at job start

OpenQA with circleci¶

• tests look running much faster (may need detailed analysis) -- unit and integration tests : 20-30 min vs 11 min -- UI tests 20-30 min vs 9 min -- fullstack 15 min vs 10 min
• it looks circleci environment is quite sensitive to unstable tests and shows failures more often. (copmaring to travis).
• easy to run tests on local machine using circleci tool. (The downside is ablsence of cache, so local commands despite being simple and explicit do execute quite long, because they need to build image).

TODO¶

• codecov may still need work to have it didn't work from circleci (because of fork?)
• 'documentation generation job' from travis was left out of scope
• troubleshoot unstable tests