action #55478
closed
Evaluate circleci for openQA
Added by okurz over 4 years ago.
Updated over 4 years ago.
Description
Motivation¶
We use Travis CI successfully but having our build environment primarily based on containers anyway we might benefit from a different approach to save time. OBS is also using it
- Target version set to Ready
The main benefit is the way it stores artefacts (i.e. logs from tests) and that you can ssh into running test containers to debug within the environment. This can make it much faster to turn around a problem
- Status changed from New to In Progress
- Assignee set to andriinikitin
- Estimated time set to 16.00 h
- Target version changed from Ready to Current Sprint
- Copied to action #56522: Create cron job which will create pull requests with list of current openQA-devel dependencies with version added
- Status changed from In Progress to Closed
- % Done changed from 0 to 100
General Considerations¶
- circleci provides list of commands which can be used in config. E.g.
checkout
will check out current branch into project
in current environment (e.g. in container) .
- circleci provides local tool
circleci
:
-- check syntax using command circleci config process .circleci/config.yml
-- execute job from .circleci/config.yml: check syntax using command circleci local execute --job testui
- choose between
machine
and docker
executor. machine
allows to manage containers with docker commands, while docker
executor allows command only inside container. Officially machine
executor is not supported for commands of local circleci
tool. E.g. for me checkout
command didn't work locally
- circleci Dashboard allows to restart jobs "with SSH" and easily login into executor using command printed at job start
OpenQA with circleci¶
- tests look running much faster (may need detailed analysis)
-- unit and integration tests : 20-30 min vs 11 min
-- UI tests 20-30 min vs 9 min
-- fullstack 15 min vs 10 min
- it looks circleci environment is quite sensitive to unstable tests and shows failures more often. (copmaring to travis).
- easy to run tests on local machine using
circleci
tool. (The downside is ablsence of cache, so local commands despite being simple and explicit do execute quite long, because they need to build image).
TODO¶
- codecov may still need work to have it didn't work from circleci (because of fork?)
- 'documentation generation job' from travis was left out of scope
- troubleshoot unstable tests
- Status changed from Closed to Resolved
- Related to action #73309: every time a direct dependency is updated in Factory our CI jobs fail until the package is updated added
Also available in: Atom
PDF