Evaluate circleci for openQA
We use Travis CI successfully but having our build environment primarily based on containers anyway we might benefit from a different approach to save time. OBS is also using it
#5 Updated by andriinikitin 10 months ago
- Status changed from In Progress to Closed
- % Done changed from 0 to 100
- circleci provides list of commands which can be used in config. E.g.
checkoutwill check out current branch into
projectin current environment (e.g. in container) .
- circleci provides local tool
circleci: -- check syntax using command
circleci config process .circleci/config.yml-- execute job from .circleci/config.yml: check syntax using command
circleci local execute --job testui
- choose between
machineallows to manage containers with docker commands, while
dockerexecutor allows command only inside container. Officially
machineexecutor is not supported for commands of local
circlecitool. E.g. for me
checkoutcommand didn't work locally
- circleci Dashboard allows to restart jobs "with SSH" and easily login into executor using command printed at job start
OpenQA with circleci¶
- tests look running much faster (may need detailed analysis) -- unit and integration tests : 20-30 min vs 11 min -- UI tests 20-30 min vs 9 min -- fullstack 15 min vs 10 min
- it looks circleci environment is quite sensitive to unstable tests and shows failures more often. (copmaring to travis).
- easy to run tests on local machine using
circlecitool. (The downside is ablsence of cache, so local commands despite being simple and explicit do execute quite long, because they need to build image).
- codecov may still need work to have it didn't work from circleci (because of fork?)
- 'documentation generation job' from travis was left out of scope
- troubleshoot unstable tests
hi andriinikitin, could you please use the status "Resolved" rather than "Closed" as we commonly do for all tickets? This is following https://progress.opensuse.org/projects/openqav3/wiki#ticket-workflow
Also, I guess you want to have a follow-up ticket for the TODOs?