openQA Project - coordination #39719: [saga][epic] Detection of "known failures" for stable tests, easy test results review and easy tracking of known issues
Data about coverage can be found inside openQA and obtained via API. We need to solve the association between incident (on the openQA and template side) and source package (the actual "coverage" side).
A simple tool is needed to fetch aforementioned data via openQA API and store them in some lightweight database. The data to store should be merged with the already stored data (union of both datasets) as we are interested in maximum coverage.
AC1: implement database storage/retrieval tool
AC2: maximum coverage needs to take the tested product/code-stream into account
AC3: include list of tests that are not running in test report
AC4: entries in DB are updatable for both adding new available coverage but also removing tests from the list
- Research available database backends
We have several options for the DB backend. As the main goal is to store JSON-like data without strong relations it might be good to evaluate NoSQL databases. We have probably following options:
- sqlite -- easy to setup, non-complicated use; SQL-like, might be slow
- redis -- very fast; not really meant for NoSQL data, more like key-value store
- CouchDB -- ideal for JSON data; currently not supported in Leap/TW
- MongoDB -- same as CouchDB; license problems
- RethinkDB -- similar to other NoSQL databases; meant for pushing data to real-time web applications
It seems, that for start we shall begin with sqlite and switch to another DB if the speed issue shows up.