coordination #80142: [saga][epic] Scale out: Redundant/load-balancing deployments of openQA, easy containers, containers on kubernetes
fallback mechanism for apache, e.g. on osd
As openQA user I want to get human readable info about problems in openqa.suse.de in case of planned/unplanned downtime
- AC1: http://openqa.suse.de is ALWAYS returns html page , no matter what is happening with openqa web service or other backend parts
- AC2: in case of planned downtime http://openqa.suse.de showing web page containing estimated downtime and some high level details of what actually happening ( "database upgrade" , "solving space issue " etc. )
- Apache which currently physically situated in same machine with openqa.suse.de should be moved to separate physical machine and probably getting Load Balancer with second node
- Need to enable apache custom error responses feature which would give users sane error message in case troubles in osd
- to achieve AC2 need to setup mechanism allowing update of custom error responses page on the fly so tools team can share more certain info about planned downtimes
This should be used as addition to email notification. To cover two main drawbacks of mailing list notifications :
1.) there are always people who are not in mailing list but using openqa.suse.de
2.) there are always people who will miss email notification and will start disturb qa tools team with question "why osd is down?"
#3 Updated by okurz almost 3 years ago
- Tracker changed from communication to action
- Project changed from openSUSE admin to openQA Tests
- Subject changed from apache needs to have fallback mechanism enabled on osd to [tools] apache needs to have fallback mechanism enabled on osd
- Category set to Infrastructure
The webserver for osd is handled by the tools team and definitely not openSUSE heroes. If at all then it is SUSE-IT that can help hence moving to "openQA tests" and adding "[tools]" tag. IMHO a more scalable future-proof approach is to use e.g. kubernetes which has internal load balancing for services available.
- Subject changed from [tools] fallback mechanism for apache on osd to fallback mechanism for apache on osd
- Target version set to future
not currently on the backlog of the SUSE QA Tools team however linked to another story that we work on which might be able to help.