action #41600
open
coordination #80142: [saga][epic] Scale out: Redundant/load-balancing deployments of openQA, easy containers, containers on kubernetes
fallback mechanism for apache, e.g. on osd
Added by asmorodskyi over 6 years ago.
Updated about 4 years ago.
Category:
Feature requests
Description
User Story¶
As openQA user I want to get human readable info about problems in openqa.suse.de in case of planned/unplanned downtime
Acceptance criteria¶
- AC1: http://openqa.suse.de is ALWAYS returns html page , no matter what is happening with openqa web service or other backend parts
- AC2: in case of planned downtime http://openqa.suse.de showing web page containing estimated downtime and some high level details of what actually happening ( "database upgrade" , "solving space issue " etc. )
Suggestions¶
- Apache which currently physically situated in same machine with openqa.suse.de should be moved to separate physical machine and probably getting Load Balancer with second node
- Need to enable apache custom error responses feature which would give users sane error message in case troubles in osd
- to achieve AC2 need to setup mechanism allowing update of custom error responses page on the fly so tools team can share more certain info about planned downtimes
Further details¶
This should be used as addition to email notification. To cover two main drawbacks of mailing list notifications :
1.) there are always people who are not in mailing list but using openqa.suse.de
2.) there are always people who will miss email notification and will start disturb qa tools team with question "why osd is down?"
- Private changed from Yes to No
opensuse (and I think even some suse services) use haproxy to archive this.
- Tracker changed from communication to action
- Project changed from openSUSE admin to openQA Tests (public)
- Subject changed from apache needs to have fallback mechanism enabled on osd to [tools] apache needs to have fallback mechanism enabled on osd
- Category set to Infrastructure
The webserver for osd is handled by the tools team and definitely not openSUSE heroes. If at all then it is SUSE-IT that can help hence moving to "openQA tests" and adding "[tools]" tag. IMHO a more scalable future-proof approach is to use e.g. kubernetes which has internal load balancing for services available.
- Project changed from openQA Tests (public) to openQA Infrastructure (public)
- Subject changed from [tools] apache needs to have fallback mechanism enabled on osd to [tools] fallback mechanism for apache on osd
- Category deleted (
Infrastructure)
- Priority changed from Normal to Low
- Related to action #69355: [spike] redundant/load-balancing webui deployments of openQA added
- Subject changed from [tools] fallback mechanism for apache on osd to fallback mechanism for apache on osd
- Target version set to future
not currently on the backlog of the SUSE QA Tools team however linked to another story that we work on which might be able to help.
- Project changed from openQA Infrastructure (public) to openQA Project (public)
- Subject changed from fallback mechanism for apache on osd to fallback mechanism for apache, e.g. on osd
- Category set to Feature requests
- Parent task set to #80142
Also available in: Atom
PDF