Project

General

Profile

action #55262

coordination #80142: [saga][epic] Scale out openQA: Redundant/load-balancing deployments of openQA, easy containers, containers on kubernetes

Install Pgpool-II or PgBouncer before PostgreSQL for openQA instances, e.g. to be used on OSD

Added by kraih over 1 year ago. Updated 5 months ago.

Status:
New
Priority:
Low
Assignee:
-
Category:
Feature requests
Target version:
Start date:
2019-08-08
Due date:
% Done:

0%

Estimated time:
Difficulty:

Description

We've recently seen quite a few log messages like this.

"Aug 07 02:19:14 openqa openqa[10258]: DBI connect('dbname=openqa','geekotest',...) failed: FATAL:  remaining connection slots are reserved for non-replication superuser connections at /usr/lib/perl5/vendor_perl/5.18.2/Mojo/Pg.pm line 83."

And there are no obvious connection leaks in the webui. So it appears that at peak times we may be reaching the
default PostgreSQL connection limit with gru jobs (100 minus a few reserved connections). But even when we are
not at peak i count at least 68 active connections on OSD. Most are idle, and that is a lot of wasted resources.

PostgreSQL operates like an old Apache prefork server, so every idle connection requires a forked process.
Pgpool-II (and the more minimalistic PgBouncer)
are little daemons that run like a reverse proxy in front of PostgreSQL and that require almost no resources
keeping even thousands of idle connections alive. Real PostgreSQL connections would only be used when
there are active queries, and they would be shared.


Related issues

Related to openQA Project - action #69355: [spike] redundant/load-balancing webui deployments of openQAResolved2020-07-252020-10-08

History

#1 Updated by okurz over 1 year ago

so you are suggesting this to be installed but so far there is no reference to it within the packages or so, shouldn't we try to handle that upstream to e.g. for example "recommend" this?

Also, I have seen the message you gave as well. However shouldn't we rather investigate why this suddenly recently appeared and not "jump to conclusions"?

#2 Updated by okurz over 1 year ago

On 2019-09-17 we had a similar incidence – see discussion in https://chat.suse.de/group/openqa-dev – which might be preventable by the suggestion mentioned in this ticket.

#3 Updated by okurz 9 months ago

  • Priority changed from Normal to Low

#4 Updated by okurz 9 months ago

  • Related to action #69355: [spike] redundant/load-balancing webui deployments of openQA added

#5 Updated by okurz 6 months ago

  • Target version set to Ready

#6 Updated by okurz 5 months ago

  • Target version changed from Ready to future

#7 Updated by okurz 5 months ago

  • Project changed from openQA Infrastructure to openQA Project
  • Subject changed from Install Pgpool-II or PgBouncer before PostgreSQL for OSD to Install Pgpool-II or PgBouncer before PostgreSQL for openQA instances, e.g. to be used on OSD
  • Category set to Feature requests
  • Parent task set to #80142

Also available in: Atom PDF