openSUSE admin

I just figured out that the same 500 happens if I try to comment to the issue.

Logs mention that connection to redis failed.

This is what I'm seeing at a cursory glance...

pagure01 (pagure):~ # systemctl status redis@default
× redis@default.service - Redis instance: default
     Loaded: loaded (/usr/lib/systemd/system/redis@.service; enabled; vendor preset: disabled)
     Active: failed (Result: signal) since Wed 2023-03-08 16:38:42 UTC; 4 days ago
    Process: 25073 ExecStart=/usr/sbin/redis-server /etc/redis/default.conf (code=killed, signal=KILL)
   Main PID: 25073 (code=killed, signal=KILL)
     Status: "Redis is loading..."

Mar 08 16:38:42 pagure01 systemd[1]: redis@default.service: Main process exited, code=killed, status=9/KILL
Mar 08 16:38:42 pagure01 systemd[1]: redis@default.service: Failed with result 'signal'.
Mar 08 16:38:42 pagure01 systemd[1]: Failed to start Redis instance: default.
Mar 08 16:38:42 pagure01 systemd[1]: redis@default.service: Scheduled restart job, restart counter is at 2183.
Mar 08 16:38:42 pagure01 systemd[1]: Stopped Redis instance: default.
Mar 08 16:38:42 pagure01 systemd[1]: redis@default.service: Start request repeated too quickly.
Mar 08 16:38:42 pagure01 systemd[1]: redis@default.service: Failed with result 'signal'.
Mar 08 16:38:42 pagure01 systemd[1]: Failed to start Redis instance: default.
pagure01 (pagure):~ # systemctl start redis@default
Job for redis@default.service failed because a fatal signal was delivered to the control process.
See "systemctl status redis@default.service" and "journalctl -xeu redis@default.service" for details.

Looks like redis is getting killed because we're out of memory. Trying to restart service to free up RAM.

SUSE IT ticket https://sd.suse.com/servicedesk/customer/portal/1/SD-114879

crameleon wrote in #note-10:

I recommend the application owner to configure the maxmemory and maxmemory-policy options in Redis to mitigate such issues.

sounds reasonable to me, I just don't know what a sane value for Redis on code.o.o would be. Any monitoring data you can use to identify an average value over the last months that would make sense to set?

Hi,

identifying an average to set as the maximum is no problem, I mostly wonder what policy suits Pagure best. These are the possible choices:

# MAXMEMORY POLICY: how Redis will select what to remove when maxmemory
# is reached. You can select one from the following behaviors:
#
# volatile-lru -> Evict using approximated LRU, only keys with an expire set.
# allkeys-lru -> Evict any key using approximated LRU.
# volatile-lfu -> Evict using approximated LFU, only keys with an expire set.
# allkeys-lfu -> Evict any key using approximated LFU.
# volatile-random -> Remove a random key having an expire set.
# allkeys-random -> Remove a random key, any key.
# volatile-ttl -> Remove the key with the nearest expire time (minor TTL)
# noeviction -> Don't evict anything, just return an error on write operations.