tickets #137843
Updated by cboltz about 1 year ago
In the last hour, I added two hotfixes on code.o.o which should be replaced by something more sane ;-)
## High load caused by Bytespider / bytedance bot
The Bytespyder bot made about 40% of all HTTP requests, resulting in 4 gunicorn processes, each of them eats 99% CPU - and gateway errors for many users.
As a hotfix, I extended /etc/nginx/vhosts/code.opensuse.org.conf with
if ($http_user_agent = "Mozilla/5.0 (Linux; Android 5.0) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; Bytespider; spider-feedback@bytedance.com)"){
return 444;
}
I'd hope that a robots.txt also does the job.
## Postfix DNS failures
The Pagure timeouts and errors caused the sending of quite some mails to root, but these could not be delivered because
(Host or domain name not found. Name service error for name=relay.infra.opensuse.org type=AAAA: Host not found, try again)
so for some reason it tried to reach relay.i.o.o over IPv6, but it only has a v4 address.
Forcing Postfix to only use IPv4 resulted in
(Host or domain name not found. Name service error for name=relay.infra.opensuse.org type=A: Host not found, try again)
so there must be something wrong with the DNS config.
As a hotfix, I added `relayhost=[192.168.47.4]` at the end of main.cf master.cf to completely avoid the DNS lookups, and the queued mails got delivered.
Needless to say that hardcoding the IP is a bad idea, so we'll need to find out why Postfix fails to do DNS lookups.
Doing manual DNS queries with `host` or `dig` work, so this must be something specific to Postfix.