action #45029
closederror 502 when triggering products with rsync.pl
0%
Description
Observation¶
sudo -u geekotest perl -I /opt/openqa-scripts /opt/openqa-scripts/rsync.pl --host https://openqa.suse.de --verbose --add-existing sle15_sp1
caused to trigger some media but failed on x86_64 with
error scheduling SLE-15-SP1-Installer-DVD-x86_64-Build121.1-Media1.iso
502 Proxy Error
"<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML 2.0//EN\">\n<html><head>\n<title>502 Proxy Error</title>\n</head><body>\n<h1>Proxy Error</h1>\n<p>The proxy server received an invalid\r\nresponse from an upstream server.<br />\r\nThe proxy server could not handle the request <em><a href=\"/api/v1/isos\">POST /api/v1/isos</a></em>.<p>\nReason: <strong>Error reading from remote server</strong></p></p>\n<p>Additionally, a 502 Bad Gateway\nerror was encountered while trying to use an ErrorDocument to handle the request.</p>\n<hr>\n<address>Apache/2.4.23 (Linux/SUSE) Server at openqa.suse.de Port 443</address>\n</body></html>\n"Can't locate object method "is_status_class" via package "Mojo::Message::Response" at /opt/openqa-scripts/rsync.pl line 1052.
Problem¶
I am not sure if we had that problem already in before and if coolo rejected it as a "openqa/scripts" issue. We could retry in rsync.pl on 502 of course.
Updated by okurz over 5 years ago
Reproduced a second time after trying the same call again. However, it seems like the product was still correctly scheduled.
Updated by okurz over 5 years ago
sure but it's because openQA did not respond in time.
/var/log/apache2/error_log tells me:
[Wed Dec 12 09:42:40.851969 2018] [proxy_http:error] [pid 30319] (70007)The timeout specified has expired: [client 2620:113:80c0:8080:10:160:0:207:46048] AH01102: error reading status line from remote server localhost:9526
[Wed Dec 12 09:42:40.855040 2018] [proxy:error] [pid 30319] [client 2620:113:80c0:8080:10:160:0:207:46048] AH00898: Error reading from remote server returned by /api/v1/isos
[Wed Dec 12 09:42:40.857616 2018] [negotiation:error] [pid 30319] [client 2620:113:80c0:8080:10:160:0:207:46048] AH00690: no acceptable variant: /usr/share/apache2/error/HTTP_BAD_GATEWAY.html.var
and /var/log/openqa says for the same time:
[2018-12-12T09:42:30.0421 CET] [info] not registering asset fixed/SLE-15-Installer-DVD-x86_64-GM-DVD1/ containing /
[2018-12-12T09:42:32.0544 CET] [info] not registering asset fixed/SLE-15-Installer-DVD-x86_64-GM-DVD1/ containing /
[2018-12-12T09:42:50.0502 CET] [warn] START_AFTER_TEST=textmode+role_kvm:64bit-ipmi not found - check for dependency typos and dependency cycles
…
[2018-12-12T09:42:50.0855 CET] [warn] textmode:uefi has no child, check its machine placed or dependency setting typos
So I see 18 seconds between the second and third line, I guess openQA is still busy processing the request?
Can you please tell the other teams to do less stuff on openQA because it's at the limits? ;)
Updated by coolo about 5 years ago
@andriinikitin could you pick all rsync.pl issues into its own category? There should be quite some.
Updated by andriinikitin about 5 years ago
@coolo sure, I will name it "obs integration" (will investigate how to create category or ask somebody to create it)
Updated by andriinikitin about 5 years ago
- Category changed from Regressions/Crashes to 443
Updated by okurz almost 5 years ago
- Category changed from 443 to Regressions/Crashes
Updated by okurz over 4 years ago
- Related to action #49535: Improve time to schedule a build added
Updated by okurz over 4 years ago
- Status changed from New to Resolved
- Assignee set to mkittler
probably solved with https://github.com/os-autoinst/openQA/pull/2036 . Didn't see the error again.