QA » openQA Project

On OSD we encounter upload errors for image creation jobs. Many (if not all) fail in the middle of uploading with a HTTP error 502.

Examples:

• aarch64:

  • https://openqa.suse.de/tests/1513505
  • https://openqa.suse.de/tests/1513484

• x86_64:

  • https://openqa.suse.de/tests/1514180
  • https://openqa.suse.de/tests/1514196

autoinst-log.txt:

[2018-03-01T13:11:14.0700 CET] [debug] QEMU: qemu-system-x86_64: terminating on signal 15 from pid 25994 (/usr/bin/isotovideo: backen)
[2018-03-01T13:11:14.0700 CET] [debug] sending magic and exit
[2018-03-01T13:11:14.0700 CET] [debug] received magic close
[2018-03-01T13:11:14.0712 CET] [debug] preparing hdd 1 for upload as SLE-15-x86_64-Build489.1-sled-gnome+development.qcow2 in qcow2
[2018-03-01T13:11:14.0712 CET] [debug] running nice ionice qemu-img convert -O qcow2 raid/l1 assets_public/SLE-15-x86_64-Build489.1-sled-gnome+development.qcow2 -c
[2018-03-01T13:15:47.0752 CET] [debug] runcmd pid 35259 returned 35259
25957: EXIT 0
[2018-03-01T13:15:48.0563 CET] [info] +++ worker notes +++
[2018-03-01T13:15:48.0563 CET] [info] end time: 2018-03-01 12:15:48
[2018-03-01T13:15:48.0563 CET] [info] result: done
[2018-03-01T13:15:48.0563 CET] [info] uploading logs_from_installation_system-y2logs.tar.bz2
[2018-03-01T13:15:48.0727 CET] [info] uploading SLE-15-x86_64-Build489.1-sled-gnome+development.qcow2
[2018-03-01T13:26:29.0611 CET] [error] Failed uploading chunk
[2018-03-01T13:26:29.0612 CET] [error] ERROR SLE-15-x86_64-Build489.1-sled-gnome+development.qcow2: 502 response: Proxy Error

worker-log.txt:

[2018-03-01T13:24:26.0299 CET] [info] SLE-15-x86_64-Build489.1-sled-gnome+development.qcow2: Processing chunk 1783/2117 avg speed ~976.562KB/s
[2018-03-01T13:24:26.0490 CET] [info] SLE-15-x86_64-Build489.1-sled-gnome+development.qcow2: Processing chunk 1784/2117 avg speed ~976.562KB/s
[2018-03-01T13:24:26.0712 CET] [info] SLE-15-x86_64-Build489.1-sled-gnome+development.qcow2: Processing chunk 1785/2117 avg speed ~976.562KB/s
[2018-03-01T13:24:26.0946 CET] [info] SLE-15-x86_64-Build489.1-sled-gnome+development.qcow2: Processing chunk 1786/2117 avg speed ~976.562KB/s
[2018-03-01T13:24:27.0217 CET] [info] SLE-15-x86_64-Build489.1-sled-gnome+development.qcow2: Processing chunk 1787/2117 avg speed ~976.562KB/s
[2018-03-01T13:24:27.0433 CET] [info] SLE-15-x86_64-Build489.1-sled-gnome+development.qcow2: Processing chunk 1788/2117 avg speed ~976.562KB/s
[2018-03-01T13:24:27.0714 CET] [info] SLE-15-x86_64-Build489.1-sled-gnome+development.qcow2: Processing chunk 1789/2117 avg speed ~976.562KB/s
[2018-03-01T13:24:27.0903 CET] [info] SLE-15-x86_64-Build489.1-sled-gnome+development.qcow2: Processing chunk 1790/2117 avg speed ~976.562KB/s
[2018-03-01T13:24:28.0145 CET] [info] SLE-15-x86_64-Build489.1-sled-gnome+development.qcow2: Processing chunk 1791/2117 avg speed ~976.562KB/s
[2018-03-01T13:24:28.0371 CET] [info] SLE-15-x86_64-Build489.1-sled-gnome+development.qcow2: Processing chunk 1792/2117 avg speed ~976.562KB/s
[2018-03-01T13:24:28.0629 CET] [info] SLE-15-x86_64-Build489.1-sled-gnome+development.qcow2: Processing chunk 1793/2117 avg speed ~976.562KB/s
[2018-03-01T13:24:28.0841 CET] [info] SLE-15-x86_64-Build489.1-sled-gnome+development.qcow2: Processing chunk 1794/2117 avg speed ~976.562KB/s
[2018-03-01T13:24:29.0110 CET] [info] SLE-15-x86_64-Build489.1-sled-gnome+development.qcow2: Processing chunk 1795/2117 avg speed ~976.562KB/s
[2018-03-01T13:26:29.0612 CET] [error] Failed uploading chunk
[2018-03-01T13:26:29.0612 CET] [error] ERROR SLE-15-x86_64-Build489.1-sled-gnome+development.qcow2: 502 response: Proxy Error

The chunk where this happens always differ so I guess with enough luck we can at least create some HDDs.