Project

General

Profile

Actions

action #77839

closed

openQA Project - coordination #103938: [saga][epic] Scale up: Efficient handling of large storage on o3

coordination #68923: [epic] Use external videoencoder in production auto_review:"External encoder not accepting data"

Use external videoencoder in production on one o3 worker

Added by okurz over 3 years ago. Updated over 3 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
-
Target version:
Start date:
2020-11-13
Due date:
% Done:

0%

Estimated time:

Description

Motivation

See #68923

Acceptance criteria

  • AC1: One worker runs with an external video encoder configured
  • AC2: The configured external video encoder provides significantly smaller video files instead of the default ones
  • AC3: The worker is not overstressed by the external video encoder, i.e. not more jobs failing or incompleting due to an overstressed worker

Suggestions

  • DONE: Install ffmpeg on the worker -> done on imagetester
  • DONE: Try to find a good encoder setting -> done with VP9 with ffmpeg -hide_banner -nostats -r 24 -f image2pipe -vcodec ppm -i - -pix_fmt yuv420p -c:v libvpx-vp9 -crf 35 -b:v 1500k -cpu-used 0, see https://progress.opensuse.org/issues/68923#note-17
  • DONE: Check whether the resulting videos work (including seeking) -> imagetester works fine, e.g. following https://openqa.opensuse.org/admin/workers/41 we find jobs like https://openqa.opensuse.org/tests/1469967#downloads showing .webm instead of .ogv with size 2.8MB instead of 6.9MB on an older job in the same scenario running on a different machine recorded in .ogv , i.e. only 41% of the original size
  • DONE: Monitor the CPU usage of ffmpeg e.g. via htop and also the overall performance of the worker, e.g. check for incomplete jobs due to performance issues. -> mkittler already did the original analysis and it was ok and no reports by users have been raised

Related issues 1 (0 open1 closed)

Copied to openQA Infrastructure - action #77842: Use external videoencoder in production on all o3 machines size:MResolvedmkittler2020-11-13

Actions
Actions #1

Updated by okurz over 3 years ago

  • Copied to action #77842: Use external videoencoder in production on all o3 machines size:M added
Actions #2

Updated by okurz over 3 years ago

  • Subject changed from Use external videoencoder in production auto_review:"External encoder not accepting data" to Use external videoencoder in production on one o3 worker
  • Status changed from New to Resolved
  • Assignee set to okurz

All done on imagetester, showing no problems and significant reduction in video file size, see updated description of this ticket. Next step: Enable on all other o3 machines -> #77842

Actions #3

Updated by okurz over 3 years ago

  • Tracker changed from coordination to action
Actions

Also available in: Atom PDF