Project

General

Profile

Actions

action #77845

closed

openQA Project - coordination #92323: [saga][epic] Scale up: Fine-grained control over use and removal of results, assets, test data

Use external videoencoder in production on all osd machines size:M

Added by okurz over 3 years ago. Updated almost 1 year ago.

Status:
Resolved
Priority:
Low
Assignee:
Category:
-
Target version:
Start date:
2020-11-13
Due date:
% Done:

0%

Estimated time:
Tags:

Description

Motivation

See #68923

Acceptance criteria

  • AC1: All osd workers run the same external video encoder configured (exceptions allowed if required and documented) as in #77839
  • AC2: All osd workers are not overstressed by the external video encoder, i.e. not more jobs failing or incompleting due to an overstressed worker

Suggestions

  • Research what has been done in #77839 and #77842
  • Ensure ffmpeg is installed on all worker machines with salt
  • Configure external video encoder as documented in #68923#note-17 but with salt
  • Do a step-wise roll-out, e.g. based on salt roles
  • Monitor the CPU usage of ffmpeg e.g. via htop and also the overall performance of each worker, e.g. check for incomplete jobs due to performance issues
  • Wait for feedback for some days

Related issues 2 (0 open2 closed)

Copied from openQA Infrastructure - action #77842: Use external videoencoder in production on all o3 machines size:MResolvedmkittler2020-11-13

Actions
Copied to openQA Infrastructure - action #77848: No more workarounds in OSD due to inefficient video encoder size:MResolvedokurz2020-11-13

Actions
Actions #1

Updated by okurz over 3 years ago

  • Copied from action #77842: Use external videoencoder in production on all o3 machines size:M added
Actions #2

Updated by okurz over 3 years ago

  • Copied to action #77848: No more workarounds in OSD due to inefficient video encoder size:M added
Actions #3

Updated by okurz over 3 years ago

  • Tracker changed from coordination to action
  • Priority changed from Normal to Low
Actions #4

Updated by okurz about 3 years ago

  • Parent task changed from #68923 to #92323
Actions #5

Updated by okurz over 1 year ago

  • Tags set to infra
Actions #6

Updated by okurz about 1 year ago

  • Target version changed from future to Ready
Actions #7

Updated by okurz about 1 year ago

  • Subject changed from Use external videoencoder in production on all osd machines to Use external videoencoder in production on all osd machines size:M
  • Description updated (diff)
  • Status changed from New to Workable
Actions #8

Updated by okurz about 1 year ago

  • Status changed from Workable to Blocked
  • Assignee set to okurz

First #129955

Actions #9

Updated by okurz about 1 year ago

  • Due date set to 2023-07-06
  • Status changed from Blocked to Feedback

#129955 has progressed to the point where we can enable the external video encoder within OSD:
https://gitlab.suse.de/openqa/salt-states-openqa/-/merge_requests/892 marked as draft

But for now I am just picking one worker to see the effect:

sudo salt --no-color '*worker16*' cmd.run 'zypper -n in ffmpeg-4'

now let's wait for some days to see the effect on openqaworker16:1

Actions #10

Updated by okurz about 1 year ago

https://openqa.suse.de/tests/11424251/video?filename=video.webm shows a VP9 video generated so all is good on that machine. In comparison worker13 is still producing ogv https://openqa.suse.de/tests/11424476/video?filename=video.ogv . I marked https://gitlab.suse.de/openqa/salt-states-openqa/-/merge_requests/892 as ready now.

Actions #12

Updated by okurz almost 1 year ago

  • Due date deleted (2023-07-06)
  • Status changed from Feedback to Resolved

On OSD in /var/lib/openqa/testresults I called timeout 600 nice ionice -c 3 find -name '*.webm' and found an endless list of videos so seems workers. I was also checking the load of worker13 as an example and found no significant difference in the periods of ogv or webm/VP9 encoding. So both ACs covered, we are good.

Actions

Also available in: Atom PDF