Project

General

Profile

Actions

action #157636

open

remove NOVIDEO=1 from ppc64le workers

Added by zcjia 9 months ago. Updated 8 months ago.

Status:
New
Priority:
Normal
Assignee:
Category:
-
Target version:
Start date:
2024-03-21
Due date:
% Done:

0%

Estimated time:

Description

https://suse.slack.com/archives/C02CANHLANP/p1710931589680009

As discussed above, there is really no reason to disable video recording of tests running on ppc64le workers.

I have tested that video recording works on a single test.

I plan to remove NOVIDEO=1 from various ppc64le machine types:

ppc64le-kernel-default-base
ppc64le-no-tmpfs
ppc64le-sap
ppc64le-sap-qam
ppc64le-virtio
ppc64le-4g
ppc64le-2g
ppc64le

If there are no objections, I will start with ppc64le-2g and monitor the results.


Related issues 1 (0 open1 closed)

Related to openQA Infrastructure (public) - action #158104: typing issue on ppc64 worker size:SResolvedokurz2024-03-27

Actions
Actions #1

Updated by okurz 9 months ago

  • Target version set to future

Thank you for looking into this. I fully support that change

Actions #2

Updated by okurz 9 months ago

  • Related to action #158104: typing issue on ppc64 worker size:S added
Actions #3

Updated by zcjia 9 months ago

I have revert back to "NOVIDEO=1" for ppc64le-2g, until we figure out why ffmpeg is taking so much time and cpu.

In the mean while, the tests triggered in build 71.1 and build 73.1 will continue to have "NOVIDEO=0" in their settings.

Actions #4

Updated by zcjia 9 months ago

The issue is in ffmpeg. The ffmpeg from system is over 100 times slower than ffmpeg-4.4.4 build from source code.

Will investigate deeper.

Actions #5

Updated by mkittler 9 months ago

Interesting. Keep in mind that ffmpeg is only used when it also supports any of the video encoder configurations os-autoinst is able to utilize. That means if your ffmpeg build doesn't support VP9 and doesn't support SVT-AV1 then it would simply not be used at all. Then the fallback video encoder would be used instead and that one is not using much CPU time (hence it would appear faster). (I guess SVT-AV1 is not relevant on PowerPC anyway; the system-provided ffmpeg also doesn't support it. It supports VP9, though. So that will be used. I just had a look at htop and it actually looks like the fallback video encoder is used nevertheless. Is that something you are currently experimenting with?)

Actions #6

Updated by zcjia 9 months ago

I was comparing speed by simply re-encoding the video.webm:

ffmpeg -i video.webm video.mkv

I double checked that system ffmpeg uses VP9 codec while my compiled ffmpeg uses MPEG4.

I checked the "vpenc", on ppc64 it's 30 times slower than my amd64 laptop.

Actions #7

Updated by mkittler 9 months ago

I double checked that system ffmpeg uses VP9 codec while my compiled ffmpeg uses MPEG4.

MPEG4 is a bunch of standards. What encoder did you/ffmpeg actually use? Note that ffmpeg might have even just re-muxed the file (under certain conditions). Note that ffmpeg will log what codec it uses and you can also specify one explicitly. It probably doesn't matter, though - we probably want to stick with royalty-free formats (so we wouldn't want to use any of the MPEG4 formats anyway).

I checked the "vpenc", on ppc64 it's 30 times slower than my amd64 laptop.

Unfortunately that's no surprise. Probably this counts for many highly optimized codecs as they are only well optimized for x86_64 (and maybe ARM to some extend).

I was comparing speed by simply re-encoding the video.webm

So you haven't changed any settings or installed packages on any of the relevant the workers? If that is true this means together with my last observation ("I just had a look at htop and it actually looks like the fallback video encoder is used nevertheless.") that at least on petrol ffmpeg is not used anyway. That would mean that even the fallback video encoder (libtheora) is very slow/problematic on PowerPC (even though it is normally not very CPU intensive by today's standards).

Actions #8

Updated by zcjia 8 months ago

Tested with Tumbleweed and Fedora on ppc64, the ffmpeg-vp9 performance is as bad as leap15.5.

Actions

Also available in: Atom PDF