Project

General

Profile

Actions

action #75256

closed

coordination #64746: [saga][epic] Scale up: Efficient handling of large storage to be able to run current tests efficiently but keep big archives of old results

Try out AV1 video codec as potential new default

Added by okurz about 4 years ago. Updated almost 4 years ago.

Status:
Resolved
Priority:
Low
Assignee:
Category:
Feature requests
Target version:
Start date:
2020-10-25
Due date:
% Done:

0%

Estimated time:

Description

Motivation

According to
https://en.m.wikipedia.org/wiki/AV1#Software_implementations
AV1 seems to have good support by now in current decoders, browsers and encoders. We should explore use of the codec as potential new default to make efficient use of storage space considering encoding performance and time impact

Acceptance criteria

  • AC1: AV1 is used as new default video encoder where supported in os-autoinst or we researched why AV1 is not suitable for our purposes
  • AC1: Older products running os-autoinst potentially not supporting AV1 still run the old or an alternative as fallback

Suggestions

  • Research how AV1 can be supported for us, e.g. by using ffmpeg as external video encoder as supported by #67342
  • Either replace the internal encoder or use an AV1 capable one as "external" default (and switch of internal in this case)
    • Note that the internal encoder is also responsible for producing the PNG for the live view so it can not be replaced completely.
  • Check if AV1 provides better performance for our needs than the old internal encoder
  • Consider to also compare against its predecessor VP9
  • Consider fallback handling for older products potentially not supporting AV1
  • Also see suggestions in #68923

Related issues 2 (0 open2 closed)

Related to openQA Infrastructure - coordination #68923: [epic] Use external videoencoder in production auto_review:"External encoder not accepting data"Resolvedokurz2020-11-13

Actions
Copied to openQA Project - action #129955: Second attempt to try out AV1 video codec as potential new default as of 2023 size:MResolvedmkittler

Actions
Actions #1

Updated by okurz about 4 years ago

  • Tracker changed from coordination to action
Actions #2

Updated by okurz about 4 years ago

  • Description updated (diff)
  • Status changed from New to Workable
  • Priority changed from Normal to Low
Actions #3

Updated by mkittler about 4 years ago

  • Related to coordination #68923: [epic] Use external videoencoder in production auto_review:"External encoder not accepting data" added
Actions #4

Updated by mkittler about 4 years ago

  • Description updated (diff)
  • Assignee set to mkittler

I assume this ticket is about playing around with it locally (e.g. to determine encoder flags which make sense for us). I've already created a ticket to use the external video encoder in production (#68923). It suggests using VP9 but AV1 would also be an option of course - at least if we update the workers (under Leap 15.1 there was still only ffmpeg<4 so AV1 wasn't an option).

Check if AV1 provides better performance for our needs than the old internal encoder

Most likely the new encoder will utilize the CPU more. That was at least the case with VP9 and since AV1 is even more sophisticated I wouldn't expect it to be any different. So we'll actually have to take care not to overload our worker machines.

Actions #5

Updated by mkittler about 4 years ago

Seems like even under Leap 15.2 there's only ffmpeg<4 (ffmpeg>=4 would be required for AV1). Strangely, the avconv libraries would be available but not the ffmpeg executable itself. So unless we've create custom packages or use packman repos we can not use AV1 on production workers.

Actions #6

Updated by mkittler about 4 years ago

Actions #7

Updated by okurz about 4 years ago

Yes, I was aware of the VP9 relation. Three link already existed to the parent ticket and the other already resolved sibling about external encoder, but your ticket link might be even more explicit. I was confident that by now openSUSE Tumbleweed had AV1 support but wasn't sure about Leap. I think you already did a very good analysis. Maybe it's a good idea if we just motivate for the necessary bits to be submitted and/or updated, same as we do for other dependencies. Maybe we don't need the ffmpeg CLI when we can use the libraries directly? Also, if it works for Tumbleweed then we already achieved the most important thing, the rest comes "for free" by waiting for any new Leap or by us submitting into the projects we need.

Actions #8

Updated by mkittler about 4 years ago

Maybe it's a good idea if we just motivate for the necessary bits to be submitted and/or updated, same as we do for other dependencies.

Doing a little bit of packaging work would certainly also not be too much to do from our side. Thinking about it further, we would likely end up with packages in openSUSE:Factory but still need to link these in our Leap 15.2 subproject. AV1 is still kind of WIP and we likely want to catch up with upstream changes faster than Leap does.

Maybe we don't need the ffmpeg CLI when we can use the libraries directly?

I've already been working with the ffmpeg libraries a little bit in the past. It is definitely possible but it will also be a lot of work. There's a reason why libraries which are build on top of ffmpeg's libraries like libmpv and qtav exist. Of course we could also use such a high-level library. But I'm not sure whether these libraries would cover our use-case¹ and even then it would be a much less generic approach. For instance, if we decide to switch to another encoder we'd needed to do some C/C++ coding again instead of simply changing some CLI flags.


¹ We need not only decoding but also encoding and a color space conversion (ffmpeg's -pix_fmt yuv420p flag).

Actions #9

Updated by mkittler about 4 years ago

I've just tested with libaom. One can use almost the same parameters as with libvpx. For testing I've been running a test performing the typical graphical installation (ext4 test for TW build) and the following parameters:

EXTERNAL_VIDEO_ENCODER_CMD=ffmpeg -y -hide_banner -nostats -r 24 -f image2pipe -vcodec ppm -i - -pix_fmt yuv420p -c:v libaom-av1 -crf 35 -b:v 1500k -cpu-used 0

The resulting video can be played by Firefox and Chromium as provided by TW. It has an average bitrate of 127 kb/s (1.08 MiB) and the quality is acceptable. However, the encoder is very slow. On my system it took a while after the actual test has been finished until ffmpeg was done. The last line within the os-autoinst log in this state is [debug] Passing remaining frames to the video encoder unless one omits the -nostats ffmpeg flag. So -cpu-used 0 was definitely too much. I conducted another test run with -cpu-used 1 which is actually the default setting. It results in a file with 140 kb/s (1.21 MiB). However, it still takes a while until ffmpeg is done. I could try with even higher -cpu-used settings because that's definitely not fast enough right now. Maybe rav1e or SVT-1 are faster as well so that's what I would try next.

For comparison: So far I've been using libvpx locally to encode VP9 with the same parameters (still using -cpu-used 0 here; of course the encoder parameter was -c:v libvpx-vp9). The resulting VP9 videos have approximately the same quality and the average bitrate ranges from ~80 kb/s to ~350 kb/s depending on the content. In case of the ext4 test VP9 needs an average bitrate of 335 kb/s (2.96 MiB). By the way, with the Theora encoder we're using by default the same job needs 2354 kb/s (20.2 MiB, not accounting the additional muxing overhead of Ogg compared to Matroska).

The sizes in brackets are the total file sizes to get an idea what the difference in bitrate actually means. Note that the tests duration slightly differs between the multiple test runs so the bitrate is more accurate for comparison.

Actions #10

Updated by okurz about 4 years ago

Can you state how long the encoding process takes or the frames per second processed?

Actions #11

Updated by mkittler about 4 years ago

With -cpu-used 1 ffmpeg utilizes up to 15 % of my CPU (which has 8 cores/threads). When there are not a lot of screen changes the CPU utilization is lower and if there are no screen changes it goes even down to 0 %. The FPS depend similarly on how fast the screen changes. I saw FPS values from 1.3 to 0 and 0.8 on average. The default screenshot interval is 0.5 s so we needed 2 FPS to encode in realtime. The fact that there are periods with 0 FPS shows that the backend process is blocked at certain times and doesn't pass any frames to the encoder - although there were very likely frames to be sent in the buffer. The backend's memory utilization also speaks for that. It went up to almost 2 GiB during the test run (just the backend process; not QEMU or anything else).

Actions #12

Updated by mkittler about 4 years ago

By the way, I'm keeping the ticket mainly as side task as it is not urgent anyways. Considering the current state of av1 encoders it is a good idea to wait a little bit longer anyways (e.g. only the next ffmpeg release will support SVT-1 without custom patches).

Actions #13

Updated by mkittler about 4 years ago

I've just tried ffmpeg … -pix_fmt yuv420p -c:v librav1e -b:v 1500k -rav1e-params speed=6 and got 1.3 FPS (it starts faster but 1.3 FPS is what I've eventually ended up with). The parameter speed=6 is rav1e's equivalent to -cpu_used from VP9/libaom (and 6 is actually the default setting). The resulting bitrate was unfortunately higher compared to VP9. Without -b:v 1500k and speed=8 I get 1.6 FPS and 41.0 kb/s. That is lower than my previous libaom encodings and the quality is still acceptable. However, I've just encoded until the bootloader screen today and the bitrate would likely be also higher for the full installation. But regardless of these details I would conclude for now that this the encoder is not significantly faster/better than libaom.

Btw, it spawns multiple threads (the thread pool size is configurable via e.g. threads=8) but still ends up using only one of them. At least that's the case on my machine using librav1e as provided by TW.

Actions #14

Updated by okurz almost 4 years ago

  • Status changed from Workable to Resolved

Great to see that you tried out rav1e this thoroughly. I would have expected that AV1 is in a more usable state by now but this is exactly what we need to find out. As https://trac.ffmpeg.org/wiki/Encode/AV1 also states that this is what we should expect and nothing better I think we can go with VP9 for now. So, with these conclusions we can resolve the ticket. (Reopen if you think there is more that can be done)

Actions #15

Updated by okurz over 1 year ago

  • Copied to action #129955: Second attempt to try out AV1 video codec as potential new default as of 2023 size:M added
Actions

Also available in: Atom PDF