QA » openQA Project

Seems like even under Leap 15.2 there's only ffmpeg<4 (ffmpeg>=4 would be required for AV1). Strangely, the avconv libraries would be available but not the ffmpeg executable itself. So unless we've create custom packages or use packman repos we can not use AV1 on production workers.

• SVT-1
  • package only in multimedia:libs (https://build.opensuse.org/package/show/multimedia:libs/SVT-AV1), likely outdated
  • requires a custom ffmpeg build using patches from https://github.com/AOMediaCodec/SVT-AV1 (might not apply to latest ffmpeg release) or from ffmpeg's master branch (don't apply to latest release)
    • better just wait for ffmpeg 4.4
  • installation guide: https://github.com/AOMediaCodec/SVT-AV1#svt-av1-ffmpeg-plugin-installation
  • documentation: https://ffmpeg.org/ffmpeg-codecs.html#libsvtav1
• libaom
  • available in ffmpeg-4 as provided by TW/packman
  • documentation: https://trac.ffmpeg.org/wiki/Encode/AV1 and https://ffmpeg.org/ffmpeg-codecs.html#libaom_002dav1
• rav1e
  • available in ffmpeg-4 as provided by TW/packman
  • documentation: https://ffmpeg.org/ffmpeg-codecs.html#librav1e
• libgav1
  • package only in multimedia:libs (https://build.opensuse.org/package/show/multimedia:libs/libgav1)
  • not supported by ffmpeg at this point
• dav1d
  • only decoder, so not really relevant here
  • available in ffmpeg-4 as provided by TW/packman
  • documentation: https://ffmpeg.org/ffmpeg-codecs.html#libdav1d

Yes, I was aware of the VP9 relation. Three link already existed to the parent ticket and the other already resolved sibling about external encoder, but your ticket link might be even more explicit. I was confident that by now openSUSE Tumbleweed had AV1 support but wasn't sure about Leap. I think you already did a very good analysis. Maybe it's a good idea if we just motivate for the necessary bits to be submitted and/or updated, same as we do for other dependencies. Maybe we don't need the ffmpeg CLI when we can use the libraries directly? Also, if it works for Tumbleweed then we already achieved the most important thing, the rest comes "for free" by waiting for any new Leap or by us submitting into the projects we need.

Maybe it's a good idea if we just motivate for the necessary bits to be submitted and/or updated, same as we do for other dependencies.

Doing a little bit of packaging work would certainly also not be too much to do from our side. Thinking about it further, we would likely end up with packages in openSUSE:Factory but still need to link these in our Leap 15.2 subproject. AV1 is still kind of WIP and we likely want to catch up with upstream changes faster than Leap does.

Maybe we don't need the ffmpeg CLI when we can use the libraries directly?

I've already been working with the ffmpeg libraries a little bit in the past. It is definitely possible but it will also be a lot of work. There's a reason why libraries which are build on top of ffmpeg's libraries like libmpv and qtav exist. Of course we could also use such a high-level library. But I'm not sure whether these libraries would cover our use-case¹ and even then it would be a much less generic approach. For instance, if we decide to switch to another encoder we'd needed to do some C/C++ coding again instead of simply changing some CLI flags.

¹ We need not only decoding but also encoding and a color space conversion (ffmpeg's -pix_fmt yuv420p flag).

I've just tested with libaom. One can use almost the same parameters as with libvpx. For testing I've been running a test performing the typical graphical installation (ext4 test for TW build) and the following parameters:

EXTERNAL_VIDEO_ENCODER_CMD=ffmpeg -y -hide_banner -nostats -r 24 -f image2pipe -vcodec ppm -i - -pix_fmt yuv420p -c:v libaom-av1 -crf 35 -b:v 1500k -cpu-used 0

The resulting video can be played by Firefox and Chromium as provided by TW. It has an average bitrate of 127 kb/s (1.08 MiB) and the quality is acceptable. However, the encoder is very slow. On my system it took a while after the actual test has been finished until ffmpeg was done. The last line within the os-autoinst log in this state is [debug] Passing remaining frames to the video encoder unless one omits the -nostats ffmpeg flag. So -cpu-used 0 was definitely too much. I conducted another test run with -cpu-used 1 which is actually the default setting. It results in a file with 140 kb/s (1.21 MiB). However, it still takes a while until ffmpeg is done. I could try with even higher -cpu-used settings because that's definitely not fast enough right now. Maybe rav1e or SVT-1 are faster as well so that's what I would try next.

For comparison: So far I've been using libvpx locally to encode VP9 with the same parameters (still using -cpu-used 0 here; of course the encoder parameter was -c:v libvpx-vp9). The resulting VP9 videos have approximately the same quality and the average bitrate ranges from ~80 kb/s to ~350 kb/s depending on the content. In case of the ext4 test VP9 needs an average bitrate of 335 kb/s (2.96 MiB). By the way, with the Theora encoder we're using by default the same job needs 2354 kb/s (20.2 MiB, not accounting the additional muxing overhead of Ogg compared to Matroska).

The sizes in brackets are the total file sizes to get an idea what the difference in bitrate actually means. Note that the tests duration slightly differs between the multiple test runs so the bitrate is more accurate for comparison.

Can you state how long the encoding process takes or the frames per second processed?

With -cpu-used 1 ffmpeg utilizes up to 15 % of my CPU (which has 8 cores/threads). When there are not a lot of screen changes the CPU utilization is lower and if there are no screen changes it goes even down to 0 %. The FPS depend similarly on how fast the screen changes. I saw FPS values from 1.3 to 0 and 0.8 on average. The default screenshot interval is 0.5 s so we needed 2 FPS to encode in realtime. The fact that there are periods with 0 FPS shows that the backend process is blocked at certain times and doesn't pass any frames to the encoder - although there were very likely frames to be sent in the buffer. The backend's memory utilization also speaks for that. It went up to almost 2 GiB during the test run (just the backend process; not QEMU or anything else).

By the way, I'm keeping the ticket mainly as side task as it is not urgent anyways. Considering the current state of av1 encoders it is a good idea to wait a little bit longer anyways (e.g. only the next ffmpeg release will support SVT-1 without custom patches).

I've just tried ffmpeg … -pix_fmt yuv420p -c:v librav1e -b:v 1500k -rav1e-params speed=6 and got 1.3 FPS (it starts faster but 1.3 FPS is what I've eventually ended up with). The parameter speed=6 is rav1e's equivalent to -cpu_used from VP9/libaom (and 6 is actually the default setting). The resulting bitrate was unfortunately higher compared to VP9. Without -b:v 1500k and speed=8 I get 1.6 FPS and 41.0 kb/s. That is lower than my previous libaom encodings and the quality is still acceptable. However, I've just encoded until the bootloader screen today and the bitrate would likely be also higher for the full installation. But regardless of these details I would conclude for now that this the encoder is not significantly faster/better than libaom.

Btw, it spawns multiple threads (the thread pool size is configurable via e.g. threads=8) but still ends up using only one of them. At least that's the case on my machine using librav1e as provided by TW.