https://progress.opensuse.org/https://progress.opensuse.org/themes/openSUSE/favicon/favicon.ico?15829177842020-10-25T16:52:42ZopenSUSE Project Management ToolopenQA Project - action #75256: Try out AV1 video codec as potential new defaulthttps://progress.opensuse.org/issues/75256?journal_id=3426822020-10-25T16:52:42Zokurzokurz@suse.com
<ul><li><strong>Tracker</strong> changed from <i>coordination</i> to <i>action</i></li></ul> openQA Project - action #75256: Try out AV1 video codec as potential new defaulthttps://progress.opensuse.org/issues/75256?journal_id=3428382020-10-26T08:34:12Zokurzokurz@suse.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/342838/diff?detail_id=338365">diff</a>)</li><li><strong>Status</strong> changed from <i>New</i> to <i>Workable</i></li><li><strong>Priority</strong> changed from <i>Normal</i> to <i>Low</i></li></ul> openQA Project - action #75256: Try out AV1 video codec as potential new defaulthttps://progress.opensuse.org/issues/75256?journal_id=3429402020-10-26T10:29:28Zmkittlermarius.kittler@suse.com
<ul><li><strong>Related to</strong> <i><a class="issue tracker-6 status-3 priority-4 priority-default closed child parent" href="/issues/68923">coordination #68923</a>: [epic] Use external videoencoder in production auto_review:"External encoder not accepting data"</i> added</li></ul> openQA Project - action #75256: Try out AV1 video codec as potential new defaulthttps://progress.opensuse.org/issues/75256?journal_id=3429462020-10-26T10:43:35Zmkittlermarius.kittler@suse.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/342946/diff?detail_id=338488">diff</a>)</li><li><strong>Assignee</strong> set to <i>mkittler</i></li></ul><p>I assume this ticket is about playing around with it locally (e.g. to determine encoder flags which make sense for us). I've already created a ticket to use the external video encoder in production (<a class="issue tracker-6 status-3 priority-4 priority-default closed child parent" title="coordination: [epic] Use external videoencoder in production auto_review:"External encoder not accepting data" (Resolved)" href="https://progress.opensuse.org/issues/68923">#68923</a>). It suggests using VP9 but AV1 would also be an option of course - at least if we update the workers (under Leap 15.1 there was still only ffmpeg<4 so AV1 wasn't an option).</p>
<blockquote>
<p>Check if AV1 provides better performance for our needs than the old internal encoder</p>
</blockquote>
<p>Most likely the new encoder will utilize the CPU more. That was at least the case with VP9 and since AV1 is even more sophisticated I wouldn't expect it to be any different. So we'll actually have to take care not to overload our worker machines.</p>
openQA Project - action #75256: Try out AV1 video codec as potential new defaulthttps://progress.opensuse.org/issues/75256?journal_id=3429492020-10-26T10:53:02Zmkittlermarius.kittler@suse.com
<ul></ul><p>Seems like even under Leap 15.2 there's only <code>ffmpeg<4</code> (<code>ffmpeg>=4</code> would be required for AV1). Strangely, the avconv libraries would be available but not the ffmpeg executable itself. So unless we've create custom packages or use <a href="http://packman.links2linux.com/package/ffmpeg-4" class="external">packman repos</a> we can not use AV1 on production workers.</p>
openQA Project - action #75256: Try out AV1 video codec as potential new defaulthttps://progress.opensuse.org/issues/75256?journal_id=3429762020-10-26T11:49:31Zmkittlermarius.kittler@suse.com
<ul></ul><ul>
<li>SVT-1
<ul>
<li>package only in <code>multimedia:libs</code> (<a href="https://build.opensuse.org/package/show/multimedia:libs/SVT-AV1" class="external">https://build.opensuse.org/package/show/multimedia:libs/SVT-AV1</a>), likely outdated</li>
<li>requires a custom ffmpeg build using patches from <a href="https://github.com/AOMediaCodec/SVT-AV1" class="external">https://github.com/AOMediaCodec/SVT-AV1</a> (might not apply to latest ffmpeg release) or from ffmpeg's master branch (don't apply to latest release)
<ul>
<li>better just wait for ffmpeg 4.4</li>
</ul></li>
<li>installation guide: <a href="https://github.com/AOMediaCodec/SVT-AV1#svt-av1-ffmpeg-plugin-installation" class="external">https://github.com/AOMediaCodec/SVT-AV1#svt-av1-ffmpeg-plugin-installation</a></li>
<li>documentation: <a href="https://ffmpeg.org/ffmpeg-codecs.html#libsvtav1" class="external">https://ffmpeg.org/ffmpeg-codecs.html#libsvtav1</a></li>
</ul></li>
<li>libaom
<ul>
<li>available in ffmpeg-4 as provided by TW/packman</li>
<li>documentation: <a href="https://trac.ffmpeg.org/wiki/Encode/AV1" class="external">https://trac.ffmpeg.org/wiki/Encode/AV1</a> and <a href="https://ffmpeg.org/ffmpeg-codecs.html#libaom_002dav1" class="external">https://ffmpeg.org/ffmpeg-codecs.html#libaom_002dav1</a></li>
</ul></li>
<li>rav1e
<ul>
<li>available in ffmpeg-4 as provided by TW/packman</li>
<li>documentation: <a href="https://ffmpeg.org/ffmpeg-codecs.html#librav1e" class="external">https://ffmpeg.org/ffmpeg-codecs.html#librav1e</a></li>
</ul></li>
<li>libgav1
<ul>
<li>package only in <code>multimedia:libs</code> (<a href="https://build.opensuse.org/package/show/multimedia:libs/libgav1" class="external">https://build.opensuse.org/package/show/multimedia:libs/libgav1</a>)</li>
<li>not supported by ffmpeg at this point</li>
</ul></li>
<li>dav1d
<ul>
<li><strong>only decoder, so not really relevant here</strong></li>
<li>available in ffmpeg-4 as provided by TW/packman</li>
<li>documentation: <a href="https://ffmpeg.org/ffmpeg-codecs.html#libdav1d" class="external">https://ffmpeg.org/ffmpeg-codecs.html#libdav1d</a></li>
</ul></li>
</ul>
openQA Project - action #75256: Try out AV1 video codec as potential new defaulthttps://progress.opensuse.org/issues/75256?journal_id=3430482020-10-26T12:15:33Zokurzokurz@suse.com
<ul></ul><p>Yes, I was aware of the VP9 relation. Three link already existed to the parent ticket and the other already resolved sibling about external encoder, but your ticket link might be even more explicit. I was confident that by now openSUSE Tumbleweed had AV1 support but wasn't sure about Leap. I think you already did a very good analysis. Maybe it's a good idea if we just motivate for the necessary bits to be submitted and/or updated, same as we do for other dependencies. Maybe we don't need the ffmpeg CLI when we can use the libraries directly? Also, if it works for Tumbleweed then we already achieved the most important thing, the rest comes "for free" by waiting for any new Leap or by us submitting into the projects we need.</p>
openQA Project - action #75256: Try out AV1 video codec as potential new defaulthttps://progress.opensuse.org/issues/75256?journal_id=3431862020-10-26T12:50:55Zmkittlermarius.kittler@suse.com
<ul></ul><blockquote>
<p>Maybe it's a good idea if we just motivate for the necessary bits to be submitted and/or updated, same as we do for other dependencies.</p>
</blockquote>
<p>Doing a little bit of packaging work would certainly also not be too much to do from our side. Thinking about it further, we would likely end up with packages in openSUSE:Factory but still need to link these in our Leap 15.2 subproject. AV1 is still kind of WIP and we likely want to catch up with upstream changes faster than Leap does.</p>
<blockquote>
<p>Maybe we don't need the ffmpeg CLI when we can use the libraries directly?</p>
</blockquote>
<p>I've already been working with the ffmpeg libraries a little bit in the past. It is definitely possible but it will also be a lot of work. There's a reason why libraries which are build on top of ffmpeg's libraries like <code>libmpv</code> and <code>qtav</code> exist. Of course we could also use such a high-level library. But I'm not sure whether these libraries would cover our use-case¹ and even then it would be a much less generic approach. For instance, if we decide to switch to another encoder we'd needed to do some C/C++ coding again instead of simply changing some CLI flags.</p>
<hr>
<p>¹ We need not only decoding but also encoding and a color space conversion (ffmpeg's <code>-pix_fmt yuv420p</code> flag).</p>
openQA Project - action #75256: Try out AV1 video codec as potential new defaulthttps://progress.opensuse.org/issues/75256?journal_id=3441852020-10-27T16:58:03Zmkittlermarius.kittler@suse.com
<ul></ul><p>I've just tested with libaom. One can use almost the same parameters as with libvpx. For testing I've been running a test performing the typical graphical installation (ext4 test for TW build) and the following parameters:</p>
<pre><code>EXTERNAL_VIDEO_ENCODER_CMD=ffmpeg -y -hide_banner -nostats -r 24 -f image2pipe -vcodec ppm -i - -pix_fmt yuv420p -c:v libaom-av1 -crf 35 -b:v 1500k -cpu-used 0
</code></pre>
<p>The resulting video can be played by Firefox and Chromium as provided by TW. It has an average bitrate of 127 kb/s (1.08 MiB) and the quality is acceptable. However, the encoder is very slow. On my system it took a while after the actual test has been finished until ffmpeg was done. The last line within the os-autoinst log in this state is <code>[debug] Passing remaining frames to the video encoder</code> unless one omits the <code>-nostats</code> ffmpeg flag. So <code>-cpu-used 0</code> was definitely too much. I conducted another test run with <code>-cpu-used 1</code> which is actually the default setting. It results in a file with 140 kb/s (1.21 MiB). However, it still takes a while until ffmpeg is done. I could try with even higher <code>-cpu-used</code> settings because that's definitely not fast enough right now. Maybe rav1e or SVT-1 are faster as well so that's what I would try next.</p>
<p>For comparison: So far I've been using libvpx locally to encode VP9 with the same parameters (still using <code>-cpu-used 0</code> here; of course the encoder parameter was <code>-c:v libvpx-vp9</code>). The resulting VP9 videos have approximately the same quality and the average bitrate ranges from ~80 kb/s to ~350 kb/s depending on the content. In case of the ext4 test VP9 needs an average bitrate of 335 kb/s (2.96 MiB). By the way, with the Theora encoder we're using by default the same job needs 2354 kb/s (20.2 MiB, not accounting the additional muxing overhead of Ogg compared to Matroska).</p>
<p>The sizes in brackets are the total file sizes to get an idea what the difference in bitrate actually means. Note that the tests duration slightly differs between the multiple test runs so the bitrate is more accurate for comparison.</p>
openQA Project - action #75256: Try out AV1 video codec as potential new defaulthttps://progress.opensuse.org/issues/75256?journal_id=3441882020-10-27T17:11:59Zokurzokurz@suse.com
<ul></ul><p>Can you state how long the encoding process takes or the frames per second processed?</p>
openQA Project - action #75256: Try out AV1 video codec as potential new defaulthttps://progress.opensuse.org/issues/75256?journal_id=3443472020-10-28T10:31:42Zmkittlermarius.kittler@suse.com
<ul></ul><p>With <code>-cpu-used 1</code> ffmpeg utilizes up to 15 % of my CPU (which has 8 cores/threads). When there are not a lot of screen changes the CPU utilization is lower and if there are no screen changes it goes even down to 0 %. The FPS depend similarly on how fast the screen changes. I saw FPS values from 1.3 to 0 and 0.8 on average. The default screenshot interval is 0.5 s so we needed 2 FPS to encode in realtime. The fact that there are periods with 0 FPS shows that the backend process is blocked at certain times and doesn't pass any frames to the encoder - although there were very likely frames to be sent in the buffer. The backend's memory utilization also speaks for that. It went up to almost 2 GiB during the test run (just the backend process; not QEMU or anything else).</p>
openQA Project - action #75256: Try out AV1 video codec as potential new defaulthttps://progress.opensuse.org/issues/75256?journal_id=3445992020-10-29T09:57:03Zmkittlermarius.kittler@suse.com
<ul></ul><p>By the way, I'm keeping the ticket mainly as side task as it is not urgent anyways. Considering the current state of av1 encoders it is a good idea to wait a little bit longer anyways (e.g. only the next ffmpeg release will support SVT-1 without custom patches).</p>
openQA Project - action #75256: Try out AV1 video codec as potential new defaulthttps://progress.opensuse.org/issues/75256?journal_id=3468642020-11-04T17:29:58Zmkittlermarius.kittler@suse.com
<ul></ul><p>I've just tried <code>ffmpeg … -pix_fmt yuv420p -c:v librav1e -b:v 1500k -rav1e-params speed=6</code> and got 1.3 FPS (it starts faster but 1.3 FPS is what I've eventually ended up with). The parameter <code>speed=6</code> is <code>rav1e</code>'s equivalent to <code>-cpu_used</code> from VP9/libaom (and 6 is actually the default setting). The resulting bitrate was unfortunately higher compared to VP9. Without <code>-b:v 1500k</code> and <code>speed=8</code> I get 1.6 FPS and 41.0 kb/s. That is lower than my previous libaom encodings and the quality is still acceptable. However, I've just encoded until the bootloader screen today and the bitrate would likely be also higher for the full installation. But regardless of these details I would conclude for now that this the encoder is not significantly faster/better than libaom.</p>
<p>Btw, it spawns multiple threads (the thread pool size is configurable via e.g. <code>threads=8</code>) but still ends up using only one of them. At least that's the case on my machine using librav1e as provided by TW.</p>
openQA Project - action #75256: Try out AV1 video codec as potential new defaulthttps://progress.opensuse.org/issues/75256?journal_id=3483252020-11-07T06:04:26Zokurzokurz@suse.com
<ul><li><strong>Status</strong> changed from <i>Workable</i> to <i>Resolved</i></li></ul><p>Great to see that you tried out rav1e this thoroughly. I would have expected that AV1 is in a more usable state by now but this is exactly what we need to find out. As <a href="https://trac.ffmpeg.org/wiki/Encode/AV1" class="external">https://trac.ffmpeg.org/wiki/Encode/AV1</a> also states that this is what we should expect and nothing better I think we can go with VP9 for now. So, with these conclusions we can resolve the ticket. (Reopen if you think there is more that can be done)</p>
openQA Project - action #75256: Try out AV1 video codec as potential new defaulthttps://progress.opensuse.org/issues/75256?journal_id=6369732023-05-28T09:51:11Zokurzokurz@suse.com
<ul><li><strong>Copied to</strong> <i><a class="issue tracker-4 status-3 priority-5 priority-high3 closed child" href="/issues/129955">action #129955</a>: Second attempt to try out AV1 video codec as potential new default as of 2023 size:M</i> added</li></ul>