action #76987
closedopenQA Project - coordination #64746: [saga][epic] Scale up: Efficient handling of large storage to be able to run current tests efficiently but keep big archives of old results
re-encode some videos from existing results to save space
0%
Description
mkittler suggested:
"If we use av1 with the ffmpeg command mentioned in the ticket it will take forever to see any improvement. It is just very slow. So we could use the ffmpeg command for VP9:
ffmpeg -y -hide_banner -nostats -r 24 -f image2pipe -vcodec ppm -i - -pix_fmt yuv420p -c:v libvpx-vp9 -crf 35 -b:v 1500k -cpu-used 0
Likely we don't need the color space conversion if we encode the existing theora steam."
Updated by okurz over 3 years ago
- Due date set to 2020-11-10
- Status changed from In Progress to Feedback
I picked an arbitrary example on osd: /results/testresults/04941/04941557-sle-15-SP3-Online-E-Staging-x86_64-BuildE.65.1-minimal+base@64bit
But first installed "ffmpeg" on osd. After some minutes the video in the above job seems to have vanished :D
Picking another example /results/testresults/04941/04941850-sle-15-SP3-JeOS-for-kvm-and-xen-x86_64-Build20.107-jeos-filesystem@uefi-virtio-vga and starting:
time ffmpeg -i video.ogv -c:v libvpx-vp9 -crf 35 -b:v 1500k -cpu-used 0 video.webm
this ran with around 1.7fps so quite low but ok for trying to optimize as much as possible. With nice -n 19 ionice -c 3 … -cpu-used 8 I get around 3-5fps.
Thinking of running the following:
find -name 'video.ogv' -exec nice -n 19 ionice -c 3 sh -c 'ffmpeg -i {} -c:v libvpx-vp9 -crf 35 -b:v 1500k -cpu-used 8 ${1%.ogv}.webm && rm {}' _ {} \;
Running it now in a screen session of user "geekotest" on osd. It is obviously not occupying more than one core, could be done in parallel as well.
Updated by okurz over 3 years ago
openqa:/var/lib/openqa/testresults # find -wholename '049*/*/*video.webm' could not find any hits. I guess re-encoding to VP9 in a single process on osd in the background is slower than we clean out older jobs :D
So I call this experiment successfully failed ;)
Updated by okurz over 3 years ago
- Related to coordination #68923: [epic] Use external videoencoder in production auto_review:"External encoder not accepting data" added
Updated by okurz over 3 years ago
- Status changed from Feedback to Resolved
- Parent task set to #64746