[FFmpeg-user] Clarification for decoding AV1 input on the GPU
Mario Roy
marioeroy at gmail.com
Thu Aug 4 19:52:13 EEST 2022
Hi Haihao,
Congrats on the patchset to FFmpeg. I saw patches df5defe and ad67ea9 were
recently committed and thought to retest FFmpeg (on master branch), using
NVIDIA 3070 RTX.
https://github.com/FFmpeg/FFmpeg/commit/df5defe15a29dfca5f13224eb5da821095f6c745
https://github.com/FFmpeg/FFmpeg/commit/ad67ea9eeed2cc8edab8662b7503ce4a49247b1d
*First, I tried -hwaccel without -c:v.*
ffmpeg -y -hwaccel nvdec -i input_vp9.webm -f null - # decodes on GPU
ffmpeg -y -hwaccel vaapi -i input_vp9.webm -f null - # *segfault*, though
okay with -c:v vp9
ffmpeg -y -hwaccel vdpau -i input_vp9.webm -f null - # decodes on GPU
ffmpeg -y -hwaccel nvdec -i input_av1.mp4 -f null - # decodes on GPU
ffmpeg -y -hwaccel vaapi -i input_av1.mp4 -f null - # decodes on GPU
ffmpeg -y -hwaccel vdpau -i input_av1.mp4 -f null - # decodes on GPU
*Next, I tried incorrect arguments.* Why does FFmpeg not exit gracefully?
ffmpeg -y -hwaccel vaapi -c:v vp9_cuvid -i input_vp9.webm -f null - #
*segfault*
ffmpeg -y -hwaccel vdpau -c:v vp9_cuvid -i input_vp9.webm -f null - #
*segfault*
ffmpeg -y -hwaccel vaapi -c:v av1_cuvid -i input_av1.mp4 -f null - #
*segfault*
ffmpeg -y -hwaccel vdpau -c:v av1_cuvid -i input_av1.mp4 -f null - #
*segfault*
*Then, I compared -hwaccel nvdec with/without -c:v* {vp9_cuvid, av1_cuvid}.
ffmpeg -y -hwaccel nvdec -c:v vp9_cuvid -i input_vp9.webm -f null - #
1436 fps
ffmpeg -y -hwaccel nvdec -i input_vp9.webm -f null - #
1428 fps
ffmpeg -y -hwaccel nvdec -c:v av1_cuvid -i input_av1.mp4 -f null - #
1629 fps
ffmpeg -y -hwaccel nvdec -i input_av1.mp4 -f null - # *895
fps*
I cannot understand why ffmpeg does not automatically choose the cuvid path
for -hwaccel nvdec. There is a noticeable difference for av1.
Best regards,
Mario
On Tue, Jul 26, 2022 at 8:50 PM Mario Roy <marioeroy at gmail.com> wrote:
> Thanks for the explanation regarding cuvid.
>
> Mario
>
>
> On Wed, Jul 20, 2022 at 11:27 PM Xiang, Haihao <
> haihao.xiang-at-intel.com at ffmpeg.org> wrote:
>
>> On Wed, 2022-07-20 at 08:02 -0500, Mario Roy wrote:
>> > Thank you Haihao, for the URL to patchset.
>> >
>> > Previously, I applied only the patch contained in the email thread. This
>> > time, I applied both patches to FFmpeg master and confirmed that
>> > av1(native) is used for nvdec, vaapi, and vdpau. I observed 100% and
>> 50% ~
>> > 68% video engine utilization for VP9 and AV1 respectively. Unix time is
>> > captured for output.ts.
>> >
>> > Video Engine Utilization: 100%
>> >
>> > $ ffmpeg -y -hwaccel nvdec -i input_vp9.webm -f null - # GPU fps=1441
>> > $ ffmpeg -y -hwaccel vaapi -i input_vp9.webm -f null - # GPU fps=1320
>> > $ ffmpeg -y -hwaccel vdpau -i input_vp9.webm -f null - # GPU fps=1308
>> >
>> > Video Engine Utilization: 50% ~ 68%
>> >
>> > $ ffmpeg -y -hwaccel nvdec -i input_av1.mp4 -f null - # GPU fps=900
>> > $ ffmpeg -y -hwaccel vaapi -i input_av1.mp4 -f null - # GPU fps=760
>> > $ ffmpeg -y -hwaccel vdpau -i input_av1.mp4 -f null - # GPU fps=799
>>
>> Glad to see the patchset works for you, I'll resend the patchset to
>> FFmpeg ML.
>>
>> >
>> > Unix Time output.ts:
>> > -hwaccel vdpau is unexpectedly slow for av1
>> >
>> > $ time ffmpeg -y -hwaccel nvdec -i input_vp9.webm output.ts # GPU
>> 6.176s
>> > $ time ffmpeg -y -hwaccel vaapi -i input_vp9.webm output.ts # GPU
>> 7.754s
>> > $ time ffmpeg -y -hwaccel vdpau -i input_vp9.webm output.ts # GPU
>> 7.213s
>> >
>> > $ time ffmpeg -y -hwaccel nvdec -i input_av1.mp4 output.ts # GPU
>> 8.709s
>> > $ time ffmpeg -y -hwaccel vaapi -i input_av1.mp4 output.ts # GPU
>> 10.816s
>> > $ time ffmpeg -y -hwaccel vdpau -i input_av1.mp4 output.ts # GPU
>> 15.162s
>> >
>> > The next two commands core dumps using FFmpeg master, ditto for FFmpeg
>> 5.1.
>> >
>> > $ ffmpeg -y -hwaccel vdpau -c:v vp9_cuvid -i input_vp9.webm -f null - #
>> > core dumped
>> > $ ffmpeg -y -hwaccel vdpau -c:v av1_cuvid -i input_av1.mp4 -f null - #
>> > core dumped
>>
>> I am not familiar with cuvid decoders, but according to
>>
>> https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/cuviddec.c#L1099-L1110,
>>
>> cuvid decoders use CUDA device. I guess you should use -hwaccel cuvid
>> which is
>> remapped to -hwaccel cuda in FFmpeg, see
>>
>>
>> https://github.com/FFmpeg/FFmpeg/blob/master/fftools/ffmpeg_opt.c#L1000-L1001
>>
>> BRs
>> Haihao
>>
>>
>> >
>> > No issues for nvdec. Passing -c:v av1_cuvid is swift, compared to 900
>> fps
>> > above without -c:v av1_cuvid.
>> >
>> > $ ffmpeg -y -hwaccel nvdec -c:v vp9_cuvid -i input_vp9.webm -f null - #
>> > fps=1435
>> > $ ffmpeg -y -hwaccel nvdec -c:v av1_cuvid -i input_av1.mp4 -f null - #
>> > fps=1642
>> >
>> >
>> > Thanks,
>> > Mario
>> >
>> >
>> > On Wed, Jul 20, 2022 at 4:30 AM Xiang, Haihao <
>> > haihao.xiang-at-intel.com at ffmpeg.org> wrote:
>> >
>> > >
>> > > > Thank you, Haihao. I applied the patchset to FFmpeg 5.1 branch.
>> Passing
>> > > > -hwaccel {nvdec|vdpau} without -c:v av1 still decoded on the CPU. I
>> > >
>> > > double
>> > > > checked to be sure the patchset was applied.
>> > > >
>> > >
>> > > libdav1d is still used for format probing but av1(native) should be
>> used
>> > > for
>> > > decoding when -hwaccel xxx is specified.
>> > >
>> > > I rebased the patches on the master(
>> > > https://github.com/xhaihao/FFmpeg/tree/ffmpeg-choose-dec) however I
>> don't
>> > > have
>> > > the environment to verify nvdec/vdpau. I verified it with vaapi
>> > >
>> > > $ ffmpeg -hwaccel vaapi -i av1.ivf -f null -
>> > >
>> > > [...]
>> > > Stream mapping:
>> > > Stream #0:0 -> #0:0 (av1 (native) -> wrapped_avframe (native))
>> > >
>> > > $ ffmpeg -i av1.ivf -f null -
>> > >
>> > > [...]
>> > > Stream mapping:
>> > > Stream #0:0 -> #0:0 (av1 (libdav1d) -> wrapped_avframe (native))
>> > >
>> > > Thanks
>> > > Haihao
>> > >
>> > >
>> > > > I obtained AV1 and VP9 samples via yt-dlp: 1920x1080 resolution.
>> > > >
>> > > > $ yt-dlp -F https://youtu.be/mIAfxj7nd9k
>> > > > $ yt-dlp -f 399 https://youtu.be/mIAfxj7nd9k -o input_av1.mp4
>> > > > $ yt-dlp -f 248 https://youtu.be/mIAfxj7nd9k -o input_vp9.webm
>> > > >
>> > > > Hardware: GeForce RTX 3070 FE
>> > > >
>> > > > AV1 Observations.
>> > > >
>> > > > $ ffmpeg -y -hwaccel nvdec -c:v av1_cuvid -i input_av1.mp4
>> output.ts #
>> > > > GPU fps=653
>> > > > $ ffmpeg -y -hwaccel vdpau -c:v av1_cuvid -i input_av1.mp4
>> output.ts #
>> > > > Core Dumps
>> > > > $ ffmpeg -y -hwaccel nvdec -c:v av1 -i input_av1.mp4 output.ts
>> #
>> > > > GPU fps=454
>> > > > $ ffmpeg -y -hwaccel vdpau -c:v av1 -i input_av1.mp4 output.ts
>> #
>> > > > GPU fps=246
>> > > > $ ffmpeg -y -hwaccel nvdec -i input_av1.mp4 output.ts
>> #
>> > > > CPU libdav1d
>> > > > $ ffmpeg -y -hwaccel vdpau -i input_av1.mp4 output.ts
>> #
>> > > > CPU libdav1d
>> > > >
>> > > > VP9 Observations.
>> > > >
>> > > > $ ffmpeg -y -hwaccel nvdec -c:v vp9_cuvid -i input_vp9.webm
>> output.ts #
>> > > > GPU fps=669
>> > > > $ ffmpeg -y -hwaccel vdpau -c:v vp9_cuvid -i input_vp9.webm
>> output.ts #
>> > > > Core Dumps
>> > > > $ ffmpeg -y -hwaccel nvdec -c:v vp9 -i input_vp9.webm output.ts
>> #
>> > > > GPU fps=662
>> > > > $ ffmpeg -y -hwaccel vdpau -c:v vp9 -i input_vp9.webm output.ts
>> #
>> > > > GPU fps=529
>> > > > $ ffmpeg -y -hwaccel nvdec -i input_vp9.webm output.ts
>> #
>> > > > GPU fps=655
>> > > > $ ffmpeg -y -hwaccel vdpau -i input_vp9.webm output.ts
>> #
>> > > > GPU fps=533
>> > > >
>> > > > Looking at the AV1 results, -hwaccel vdpau -c:v av1 is nearly 2x
>> slower
>> > > > than -hwaccel nvdec -c:v av1.
>> > > >
>> > > > Should FFmpeg exit gracefully if passing unsupported options for
>> -hwaccel
>> > > > vdpau? Or is this a bug?
>> > > > -hwaccel vdpau -c:v av1_cuvid # Crashes
>> > > > -hwaccel vdpau -c:v vp9_cuvid # Crashes
>> > > >
>> > > > Thanks,
>> > > > Mario
>> > > >
>> > > > On Tue, Jul 19, 2022 at 8:42 PM Xiang, Haihao <
>> > > > haihao.xiang-at-intel.com at ffmpeg.org> wrote:
>> > > >
>> > > > > On Tue, 2022-07-19 at 16:54 -0500, Mario Roy wrote:
>> > > > > > Greetings,
>> > > > > >
>> > > > > > With the recent FFmpeg 5.1 supporting AV1 using VDPAU, should
>> > >
>> > > decoding
>> > > > >
>> > > > > AV1
>> > > > > > occur on the GPU automatically (NVIDIA 3000 series), similarly
>> to
>> > > > >
>> > > > > decoding
>> > > > > > VP9?
>> > > > > >
>> > > > > > ffmpeg -y -hwaccel vdpau -i input_av1.mp4 output.ts # decodes
>> on
>> > >
>> > > the
>> > > > >
>> > > > > CPU
>> > > > >
>> > > > >
>> > > > > libdav1d has higher priority than the the native av1 decoder in
>> > >
>> > > FFmpeg, you
>> > > > > should specify the native av1 in your command line if you want to
>> use
>> > > > > vdpau.
>> > > > >
>> > > > > $ ffmpeg -y -hwaccel vdpau -c:v av1 -i input_av1.mp4 output.ts
>> > > > >
>> > > > > Or you may try
>> > > > >
>> > > > >
>> > >
>> > >
>>
>> https://patchwork.ffmpeg.org/project/ffmpeg/patch/20210922074207.19451-2-haihao.xiang@intel.com/
>> > > > > which can decode AV1 on the GPU automatically when -hwaccel xxx
>> is
>> > > > > specified in
>> > > > > the command line. (You need to rebase the patchset).
>> > > > >
>> > > > > Thanks
>> > > > > Haihao
>> > > > >
>> > > > >
>> > > > > > ffmpeg -y -hwaccel vdpau -i input_vp9.webm output.ts # decodes
>> on
>> > >
>> > > the
>> > > > >
>> > > > > GPU
>> > > > > >
>> > > > > > Are the following incorrect usage? Both segfaults.
>> > > > > >
>> > > > > > ffmpeg -y -hwaccel vdpau -c:v av1_cuvid -i input_av1.mp4
>> output.ts
>> > >
>> > > #
>> > > > > > segfaults
>> > > > > > ffmpeg -y -hwaccel vdpau -c:v vp9_cuvid -i input_vp9.webm
>> output.ts
>> > >
>> > > #
>> > > > > > segfaults
>> > > > > >
>> > > > > > Thanks,
>> > > > > > Mario
>> > > > > >
>> > > > > > On Wed, Jul 13, 2022 at 9:44 AM Mario Roy <marioeroy at gmail.com>
>> > >
>> > > wrote:
>> > > > > >
>> > > > > > > Hi,
>> > > > > > >
>> > > > > > > What are the recommended arguments for folks using NVIDIA 3000
>> > >
>> > > series
>> > > > > > > graphics and decoding AV1 media? I tried the same with VP9
>> media.
>> > > > > > >
>> > > > > > > The following was done using FFmpeg 5.1.
>> > > > > > >
>> > > > > > > ffmpeg -y -hwaccel vdpau -c:v av1_cuvid -i input_av1.mp4
>> > > > > > > output.ts segfaults
>> > > > > > > ffmpeg -y -hwaccel vdpau -c:v vp9_cuvid -i input_vp9.webm
>> > > > > > > output.ts segfaults
>> > > > > > >
>> > > > > > > ffmpeg -y -hwaccel vdpau -i input_av1.mp4 output.ts decodes
>> on
>> > >
>> > > the
>> > > > >
>> > > > > CPU
>> > > > > > > ffmpeg -y -hwaccel vdpau -i input_vp9.webm output.ts decodes
>> on
>> > >
>> > > the
>> > > > >
>> > > > > GPU
>> > > > > > >
>> > > > > > > ffmpeg -y -hwaccel nvdec -c:v av1_cuvid -i input_av1.mp4
>> output.ts
>> > > > > > > ffmpeg -y -hwaccel nvdec -c:v vp9_cuvid -i input_vp9.webm
>> output.ts
>> > > > > > >
>> > > > > > > ffmpeg -y -c:v av1_cuvid -i input_av1.mp4 output.ts
>> > > > > > > ffmpeg -y -c:v vp9_cuvid -i input_vp9.webm output.ts
>> > > > > > >
>> > > > > > > Best,
>> > > > > > > Mario
>> > > > > > >
>> > > > > > >
>> > > > > >
>> > > > > > _______________________________________________
>> > > > > > ffmpeg-user mailing list
>> > > > > > ffmpeg-user at ffmpeg.org
>> > > > > > https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>> > > > > >
>> > > > > > To unsubscribe, visit link above, or email
>> > > > > > ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
>> > > > >
>> > > > > _______________________________________________
>> > > > > ffmpeg-user mailing list
>> > > > > ffmpeg-user at ffmpeg.org
>> > > > > https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>> > > > >
>> > > > > To unsubscribe, visit link above, or email
>> > > > > ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
>> > > > >
>> > > >
>> > > > _______________________________________________
>> > > > ffmpeg-user mailing list
>> > > > ffmpeg-user at ffmpeg.org
>> > > > https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>> > > >
>> > > > To unsubscribe, visit link above, or email
>> > > > ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
>> > >
>> > > _______________________________________________
>> > > ffmpeg-user mailing list
>> > > ffmpeg-user at ffmpeg.org
>> > > https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>> > >
>> > > To unsubscribe, visit link above, or email
>> > > ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
>> > >
>> >
>> > _______________________________________________
>> > ffmpeg-user mailing list
>> > ffmpeg-user at ffmpeg.org
>> > https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>> >
>> > To unsubscribe, visit link above, or email
>> > ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
>> _______________________________________________
>> ffmpeg-user mailing list
>> ffmpeg-user at ffmpeg.org
>> https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>>
>> To unsubscribe, visit link above, or email
>> ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
>>
>
More information about the ffmpeg-user
mailing list