[FFmpeg-devel] [PATCH] avcodec/cuviddec: Add handling HDR10+ sidedata on cuviddec.

Lynne dev at lynne.ee
Fri Sep 20 22:14:26 EEST 2024


On 20/09/2024 20:41, Carlos Ruiz wrote:
>> It's just that the cuviddec decoder is more of a relic, from before the
>> native hwaccel existed.
>> The only reason it's not straight up deprecated is that it's sometimes
>> nice to have a "second opinion" when issues crop up, and there are a few
>> specific features like hardware-deinterlacing that can't be exposed via
>> the native hwaccel.
>>
>> But I'd normally not like to expand it even further and add complex and
>> large features to it. Whenever possible, the native nvdec hwaccel should
>> be used.
> 
> As someone who recently switched from *_cuvid to * + hwaccel=cuda I have to
> agree
> that the performance (timing-wise at least) has been very comparable for
> decoding.
> 
> The only comment I'd like to make though, and this might be a bit of
> unpopular opinion
> based on some other threads I read, but there are huge advantages of hwaccel
> accelerating not just decoding but also resizing (and I guess optionally
> cropping).
> I know that ideally a decoder should only decode, but think about a common
> usecase
> in the AI world we live in: you get a bunch of simultaneous 4k (or 1080p)
> incoming
> rtsp streams and you want to decode the video and pass it through some ML
> model,
> e.g. in TensorRT (to stick with the Nvidia example). The native hevc codec
> doesn't
> support resizing, so you decode video at full 4k on the gpu, which means
> allocating
> something like 5-10 surfaces at 3840x2160 which becomes 250MB of GPU memory,
> and then you have immediately take all of those frames, pass them through a
> filterchain,
> scale them down to e.g. 640x360, and waste CUDA cores instead of leveraging
> the
> dedicated video downsizing inside the NVDEC chip. Now do that for 50 camera
> streams
> and you'll quickly run out of GPU memory with a GPU utilization under 10%
> haha.
> 
> This is exactly why I submitted a patch yesterday that would allow using
> the hevc
> codec with nvdec hwaccel, while resizing on the gpu like hevc_cuviddec
> does, and
> the memory (and GPU) consuption goes waaay down (e.g. 6MB of GPU VRAM
> instead of 250MB per camera). I know this is a different discussion but
> thought
> it was appropriate to share because deprecating cuviddec or rejecting my
> patch
> would leave part of the community out.
> 
> 
> On Fri, Sep 20, 2024 at 12:48 PM Timo Rothenpieler <timo at rothenpieler.org>
> wrote:
> 
>> On 20/09/2024 04:08, 김윤주 wrote:
>>> Does native decoder refer to hevc (hevcdec.c)?
>>> I tried using hevc and in environments with low CPU performance,
>> hevc_cuvid
>>> was much faster.
>>> So, I used hevc_cuvid for decoding but encountered an issue where HDR10+
>>> sidedata did not exist.
>>> That's why I wrote this patch.
>>
>> You did turn on hwaccel, right?
>> I don't see why the native decoder would be much slower at parsing then
>> nvidias parsers.
>>
>>> I thought that the original content's sidedata should be preserved even
>>> after decoding regardless of which decoder is used.
>>> And I thought hevc_cuvid was the only way to get Nvidia hwaccel support.
>>> If I'm mistaken about anything, please let me know.
>>
>> It's just that the cuviddec decoder is more of a relic, from before the
>> native hwaccel existed.
>> The only reason it's not straight up deprecated is that it's sometimes
>> nice to have a "second opinion" when issues crop up, and there are a few
>> specific features like hardware-deinterlacing that can't be exposed via
>> the native hwaccel.
>>
>> But I'd normally not like to expand it even further and add complex and
>> large features to it. Whenever possible, the native nvdec hwaccel should
>> be used.
>>
>>> Also, is it correct to respond to your comments like this?
>>> It seems quite different from the format you sent.
>>
>> Top-posting isn't exactly liked here, though I don't really have a
>> strong opinion on it.
>>
>>> Apologies, as I'm still relatively new to the FFmpeg community and have a
>>> lot to learn.
>>> Any additional guidance would be greatly appreciated.
>>>
>>> -----Original Message-----
>>> From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of Timo
>>> Rothenpieler
>>> Sent: Friday, September 20, 2024 4:48 AM
>>> To: ffmpeg-devel at ffmpeg.org
>>> Subject: Re: [FFmpeg-devel] [PATCH] avcodec/cuviddec: Add handling HDR10+
>>> sidedata on cuviddec.
>>>
>>> On 19.09.2024 06:43, yoonjoo wrote:
>>>> Implemented decoding of NAL units and handling HDR10+ sidedata by
>>>> referring to hevcdec.
>>>
>>> Why? Can't you just use the native decoder with nvdec hwaccel?
>>> _______________________________________________
>>> ffmpeg-devel mailing list
>>> ffmpeg-devel at ffmpeg.org
>>> https://protect2.fireeye.com/v1/url?k=c11e9bb6-9e85a2a9-c11f10f9-
>>> 000babdfecba-c4d76825032a0b52&q=1&e=342e88a0-295e-4dc1-a57f-
>>>
>> 8f5aab974009&u=https%3A%2F%2Fffmpeg.org%2Fmailman%2Flistinfo%2Fffmpeg-devel
>>>
>>> To unsubscribe, visit link above, or email
>> ffmpeg-devel-request at ffmpeg.org
>>> with subject "unsubscribe".
>>>
>>> _______________________________________________
>>> ffmpeg-devel mailing list
>>> ffmpeg-devel at ffmpeg.org
>>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>>>
>>> To unsubscribe, visit link above, or email
>>> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>>
>> _______________________________________________
>> ffmpeg-devel mailing list
>> ffmpeg-devel at ffmpeg.org
>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>>
>> To unsubscribe, visit link above, or email
>> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".

Its not possible for a decoder to save memory by scaling references. 
Decoding would simply break.
Only way of saving memory while decoding is to simply get rid of output 
frames as soon as possible.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_0xA2FEA5F03F034464.asc
Type: application/pgp-keys
Size: 624 bytes
Desc: OpenPGP public key
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20240920/1d18827b/attachment.key>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature.asc
Type: application/pgp-signature
Size: 236 bytes
Desc: OpenPGP digital signature
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20240920/1d18827b/attachment.sig>


More information about the ffmpeg-devel mailing list