[FFmpeg-devel] [PATCH] Fix segv when decoding gray8
Thu Jun 3 23:56:17 CEST 2010
On 06/03/2010 01:48 PM, Michael Niedermayer wrote:
> On Thu, Jun 03, 2010 at 12:11:41PM -0700, Baptiste Coudurier wrote:
>> On 06/03/2010 03:36 AM, Michael Niedermayer wrote:
>>> On Wed, Jun 02, 2010 at 06:39:26PM -0700, Baptiste Coudurier wrote:
>>>> Hi guys,
>>>> frame->data is set by avpicture_fill which is called with buf passed
>>>> from the demuxer.
>>>> However, for gray8 the data stored is only width*height, so is too small
>>>> for the palette, and will segv, memcpying.
>>>> I'm not sure what is the right fix here, buf is supposed to be const.
>>> as this case can only arrise from use of deprecated (and buggy) palette
>>> passing API, the correct (long term goal) is to change all codecs so they
>>> finally stop using this highly unpredictable race condition ridden api.
>>> until then, your patch may be ok
>> I don't think this is related to the API.
>> You cannot generate the palette for gray8 (according to pixfmt.h) _anyway_
>> since the data buffer is not big enough, since it comes from the demuxer
>> So either, the picture needs to be copied and the palette generated, or we
>> skip the palette generation.
> maybe iam missing something but why cant data that contains the palette
> point to some pal from the context while data would point to the
> buffer that originates from the demuxer?
Sorry, I'll put this aside for now. This whole palette thing is too
complicated, and I prefer spending this time on something else.
Key fingerprint 8D77134D20CC9220201FC5DB0AC9325C5C1ABAAA
FFmpeg maintainer http://www.ffmpeg.org
More information about the ffmpeg-devel