[FFmpeg-devel] [PATCH] pgssubdec: only set w/h/linesize when allocating data

[Andreas Cadhalpun]

On 23.11.2016 14:05, Petri Hintukainen wrote:
> ti, 2016-11-22 kello 23:21 +0100, Andreas Cadhalpun kirjoitti:
>> On 09.11.2016 23:27, Andreas Cadhalpun wrote:
>>>
>>> Rects with positive w/h/linesize but no data are invalid.
>>>
>>> Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun at googlemail.com>
>>> ---
>>>  libavcodec/pgssubdec.c | 9 +++++----
>>>  1 file changed, 5 insertions(+), 4 deletions(-)
>>>
>>> diff --git a/libavcodec/pgssubdec.c b/libavcodec/pgssubdec.c
>>> index cef477d..b50b37b 100644
>>> --- a/libavcodec/pgssubdec.c
>>> +++ b/libavcodec/pgssubdec.c
>>> @@ -556,12 +556,13 @@ static int display_end_segment(AVCodecContext
>>> *avctx, void *data,
>>>  
>>>          sub->rects[i]->x    = ctx->presentation.objects[i].x;
>>>          sub->rects[i]->y    = ctx->presentation.objects[i].y;
>>> -        sub->rects[i]->w    = object->w;
>>> -        sub->rects[i]->h    = object->h;
>>> -
>>> -        sub->rects[i]->linesize[0] = object->w;
>>>  
>>>          if (object->rle) {
>>> +            sub->rects[i]->w    = object->w;
>>> +            sub->rects[i]->h    = object->h;
>>> +
>>> +            sub->rects[i]->linesize[0] = object->w;
>>> +
>>>              if (object->rle_remaining_len) {
>>>                  av_log(avctx, AV_LOG_ERROR, "RLE data length %u is
>>> %u bytes shorter than expected\n",
>>>                         object->rle_data_len, object-
>>>> rle_remaining_len);
>>>
>> Ping. I intend to apply this in a few days.
> 
> Looks OK, w and h may contain old values if rle is NULL.

Pushed.

> Maybe those could be set after decode_rle() result check, you could
> then also simplify the error branch (no need to set w, h back to 0) ?

That's not possible, because decode_rle actually needs w and h set to
work correctly.

Best regards,
Andreas