[FFmpeg-trac] #4873(avcodec:new): crashes in h264 decoder(decode_postinit)

[FFmpeg]

#4873: crashes in h264 decoder(decode_postinit)
-------------------------------------+-----------------------------------
             Reporter:  zylthinking  |                    Owner:
                 Type:  defect       |                   Status:  new
             Priority:  important    |                Component:  avcodec
              Version:  unspecified  |               Resolution:
             Keywords:  h264 crash   |               Blocked By:
             Blocking:               |  Reproduced by developer:  0
Analyzed by developer:  0            |
-------------------------------------+-----------------------------------

Comment (by zylthinking):

 {{{
 static void decode_postinit(H264Context *h, int setup_finished)
 {
     Picture *out = h->cur_pic_ptr;
     Picture *cur = h->cur_pic_ptr;
     int i, pics, out_of_order, out_idx;

    ''' h->cur_pic_ptr->f.pict_type = h->pict_type;'''

      crash here, due the crash log and the disassembly code, it should be
 h->cur_pic_ptr == NULL, there are some analyse below

     if (h->next_output_pic)
         return;
 ........................................
 }
 }}}

 disassembly code for the function is:

 {{{
   1633f4:       e59055e0        ldr     r5, [r0, #1504] ; 0x5e0
 ----------------- r5 is ldr from r0 add an offset, r0 should be
 H264Context *h, then r5 be a field of h
   1633f8:       e24dd014        sub     sp, sp, #20
   1633fc:       e7902002        ldr     r2, [r0, r2]
   163400:       e1a04000        mov     r4, r0
   163404:       e1a06001        mov     r6, r1
  ''' 163408:    e5852058        str     r2, [r5, #88]   ; 0x58
 -------------------------- here then  store r2 to address  r5 + 0x58  '''
 }}}

 look the crash log:
 signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr '''00000058''' here
 has a 0x58,

 and the code after that is
 {{{
   16340c:       0a000001        beq     163418 <decode_postinit+0x40>
   163410:       e28dd014        add     sp, sp, #20
   163414:       e8bd8ff0        pop     {r4, r5, r6, r7, r8, r9, sl, fp,
 pc}
 }}}

 check something == 0, if not equal to 0, then return.

 which is absolutely
     if (h->next_output_pic)
         return;

  OK, we can say something write into memory before a potential returning;
 then check the c code; it is only
  h->cur_pic_ptr->f.pict_type = h->pict_type;  satisfy this.

 OK, now we know this line crashes. while, because r5 is some filed of h;
 and    str     r2, [r5, #88]   seems to be writing something to r5's
 field; then we can know r5 should be the  h->cur_pic_ptr;

 OK, the crashing address is 0x58, and   str     r2, [r5, #88] is writing
 to r5 + 0x58; we know r5 is 0;
 e.g.   h->cur_pic_ptr == NULL

--
Ticket URL: <https://trac.ffmpeg.org/ticket/4873#comment:13>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker