[FFmpeg-devel] [PATCH] Implement PAFF in H.264

Jeff Downs heydowns
Thu Sep 20 21:04:50 CEST 2007


On Wed, 19 Sep 2007, Michael Niedermayer wrote:

> just a quick review below, ill wait with a full review until there are
> clean and well split patches ...

Thanks.  I knew it had to be split and I'm happy to be receiving guidance 
as to where.

I've attached 3 patches, which are intended to be examined/applied in 
order:

1. paff-cosmetics.patch: documentation clarifications and change PICT_* to 
reinforce that the forthcoming PAFF code depends on
PICT_FRAME == PICT_TOP_FIELD | PICT_BOTTOM_FIELD

2. paff-mmcorename.patch:  Renaming for mmco structure fields; new names 
make more sense than old when considering their operations on field 
pictures (in addition to frame pictures).

3. paff-noindent-substance.patch: The actual substance of the PAFF 
implementation. This is the remainder of my original patch, with all large 
blocks of reindentation removed at Vitor's suggestion, intending to be 
re-indented after this is accepted.  It also contains several fixes and 
changes from the last patch to address some of Michael's concerns (below), 
Diego's comments about long lines (all substantially changed or added 
lines are <= 80 chars), and incorporates Martin Zlomek's 
posted contributions.

Comments to detailed concerns below.


>>> Maybe it would be better to do the renaming of that var before or after
>>> the bulk of your patch is applied.
>>
>> I can do that if necessary; the renaming was done to have the variable make
>> sense in both frame and field contexts.
>
> yes, please do, all cosmetics must be in a seperate patch

Done via paff-mmcorename.patch.

>
> i dont think this should be inline, it doesnt appear speed critical

No longer inline.

>
>> +    int same_i, opp_i;
>> +    int i;
>> +    int same;
>> +    int out_i;
>> +
>> +    same_i = 0;
>> +    opp_i = 0;
>> +    same = 1;
>> +    out_i = 0;
>> +
>> +    while (out_i < dest_len) {
>
>> +        if (same && same_i < src_len) {
>> +            if ((src[same_i].valid_structure & parity)) {
>> +                same = 0;
>> +                dest[out_i] = src[same_i];
>> +                for (i = 0; i < 4; ++i) {
>> +                    if (parity == PICT_BOTTOM_FIELD)
>> +                        dest[out_i].data[i] += dest[out_i].linesize[i];
>> +                    dest[out_i].linesize[i] *= 2;
>> +                    dest[out_i].pic_id *= 2;
>> +                }
>> +                out_i++;
>> +            }
>> +            same_i++;
>> +
>> +        } else if (opp_i < src_len) {
>> +            if ((src[opp_i].valid_structure & (PICT_FRAME - parity))) {
>> +                same = 1;
>> +                dest[out_i] = src[opp_i];
>> +                for (i = 0; i < 4; ++i) {
>> +                    if (parity == PICT_TOP_FIELD)
>> +                        dest[out_i].data[i] += dest[out_i].linesize[i];
>> +                    dest[out_i].linesize[i] *= 2;
>> +                    dest[out_i].pic_id *= 2;
>> +                    dest[out_i].pic_id++;
>> +                }
>> +                out_i++;
>> +            }
>> +            opp_i++;
>> +
>
> near duplicate

Fixed by pulling common pieces to a function.

> [...]
>>  /**
>> + * Removes reference marking from a field or field pair by picture number.
>> + * If an unmatched field pair, or both fields in a pair become unreferenced,
>> + * the field (pair) is removed from the short term reference list and list
>> + * state is updated accordingly.
>> + * @param picret set to the unreferenced and removed picture, or NULL if the
>> + *         picture still has fields in reference or was not found.
>> + * @return -1 if picture with pic_num not found. 0 otherwise
>> + */
>> +static int remove_field_short(H264Context *h, int pic_num, Picture **picret){
>> +    MpegEncContext * const s = &h->s;
>> +    int i;
>> +    Picture *pic = NULL;
>> +    int frame_num = pic_num >> 1;
>> +
>> +    if(s->avctx->debug&FF_DEBUG_MMCO)
>> +        av_log(h->s.avctx, AV_LOG_DEBUG, "remove field short %d count %d\n", pic_num, h->short_ref_count);
>> +
>> +    for(i=0; i<h->short_ref_count; i++){
>> +        pic= h->short_ref[i];
>> +
>> +        if(s->avctx->debug&FF_DEBUG_MMCO)
>> +            av_log(h->s.avctx, AV_LOG_DEBUG, "%d %d %p\n", i, pic->frame_num, pic);
>> +
>> +        if (pic->frame_num == frame_num)
>> +            break;
>> +    }
>> +
>> +    *picret = NULL;
>> +    if (pic) {
>> +        int mask;
>> +
>> +        mask = (pic_num & 1) ? ~s->picture_structure : s->picture_structure;
>> +        pic->valid_structure &= mask;
>> +
>> +        if (pic->valid_structure == 0) {
>> +            h->short_ref[i]= NULL;
>> +            if (--h->short_ref_count)
>> +                memmove(&h->short_ref[i], &h->short_ref[i+1], (h->short_ref_count - i)*sizeof(Picture*));
>> +            *picret = pic;
>> +        }
>> +        return 0;
>> +    }
>> +
>> +    return -1;
>> +}
>
> this is largly a duplicate of remove_short()

Fixed by folding functionality into remove_short.


> [...]
>> @@ -7402,13 +7788,26 @@
>>
>>          h->prev_frame_num_offset= h->frame_num_offset;
>>          h->prev_frame_num= h->frame_num;
>> -        if(s->current_picture_ptr->reference){
>> +        if(s->current_picture.reference){
>
> this is wrong, current_picture is a copy of current_picture_ptr
> if current_picture is correct while _ptr is not theres a bug
>

This is different for the second field in a pair, a by product of not 
calling MPV_frame_start to get a new picture for the second field in a 
pair. I implemented it that way partly because that is how the mpeg2
decoder works.
mpeg2 decoder uses current_picture to represent the current field 
parameters (data and linesize, mostly) which differ from 
*current_picture_ptr parameters (represents full frame).

In this case, current_picture_ptr->reference is combined reference status 
of top and bottom fields, while current_picture.reference is reference 
status of currently decoding field (as in data/linesize above).

This area needs the reference marking of the currently decoding field, 
regardless of any prior field in a pair.

If it cannot be done using current_picture.reference, then should a field 
be added to H264Context to represent this?

>
> [...]
>
>> +            /*
>> +             * FIXME: Error handling code does not seem to support interlaced
>> +             * when slices span multiple rows
>> +             */
>> +            if (!FIELD_PICTURE)
>>          ff_er_frame_end(s);
>
> why doesnt this work?

The error concealer doesn't seem to have any notion of being able to skip 
rows, both when marking errors and doing the concealing calculations. 
Calls to ff_er_add_slice end result in error_count going through the roof 
for perfectly ok frames. The frame_end call then falsely corrects the 
errors.  It was easiest and arguably more efficient to abort the frame_end 
call instead of all the different add_slice calls.

I would be willing to work to fix this after we get through this already 
behemoth patch, at which time this could be removed.

There was a comment explaining this near an add_slice call in my previous 
patch. I've moved that here in the current patch.

>> --- libavcodec/mpegvideo.h	(revision 10526)
>> +++ libavcodec/mpegvideo.h	(working copy)
>> @@ -138,8 +138,9 @@
>>
>>      int field_poc[2];           ///< h264 top/bottom POC
>>      int poc;                    ///< h264 frame POC
>> +    int valid_structure;        ///< h264 one of PICT_XXXX, stating which fields are referenced
>
> if my memory doesnt fail my then we already have this variable, its called
> reference

Yes. From what I've been able to tell by reading the existing code, 
Picture.reference == 1 means keep it around because I need to display it 
still.  Picture.reference == 3 means keep it around because it is a 
reference frame.

This new variable is used to track reference marking of the 
top and bottom fields. Most of the new reference marking code uses this to 
keep track of which field in a pair (or both) is used for reference. I 
created a new variable to avoid modifying the behavior of the reference 
variable and because it was very nice to be able to set it to PICT_XXX 
values, something I couldn't do with the existing variable because 
PICT_TOP_FIELD == 1 == existing semantic meaning above.

I'm open to suggestions on changing this if needed;  if the replacement 
doesn't have it taking on values of PICT_XXX, then it could be a 
substantial rework.

 	-Jeff
-------------- next part --------------
Index: libavcodec/h264.h
===================================================================
--- libavcodec/h264.h	(revision 10526)
+++ libavcodec/h264.h	(working copy)
@@ -84,8 +86,8 @@
     int poc_cycle_length;              ///< num_ref_frames_in_pic_order_cnt_cycle
     int ref_frame_count;               ///< num_ref_frames
     int gaps_in_frame_num_allowed_flag;
-    int mb_width;                      ///< frame_width_in_mbs_minus1 + 1
-    int mb_height;                     ///< frame_height_in_mbs_minus1 + 1
+    int mb_width;                      ///< pic_width_in_mbs_minus1 + 1
+    int mb_height;                     ///< pic_height_in_map_units_minus1 + 1
     int frame_mbs_only_flag;
     int mb_aff;                        ///<mb_adaptive_frame_field_flag
     int direct_8x8_inference_flag;
@@ -323,8 +325,9 @@
     unsigned int list_count;
     Picture *short_ref[32];
     Picture *long_ref[32];
-    Picture default_ref_list[2][32];
+    Picture default_ref_list[2][32]; ///< base reference list for all slices of a coded picture
     Picture ref_list[2][48];     ///< 0..15: frame refs, 16..47: mbaff field refs
+                                 ///< Reordered version of default_ref_list according to picture reordering in slice header
     Picture *delayed_pic[18]; //FIXME size?
     Picture *delayed_output_pic;
 
Index: libavcodec/mpegvideo.h
===================================================================
--- libavcodec/mpegvideo.h	(revision 10526)
+++ libavcodec/mpegvideo.h	(working copy)
@@ -138,8 +138,8 @@
 
     int field_poc[2];           ///< h264 top/bottom POC
     int poc;                    ///< h264 frame POC
-    int frame_num;              ///< h264 frame_num
-    int pic_id;                 ///< h264 pic_num or long_term_pic_idx
+    int frame_num;              ///< h264 frame_num (raw frame_num from slice header)
+    int pic_id;                 ///< h264 pic_num (short or long term)
     int long_ref;               ///< 1->long term reference 0->short term reference
     int ref_poc[2][16];         ///< h264 POCs of the frames used as reference
     int ref_count[2];           ///< number of entries in ref_poc
@@ -622,9 +623,9 @@
     int mpeg_f_code[2][2];
     int picture_structure;
 /* picture type */
-#define PICT_TOP_FIELD     1
-#define PICT_BOTTOM_FIELD  2
-#define PICT_FRAME         3
+#define PICT_TOP_FIELD     0x1
+#define PICT_BOTTOM_FIELD  0x2
+#define PICT_FRAME         (PICT_TOP_FIELD | PICT_BOTTOM_FIELD)
 
     int intra_dc_precision;
     int frame_pred_frame_dct;
-------------- next part --------------
Index: h264.c
===================================================================
--- h264.c	(revision 10527)
+++ h264.c	(working copy)
@@ -3222,47 +3222,47 @@
 
     for(i=0; i<mmco_count; i++){
         if(s->avctx->debug&FF_DEBUG_MMCO)
-            av_log(h->s.avctx, AV_LOG_DEBUG, "mmco:%d %d %d\n", h->mmco[i].opcode, h->mmco[i].short_frame_num, h->mmco[i].long_index);
+            av_log(h->s.avctx, AV_LOG_DEBUG, "mmco:%d %d %d\n", h->mmco[i].opcode, h->mmco[i].short_pic_num, h->mmco[i].long_arg);
 
         switch(mmco[i].opcode){
         case MMCO_SHORT2UNUSED:
-            pic= remove_short(h, mmco[i].short_frame_num);
+            pic= remove_short(h, mmco[i].short_pic_num);
             if(pic)
                 unreference_pic(h, pic);
             else if(s->avctx->debug&FF_DEBUG_MMCO)
                 av_log(h->s.avctx, AV_LOG_DEBUG, "mmco: remove_short() failure\n");
             break;
         case MMCO_SHORT2LONG:
-            pic= remove_long(h, mmco[i].long_index);
+            pic= remove_long(h, mmco[i].long_arg);
             if(pic) unreference_pic(h, pic);
 
-            h->long_ref[ mmco[i].long_index ]= remove_short(h, mmco[i].short_frame_num);
-            if (h->long_ref[ mmco[i].long_index ]){
-                h->long_ref[ mmco[i].long_index ]->long_ref=1;
+            h->long_ref[ mmco[i].long_arg ]= remove_short(h, mmco[i].short_pic_num);
+            if (h->long_ref[ mmco[i].long_arg ]){
+                h->long_ref[ mmco[i].long_arg ]->long_ref=1;
                 h->long_ref_count++;
             }
             break;
         case MMCO_LONG2UNUSED:
-            pic= remove_long(h, mmco[i].long_index);
+            pic= remove_long(h, mmco[i].long_arg);
             if(pic)
                 unreference_pic(h, pic);
             else if(s->avctx->debug&FF_DEBUG_MMCO)
                 av_log(h->s.avctx, AV_LOG_DEBUG, "mmco: remove_long() failure\n");
             break;
         case MMCO_LONG:
-            pic= remove_long(h, mmco[i].long_index);
+            pic= remove_long(h, mmco[i].long_arg);
             if(pic) unreference_pic(h, pic);
 
-            h->long_ref[ mmco[i].long_index ]= s->current_picture_ptr;
-            h->long_ref[ mmco[i].long_index ]->long_ref=1;
+            h->long_ref[ mmco[i].long_arg ]= s->current_picture_ptr;
+            h->long_ref[ mmco[i].long_arg ]->long_ref=1;
             h->long_ref_count++;
 
             current_is_long=1;
             break;
         case MMCO_SET_MAX_LONG:
-            assert(mmco[i].long_index <= 16);
+            assert(mmco[i].long_arg <= 16);
             // just remove the long term which index is greater than new max
-            for(j = mmco[i].long_index; j<16; j++){
+            for(j = mmco[i].long_arg; j<16; j++){
                 pic = remove_long(h, j);
                 if (pic) unreference_pic(h, pic);
             }
@@ -3307,8 +3307,8 @@
 
     if(h->nal_unit_type == NAL_IDR_SLICE){ //FIXME fields
         s->broken_link= get_bits1(gb) -1;
-        h->mmco[0].long_index= get_bits1(gb) - 1; // current_long_term_idx
-        if(h->mmco[0].long_index == -1)
+        h->mmco[0].long_arg= get_bits1(gb) - 1; // current_long_term_idx
+        if(h->mmco[0].long_arg == -1)
             h->mmco_index= 0;
         else{
             h->mmco[0].opcode= MMCO_LONG;
@@ -3321,19 +3321,19 @@
 
                 h->mmco[i].opcode= opcode;
                 if(opcode==MMCO_SHORT2UNUSED || opcode==MMCO_SHORT2LONG){
-                    h->mmco[i].short_frame_num= (h->frame_num - get_ue_golomb(gb) - 1) & ((1<<h->sps.log2_max_frame_num)-1); //FIXME fields
-/*                    if(h->mmco[i].short_frame_num >= h->short_ref_count || h->short_ref[ h->mmco[i].short_frame_num ] == NULL){
+                    h->mmco[i].short_pic_num= (h->frame_num - get_ue_golomb(gb) - 1) & ((1<<h->sps.log2_max_frame_num)-1); //FIXME fields
+/*                    if(h->mmco[i].short_pic_num >= h->short_ref_count || h->short_ref[ h->mmco[i].short_pic_num ] == NULL){
                         av_log(s->avctx, AV_LOG_ERROR, "illegal short ref in memory management control operation %d\n", mmco);
                         return -1;
                     }*/
                 }
                 if(opcode==MMCO_SHORT2LONG || opcode==MMCO_LONG2UNUSED || opcode==MMCO_LONG || opcode==MMCO_SET_MAX_LONG){
                     unsigned int long_index= get_ue_golomb(gb);
-                    if(/*h->mmco[i].long_index >= h->long_ref_count || h->long_ref[ h->mmco[i].long_index ] == NULL*/ long_index >= 16){
+                    if(/*h->mmco[i].long_arg >= h->long_ref_count || h->long_ref[ h->mmco[i].long_arg ] == NULL*/ long_index >= 16){
                         av_log(h->s.avctx, AV_LOG_ERROR, "illegal long ref in memory management control operation %d\n", opcode);
                         return -1;
                     }
-                    h->mmco[i].long_index= long_index;
+                    h->mmco[i].long_arg= long_index;
                 }
 
                 if(opcode > (unsigned)MMCO_LONG){
@@ -3349,7 +3349,7 @@
 
             if(h->long_ref_count + h->short_ref_count == h->sps.ref_frame_count){ //FIXME fields
                 h->mmco[0].opcode= MMCO_SHORT2UNUSED;
-                h->mmco[0].short_frame_num= h->short_ref[ h->short_ref_count - 1 ]->frame_num;
+                h->mmco[0].short_pic_num= h->short_ref[ h->short_ref_count - 1 ]->frame_num;
                 h->mmco_index= 1;
             }else
                 h->mmco_index= 0;
Index: h264.h
===================================================================
--- h264.h	(revision 10527)
+++ h264.h	(working copy)
@@ -151,8 +151,8 @@
  */
 typedef struct MMCO{
     MMCOOpcode opcode;
-    int short_frame_num;
-    int long_index;
+    int short_pic_num;
+    int long_arg;  ///< index, pic_num, or num long refs depending on opcode
 } MMCO;
 
 /**
-------------- next part --------------
--- ../ffmpeg-cosmetics/libavcodec/h264.c	2007-09-18 17:35:33.000000000 -0400
+++ libavcodec/h264.c	2007-09-20 14:29:20.000000000 -0400
@@ -169,11 +169,11 @@
     if(for_deblock && (h->slice_num == 1 || h->slice_table[mb_xy] == h->slice_table[mb_xy-s->mb_stride]) && !FRAME_MBAFF)
         return;
 
     //wow what a mess, why didn't they simplify the interlacing&intra stuff, i can't imagine that these complex rules are worth it
 
-    top_xy     = mb_xy  - s->mb_stride;
+    top_xy     = mb_xy  - (s->mb_stride << FIELD_PICTURE);
     topleft_xy = top_xy - 1;
     topright_xy= top_xy + 1;
     left_xy[1] = left_xy[0] = mb_xy-1;
     left_block[0]= 0;
     left_block[1]= 1;
@@ -1695,11 +1695,11 @@
     int extra_height= h->emu_edge_height;
     int emu=0;
     const int full_mx= mx>>2;
     const int full_my= my>>2;
     const int pic_width  = 16*s->mb_width;
-    const int pic_height = 16*s->mb_height >> MB_MBAFF;
+    const int pic_height = 16*s->mb_height >> (MB_MBAFF || FIELD_PICTURE);
 
     if(!pic->data[0]) //FIXME this is unacceptable, some senseable error concealment must be done for missing reference frames
         return;
 
     if(mx&7) extra_width -= 3;
@@ -1719,11 +1719,11 @@
         qpix_op[luma_xy](dest_y + delta, src_y + delta, h->mb_linesize);
     }
 
     if(ENABLE_GRAY && s->flags&CODEC_FLAG_GRAY) return;
 
-    if(MB_MBAFF){
+    if(MB_MBAFF || FIELD_PICTURE){
         // chroma offset when predicting from a field of opposite parity
         my += 2 * ((s->mb_y & 1) - (h->ref_cache[list][scan8[n]] & 1));
         emu |= (my>>3) < 0 || (my>>3) + 8 >= (pic_height>>1);
     }
     src_cb= pic->data[1] + (mx>>3) + (my>>3)*h->mb_uvlinesize;
@@ -1754,11 +1754,11 @@
 
     dest_y  += 2*x_offset + 2*y_offset*h->  mb_linesize;
     dest_cb +=   x_offset +   y_offset*h->mb_uvlinesize;
     dest_cr +=   x_offset +   y_offset*h->mb_uvlinesize;
     x_offset += 8*s->mb_x;
-    y_offset += 8*(s->mb_y >> MB_MBAFF);
+    y_offset += 8*(s->mb_y >> (MB_MBAFF || FIELD_PICTURE));
 
     if(list0){
         Picture *ref= &h->ref_list[0][ h->ref_cache[0][ scan8[n] ] ];
         mc_dir_part(h, ref, n, square, chroma_height, delta, 0,
                            dest_y, dest_cb, dest_cr, x_offset, y_offset,
@@ -1787,11 +1787,11 @@
 
     dest_y  += 2*x_offset + 2*y_offset*h->  mb_linesize;
     dest_cb +=   x_offset +   y_offset*h->mb_uvlinesize;
     dest_cr +=   x_offset +   y_offset*h->mb_uvlinesize;
     x_offset += 8*s->mb_x;
-    y_offset += 8*(s->mb_y >> MB_MBAFF);
+    y_offset += 8*(s->mb_y >> (MB_MBAFF || FIELD_PICTURE));
 
     if(list0 && list1){
         /* don't optimize for luma-only case, since B-frames usually
          * use implicit weights => chroma too. */
         uint8_t *tmp_cb = s->obmc_scratchpad;
@@ -2749,20 +2749,140 @@
     if (is_complex)
         hl_decode_mb_complex(h);
     else hl_decode_mb_simple(h);
 }
 
+static void pic_set_as_field(Picture *pic, const int bottom){
+    int i;
+    for (i = 0; i < 4; ++i) {
+        if (bottom)
+            pic->data[i] += pic->linesize[i];
+        pic->linesize[i] *= 2;
+    }
+}
+
+static int split_field_copy(Picture *dest, Picture *src,
+                            int parity, int id_add){
+    int i;
+    int match = (src->valid_structure & parity) != 0;
+
+    if (match) {
+        *dest = *src;
+        pic_set_as_field(dest, parity == PICT_BOTTOM_FIELD);
+        dest->pic_id *= 2;
+        dest->pic_id += id_add;
+    }
+
+    return match;
+}
+
+/**
+ * Split one reference list into field parts, interleaving by parity
+ * as per H.264 spec section 8.2.4.2.5. Output fields have their data pointers
+ * set to look at the actual start of data for that field.
+ *
+ * @param dest output list
+ * @param dest_len maximum number of fields to put in dest
+ * @param src the source reference list containing fields and/or field pairs
+ *            (aka short_ref/long_ref, or
+ *             refFrameListXShortTerm/refFrameListLongTerm in spec-speak)
+ * @param src_len number of Picture's in source (pairs and unmatched fields)
+ * @param parity the parity of the picture being decoded/needing
+ *        these ref pics (PICT_{TOP,BOTTOM}_FIELD)
+ * @return number of fields placed in dest
+ */
+static int split_field_half_ref_list(Picture *dest, int dest_len,
+                                     Picture *src,  int src_len,  int parity){
+    int same_i, opp_i;
+    int i;
+    int same;
+    int out_i;
+
+    same_i = 0;
+    opp_i = 0;
+    same = 1;
+    out_i = 0;
+
+    while (out_i < dest_len) {
+        if (same && same_i < src_len) {
+            i = split_field_copy(dest + out_i, src + same_i, parity, 1);
+            same = !i;
+            same_i++;
+
+        } else if (opp_i < src_len) {
+            i = split_field_copy(dest + out_i, src + opp_i,
+                                 PICT_FRAME - parity, 0);
+            same = i;
+            opp_i++;
+
+        } else {
+            break;
+        }
+        out_i += i;
+    }
+
+    return out_i;
+}
+
+/**
+ * Split a the reference frame list into a reference field list.
+ * This implements H.264 spec 8.2.4.2.5 for a combined input list.
+ * The input list contains both reference field pairs and
+ * unmatched reference fields; it is ordered as spec describes
+ * RefPicListX for frames in 8.2.4.2.1 and 8.2.4.2.3, except that
+ * unmatched field pairs are also present. Conceptually this is equivalent
+ * to concatenation of refFrameListXShortTerm with refFrameListLongTerm.
+ *
+ * @param dest output reference list where ordered fields are to be placed
+ * @param dest_len max number of fields to place at dest
+ * @param src source reference list, as described above
+ * @param src_len number of pictures (pairs and unmatched fields) in src
+ * @param parity parity of field being currently decoded
+ *        (one of PICT_{TOP,BOTTOM}_FIELD)
+ * @param long_i index into src array that holds first long reference picture,
+ *        or src_len if no long refs present.
+ */
+static int split_field_ref_list(Picture *dest, int dest_len,
+                                Picture *src,  int src_len,
+                                int parity,    int long_i){
+
+    int i = split_field_half_ref_list(dest, dest_len, src, src_len, parity);
+    dest += i;
+    dest_len -= i;
+
+    i += split_field_half_ref_list(dest, dest_len, src + long_i,
+                                   src_len - long_i, parity);
+    return i;
+}
+
 /**
  * fills the default_ref_list.
  */
 static int fill_default_ref_list(H264Context *h){
     MpegEncContext * const s = &h->s;
     int i;
     int smallest_poc_greater_than_current = -1;
+    const int max_frame_num = 1<<h->sps.log2_max_frame_num;
+    int structure_sel;
     Picture sorted_short_ref[32];
+    Picture field_entry_list[2][32];
+    Picture *frame_list[2];
+
+    if (!FIELD_PICTURE) {
+        structure_sel = 0;
+        frame_list[0] = h->default_ref_list[0];
+        frame_list[1] = h->default_ref_list[1];
+    } else {
+        structure_sel = PICT_FRAME;
+        frame_list[0] = field_entry_list[0];
+        frame_list[1] = field_entry_list[1];
+    }
 
     if(h->slice_type==B_TYPE){
+        int list;
+        int len[2];
+        int short_len[2];
         int out_i;
         int limit= INT_MIN;
 
         /* sort frame according to poc in B slice */
         for(out_i=0; out_i<h->short_ref_count; out_i++){
@@ -2786,75 +2906,107 @@
                 if (h->short_ref[best_i]->poc >= s->current_picture_ptr->poc) {
                     smallest_poc_greater_than_current = out_i;
                 }
             }
         }
-    }
 
-    if(s->picture_structure == PICT_FRAME){
-        if(h->slice_type==B_TYPE){
-            int list;
             tprintf(h->s.avctx, "current poc: %d, smallest_poc_greater_than_current: %d\n", s->current_picture_ptr->poc, smallest_poc_greater_than_current);
 
             // find the largest poc
             for(list=0; list<2; list++){
                 int index = 0;
                 int j= -99;
                 int step= list ? -1 : 1;
 
                 for(i=0; i<h->short_ref_count && index < h->ref_count[list]; i++, j+=step) {
+                    int sel;
                     while(j<0 || j>= h->short_ref_count){
                         if(j != -99 && step == (list ? -1 : 1))
                             return -1;
                         step = -step;
                         j= smallest_poc_greater_than_current + (step>>1);
                     }
                     if(sorted_short_ref[j].reference != 3) continue;
-                    h->default_ref_list[list][index  ]= sorted_short_ref[j];
-                    h->default_ref_list[list][index++].pic_id= sorted_short_ref[j].frame_num;
+                    sel = sorted_short_ref[j].valid_structure | structure_sel;
+                    if(sel != PICT_FRAME) continue;
+                    frame_list[list][index]= sorted_short_ref[j];
+                    frame_list[list][index].pic_id= sorted_short_ref[j].frame_num;
+                    if (frame_list[list][index].pic_id > h->frame_num)
+                        frame_list[list][index].pic_id -= max_frame_num;
+                    index++;
                 }
+                short_len[list] = index;
 
                 for(i = 0; i < 16 && index < h->ref_count[ list ]; i++){
+                    int sel;
                     if(h->long_ref[i] == NULL) continue;
                     if(h->long_ref[i]->reference != 3) continue;
+                    sel = h->long_ref[i]->valid_structure | structure_sel;
+                    if(sel != PICT_FRAME) continue;
 
-                    h->default_ref_list[ list ][index  ]= *h->long_ref[i];
-                    h->default_ref_list[ list ][index++].pic_id= i;;
+                    frame_list[ list ][index  ]= *h->long_ref[i];
+                    frame_list[ list ][index++].pic_id= i;;
                 }
+                len[list] = index;
 
                 if(list && (smallest_poc_greater_than_current<=0 || smallest_poc_greater_than_current>=h->short_ref_count) && (1 < index)){
                     // swap the two first elements of L1 when
                     // L0 and L1 are identical
-                    Picture temp= h->default_ref_list[1][0];
-                    h->default_ref_list[1][0] = h->default_ref_list[1][1];
-                    h->default_ref_list[1][1] = temp;
+                    Picture temp= frame_list[1][0];
+                    frame_list[1][0] = frame_list[1][1];
+                    frame_list[1][1] = temp;
                 }
 
-                if(index < h->ref_count[ list ])
-                    memset(&h->default_ref_list[list][index], 0, sizeof(Picture)*(h->ref_count[ list ] - index));
             }
+
+            for(list=0; list<2; list++){
+                if (s->picture_structure != PICT_FRAME)
+                    len[list] = split_field_ref_list(h->default_ref_list[list],
+                                                     h->ref_count[list],
+                                                     frame_list[list],
+                                                     len[list],
+                                                     s->picture_structure,
+                                                     short_len[list]);
+
+                if(len[list] < h->ref_count[ list ])
+                    memset(&h->default_ref_list[list][len[list]], 0, sizeof(Picture)*(h->ref_count[ list ] - len[list]));
+            }
+
+
         }else{
             int index=0;
+            int short_len;
             for(i=0; i<h->short_ref_count; i++){
-                if(h->short_ref[i]->reference != 3) continue; //FIXME refernce field shit
-                h->default_ref_list[0][index  ]= *h->short_ref[i];
-                h->default_ref_list[0][index++].pic_id= h->short_ref[i]->frame_num;
+                int sel;
+                if(h->short_ref[i]->reference != 3) continue;
+                sel = h->short_ref[i]->valid_structure | structure_sel;
+                if(sel != PICT_FRAME) continue;
+                frame_list[0][index]= *h->short_ref[i];
+                frame_list[0][index].pic_id= h->short_ref[i]->frame_num;
+                if (frame_list[0][index].pic_id > h->frame_num)
+                    frame_list[0][index].pic_id -= max_frame_num;
+                index++;
             }
+            short_len = index;
             for(i = 0; i < 16; i++){
+                int sel;
                 if(h->long_ref[i] == NULL) continue;
                 if(h->long_ref[i]->reference != 3) continue;
-                h->default_ref_list[0][index  ]= *h->long_ref[i];
-                h->default_ref_list[0][index++].pic_id= i;;
+                sel = h->long_ref[i]->valid_structure | structure_sel;
+                if(sel != PICT_FRAME) continue;
+                frame_list[0][index  ]= *h->long_ref[i];
+                frame_list[0][index++].pic_id= i;;
             }
+
+            if (s->picture_structure != PICT_FRAME)
+                index = split_field_ref_list(h->default_ref_list[0],
+                                             h->ref_count[0], frame_list[0],
+                                             index, s->picture_structure,
+                                             short_len);
+
             if(index < h->ref_count[0])
                 memset(&h->default_ref_list[0][index], 0, sizeof(Picture)*(h->ref_count[0] - index));
-        }
-    }else{ //FIELD
-        if(h->slice_type==B_TYPE){
-        }else{
-            //FIXME second field balh
-        }
     }
 #ifdef TRACE
     for (i=0; i<h->ref_count[0]; i++) {
         tprintf(h->s.avctx, "List0: %s fn:%d 0x%p\n", (h->default_ref_list[0][i].long_ref ? "LT" : "ST"), h->default_ref_list[0][i].pic_id, h->default_ref_list[0][i].data[0]);
     }
@@ -2870,11 +3022,11 @@
 static void print_short_term(H264Context *h);
 static void print_long_term(H264Context *h);
 
 static int decode_ref_pic_list_reordering(H264Context *h){
     MpegEncContext * const s = &h->s;
-    int list, index;
+    int list, index, pic_structure;
 
     print_short_term(h);
     print_long_term(h);
     if(h->slice_type==I_TYPE || h->slice_type==SI_TYPE) return 0; //FIXME move before func
 
@@ -2897,39 +3049,76 @@
                     av_log(h->s.avctx, AV_LOG_ERROR, "reference count overflow\n");
                     return -1;
                 }
 
                 if(reordering_of_pic_nums_idc<3){
+                    int add;
                     if(reordering_of_pic_nums_idc<2){
                         const unsigned int abs_diff_pic_num= get_ue_golomb(&s->gb) + 1;
+                        int frame_num;
 
                         if(abs_diff_pic_num >= h->max_pic_num){
                             av_log(h->s.avctx, AV_LOG_ERROR, "abs_diff_pic_num overflow\n");
                             return -1;
                         }
 
                         if(reordering_of_pic_nums_idc == 0) pred-= abs_diff_pic_num;
                         else                                pred+= abs_diff_pic_num;
                         pred &= h->max_pic_num - 1;
 
+                        if (FIELD_PICTURE) {
+                            frame_num = pred >> 1;
+                            if (pred & 1) {
+                                pic_structure = s->picture_structure;
+                                add = (s->picture_structure == PICT_BOTTOM_FIELD);
+                            } else {
+                                pic_structure = PICT_FRAME - s->picture_structure;
+                                add = (s->picture_structure == PICT_TOP_FIELD);
+                            }
+                        } else {
+                            frame_num = pred;
+                            pic_structure = PICT_FRAME;
+                            add = 0;
+                        }
+
                         for(i= h->short_ref_count-1; i>=0; i--){
                             ref = h->short_ref[i];
                             assert(ref->reference == 3);
                             assert(!ref->long_ref);
-                            if(ref->data[0] != NULL && ref->frame_num == pred && ref->long_ref == 0) // ignore non existing pictures by testing data[0] pointer
+                            if(ref->data[0] != NULL &&
+                                   ref->frame_num == frame_num &&
+                                   (ref->valid_structure & pic_structure) &&
+                                   ref->long_ref == 0) // ignore non existing pictures by testing data[0] pointer and valid_structure
                                 break;
                         }
                         if(i>=0)
-                            ref->pic_id= ref->frame_num;
+                            ref->pic_id= pred;
                     }else{
+                        int long_idx;
                         pic_id= get_ue_golomb(&s->gb); //long_term_pic_idx
-                        if(pic_id>31){
+
+                        if (FIELD_PICTURE) {
+                            long_idx = pic_id >> 1;
+                            if (pic_id & 1) {
+                                pic_structure = s->picture_structure;
+                                add = (s->picture_structure == PICT_BOTTOM_FIELD);
+                            } else {
+                                pic_structure = PICT_FRAME - s->picture_structure;
+                                add = (s->picture_structure == PICT_TOP_FIELD);
+                            }
+                        } else {
+                            long_idx = pic_id;
+                            pic_structure = PICT_FRAME;
+                            add = 0;
+                        }
+
+                        if(long_idx>31){
                             av_log(h->s.avctx, AV_LOG_ERROR, "long_term_pic_idx overflow\n");
                             return -1;
                         }
-                        ref = h->long_ref[pic_id];
-                        if(ref){
+                        ref = h->long_ref[long_idx];
+                        if(ref && (ref->valid_structure & pic_structure)){
                             ref->pic_id= pic_id;
                             assert(ref->reference == 3);
                             assert(ref->long_ref);
                             i=0;
                         }else{
@@ -2947,10 +3136,12 @@
                         }
                         for(; i > index; i--){
                             h->ref_list[list][i]= h->ref_list[list][i-1];
                         }
                         h->ref_list[list][index]= *ref;
+                        if (FIELD_PICTURE)
+                            pic_set_as_field(&h->ref_list[list][index], add);
                     }
                 }else{
                     av_log(h->s.avctx, AV_LOG_ERROR, "illegal reordering_of_pic_nums_idc\n");
                     return -1;
                 }
@@ -3088,10 +3279,11 @@
 }
 
 static inline void unreference_pic(H264Context *h, Picture *pic){
     int i;
     pic->reference=0;
+    pic->valid_structure=0;
     if(pic == h->delayed_output_pic)
         pic->reference=1;
     else{
         for(i = 0; h->delayed_pic[i]; i++)
             if(pic == h->delayed_pic[i]){
@@ -3135,47 +3327,94 @@
         h->delayed_output_pic->reference= 0;
     h->delayed_output_pic= NULL;
     idr(h);
     if(h->s.current_picture_ptr)
         h->s.current_picture_ptr->reference= 0;
+    h->s.first_field= 0;
 }
 
 /**
+ * Removes a Picture from the short term reference list by picture number.
+ * Frame or field operation is designated by frame_mask.
+ * For frames, the frame is simply removed from the reference list.
+ * For fields, the designated field is marked as unref. If the field
+ * removal causes both fields of the frame to become unreferenced, the field
+ * (pair) is removed from the short term list.
  *
- * @return the removed picture or NULL if an error occurs
+ * @param field_mask if non-null, points to a mask to apply to field reference
+ *                   value (PICT_XXX_FIELD) designating which field to remove
+ *                   from short reference. If a field for pic_num is found,
+ *                   and, after unreferencing the masked field, the pair's
+ *                   other field is still in reference, this is set to zero.
+ * @param pic_num picture number for pic to remove. Interpreted as field
+ *                picture number if field_mask != NULL
+ * @return the removed picture, or NULL if a picture with pic_num was
+ *               not found or a field_mask was supplied and, after
+ *               unreferencing the masked field, the frame's other field
+ *               is still in reference.
  */
-static Picture * remove_short(H264Context *h, int frame_num){
+static Picture * remove_short(H264Context *h, int pic_num, int *field_mask){
     MpegEncContext * const s = &h->s;
     int i;
+    int frame_num = pic_num >> (field_mask != NULL);
 
     if(s->avctx->debug&FF_DEBUG_MMCO)
-        av_log(h->s.avctx, AV_LOG_DEBUG, "remove short %d count %d\n", frame_num, h->short_ref_count);
+        av_log(h->s.avctx, AV_LOG_DEBUG, "remove short %d count %d\n", pic_num, h->short_ref_count);
 
     for(i=0; i<h->short_ref_count; i++){
         Picture *pic= h->short_ref[i];
         if(s->avctx->debug&FF_DEBUG_MMCO)
             av_log(h->s.avctx, AV_LOG_DEBUG, "%d %d %p\n", i, pic->frame_num, pic);
         if(pic->frame_num == frame_num){
+            if (field_mask && (pic->valid_structure &= *field_mask) != 0) {
+                *field_mask = 0;
+                return NULL;
+            }
+
             h->short_ref[i]= NULL;
             if (--h->short_ref_count)
                 memmove(&h->short_ref[i], &h->short_ref[i+1], (h->short_ref_count - i)*sizeof(Picture*));
             return pic;
         }
     }
     return NULL;
 }
 
 /**
+ * Removes a Picture from the long term reference list by picture number.
+ * Frame or field operation is designated by frame_mask.
+ * For frames, the frame is simply removed from the reference list.
+ * For fields, the designated field is marked as unref. If the field
+ * removal causes both fields of the frame to become unreferenced, the field
+ * (pair) is removed from the short term list.
  *
- * @return the removed picture or NULL if an error occurs
+ * @param field_mask if non-null, points to a mask to apply to field reference
+ *                   value (PICT_XXX_FIELD) designating which field to remove
+ *                   from long reference. If a field for pic_num is found,
+ *                   and, after unreferencing the masked field, the pair's
+ *                   other field is still in reference, this is set to zero.
+ * @param pic_num picture number for pic to remove. Interpreted as field
+ *                picture number if field_mask != NULL
+ * @return the removed picture, or NULL if a picture with pic_num was
+ *               not found or a field_mask was supplied and, after
+ *               unreferencing the masked field, the picture's other field
+ *               is still in reference.
  */
-static Picture * remove_long(H264Context *h, int i){
+static Picture * remove_long(H264Context *h, int pic_num, int *field_mask){
     Picture *pic;
+    int i = pic_num >> (field_mask != NULL);
 
     pic= h->long_ref[i];
-    h->long_ref[i]= NULL;
-    if(pic) h->long_ref_count--;
+    if (pic) {
+        if (field_mask && (pic->valid_structure &= *field_mask) != 0) {
+            *field_mask = 0;
+            return NULL;
+        }
+
+        h->long_ref[i]= NULL;
+        h->long_ref_count--;
+    }
 
     return pic;
 }
 
 /**
@@ -3212,11 +3451,11 @@
  * Executes the reference picture marking (memory management control operations).
  */
 static int execute_ref_pic_marking(H264Context *h, MMCO *mmco, int mmco_count){
     MpegEncContext * const s = &h->s;
     int i, j;
-    int current_is_long=0;
+    int current_ref_assigned=0;
     Picture *pic;
 
     if((s->avctx->debug&FF_DEBUG_MMCO) && mmco_count==0)
         av_log(h->s.avctx, AV_LOG_DEBUG, "no mmco here\n");
 
@@ -3224,67 +3463,141 @@
         if(s->avctx->debug&FF_DEBUG_MMCO)
             av_log(h->s.avctx, AV_LOG_DEBUG, "mmco:%d %d %d\n", h->mmco[i].opcode, h->mmco[i].short_pic_num, h->mmco[i].long_arg);
 
         switch(mmco[i].opcode){
         case MMCO_SHORT2UNUSED:
-            pic= remove_short(h, mmco[i].short_pic_num);
+            if (FIELD_PICTURE) {
+                j= (mmco[i].short_pic_num & 1) ?
+                         ~s->picture_structure : s->picture_structure;
+                pic= remove_short(h, mmco[i].short_pic_num, &j);
+            } else {
+                j= 1;
+                pic= remove_short(h, mmco[i].short_pic_num, NULL);
+            }
+
             if(pic)
                 unreference_pic(h, pic);
-            else if(s->avctx->debug&FF_DEBUG_MMCO)
+            else if(s->avctx->debug&FF_DEBUG_MMCO && j != 0)
                 av_log(h->s.avctx, AV_LOG_DEBUG, "mmco: remove_short() failure\n");
             break;
         case MMCO_SHORT2LONG:
-            pic= remove_long(h, mmco[i].long_arg);
-            if(pic) unreference_pic(h, pic);
+            if (FIELD_PICTURE && mmco[i].long_arg < h->long_ref_count &&
+                    h->long_ref[mmco[i].long_arg]->frame_num ==
+                                              mmco[i].short_pic_num / 2) {
+                /* do nothing, we've already moved this field pair. */
+            } else {
+                int frame_num = mmco[i].short_pic_num >> FIELD_PICTURE;
 
-            h->long_ref[ mmco[i].long_arg ]= remove_short(h, mmco[i].short_pic_num);
-            if (h->long_ref[ mmco[i].long_arg ]){
-                h->long_ref[ mmco[i].long_arg ]->long_ref=1;
-                h->long_ref_count++;
+                pic= remove_long(h, mmco[i].long_arg, NULL);
+                if(pic) unreference_pic(h, pic);
+
+                h->long_ref[ mmco[i].long_arg ]= remove_short(h, frame_num, NULL);
+                if (h->long_ref[ mmco[i].long_arg ]){
+                    h->long_ref[ mmco[i].long_arg ]->long_ref=1;
+                    h->long_ref_count++;
+                }
             }
             break;
         case MMCO_LONG2UNUSED:
-            pic= remove_long(h, mmco[i].long_arg);
+            if (FIELD_PICTURE) {
+                j = (mmco[i].long_arg & 1) ?
+                        ~s->picture_structure : s->picture_structure;
+                pic = remove_long(h, mmco[i].long_arg, &j);
+            } else {
+                j = 1;
+                pic= remove_long(h, mmco[i].long_arg, NULL);
+            }
+
             if(pic)
                 unreference_pic(h, pic);
-            else if(s->avctx->debug&FF_DEBUG_MMCO)
+            else if(s->avctx->debug&FF_DEBUG_MMCO && j != 0)
                 av_log(h->s.avctx, AV_LOG_DEBUG, "mmco: remove_long() failure\n");
             break;
         case MMCO_LONG:
-            pic= remove_long(h, mmco[i].long_arg);
-            if(pic) unreference_pic(h, pic);
+            j = 1;
+            if (FIELD_PICTURE && !s->first_field) {
+                if (h->long_ref[mmco[i].long_arg] == s->current_picture_ptr) {
+                    /* Just mark second field as referenced */
+                    j = 0;
+                } else if (s->current_picture_ptr->valid_structure) {
+                    /* First field in pair in short term list.
+                     * This is not allowed; see 7.4.3, notes 2 and 3.
+                     */
+                    av_log(h->s.avctx, AV_LOG_ERROR,
+                        "illegal long term reference assignment for second "
+                        "field in complementary field pair (first field is "
+                        "short term)\n");
+                    j = 0;
+                }
+            }
 
-            h->long_ref[ mmco[i].long_arg ]= s->current_picture_ptr;
-            h->long_ref[ mmco[i].long_arg ]->long_ref=1;
-            h->long_ref_count++;
+            if (j) {
+                pic= remove_long(h, mmco[i].long_arg, NULL);
+                if(pic) unreference_pic(h, pic);
 
-            current_is_long=1;
+                h->long_ref[ mmco[i].long_arg ]= s->current_picture_ptr;
+                h->long_ref[ mmco[i].long_arg ]->long_ref=1;
+                h->long_ref_count++;
+            }
+
+            s->current_picture_ptr->valid_structure |= s->picture_structure;
+            current_ref_assigned=1;
             break;
         case MMCO_SET_MAX_LONG:
             assert(mmco[i].long_arg <= 16);
             // just remove the long term which index is greater than new max
             for(j = mmco[i].long_arg; j<16; j++){
-                pic = remove_long(h, j);
+                pic = remove_long(h, j, NULL);
                 if (pic) unreference_pic(h, pic);
             }
             break;
         case MMCO_RESET:
             while(h->short_ref_count){
-                pic= remove_short(h, h->short_ref[0]->frame_num);
+                pic= remove_short(h, h->short_ref[0]->frame_num, NULL);
                 if(pic) unreference_pic(h, pic);
             }
             for(j = 0; j < 16; j++) {
-                pic= remove_long(h, j);
+                pic= remove_long(h, j, NULL);
                 if(pic) unreference_pic(h, pic);
             }
             break;
         default: assert(0);
         }
     }
 
-    if(!current_is_long){
-        pic= remove_short(h, s->current_picture_ptr->frame_num);
+    if (!current_ref_assigned && FIELD_PICTURE &&
+            !s->first_field && s->current_picture_ptr->valid_structure) {
+
+        /* Second field of complementary field pair; the first field of
+         * which is already referenced. If short referenced, it
+         * should be first entry in short_ref. If not, it must exist
+         * in long_ref; trying to put it on the short list here is an
+         * error in the encoded bit stream (ref: 7.4.3, NOTE 2 and 3).
+         * If on neither list, we have a logic problem elsewhere
+         */
+        if (h->short_ref_count && h->short_ref[0] == s->current_picture_ptr) {
+            /* Just mark the second field valid */
+            s->current_picture_ptr->valid_structure = PICT_FRAME;
+            current_ref_assigned = 1;
+        } else if (s->current_picture_ptr->long_ref) {
+            av_log(h->s.avctx, AV_LOG_ERROR, "illegal short term reference "
+                                             "assignment for second field "
+                                             "in complementary field pair "
+                                             "(first field is long term)\n");
+            current_ref_assigned = 1;
+        } else {
+            av_log(h->s.avctx, AV_LOG_ERROR, "problem in internal reference "
+                                             "list handling; second field in "
+                                             "pair claims first field is "
+                                             "reference, but not in any ref "
+                                             "list\n");
+            s->current_picture_ptr->valid_structure = 0;
+        }
+    }
+
+    if(!current_ref_assigned){
+        pic= remove_short(h, s->current_picture_ptr->frame_num, NULL);
         if(pic){
             unreference_pic(h, pic);
             av_log(h->s.avctx, AV_LOG_ERROR, "illegal short term buffer state detected\n");
         }
 
@@ -3292,10 +3605,11 @@
             memmove(&h->short_ref[1], &h->short_ref[0], h->short_ref_count*sizeof(Picture*));
 
         h->short_ref[0]= s->current_picture_ptr;
         h->short_ref[0]->long_ref=0;
         h->short_ref_count++;
+        s->current_picture_ptr->valid_structure |= s->picture_structure;
     }
 
     print_short_term(h);
     print_long_term(h);
     return 0;
@@ -3319,23 +3633,23 @@
             for(i= 0; i<MAX_MMCO_COUNT; i++) {
                 MMCOOpcode opcode= get_ue_golomb(gb);
 
                 h->mmco[i].opcode= opcode;
                 if(opcode==MMCO_SHORT2UNUSED || opcode==MMCO_SHORT2LONG){
-                    h->mmco[i].short_pic_num= (h->frame_num - get_ue_golomb(gb) - 1) & ((1<<h->sps.log2_max_frame_num)-1); //FIXME fields
-/*                    if(h->mmco[i].short_pic_num >= h->short_ref_count || h->short_ref[ h->mmco[i].short_pic_num ] == NULL){
+                    h->mmco[i].short_pic_num= h->curr_pic_num - get_ue_golomb(gb) - 1;
+/*                    if(h->mmco[i].short_frame_num >= h->short_ref_count || h->short_ref[ h->mmco[i].short_frame_num ] == NULL){
                         av_log(s->avctx, AV_LOG_ERROR, "illegal short ref in memory management control operation %d\n", mmco);
                         return -1;
                     }*/
                 }
                 if(opcode==MMCO_SHORT2LONG || opcode==MMCO_LONG2UNUSED || opcode==MMCO_LONG || opcode==MMCO_SET_MAX_LONG){
-                    unsigned int long_index= get_ue_golomb(gb);
-                    if(/*h->mmco[i].long_arg >= h->long_ref_count || h->long_ref[ h->mmco[i].long_arg ] == NULL*/ long_index >= 16){
+                    unsigned int long_arg= get_ue_golomb(gb);
+                    if(long_arg >= 32 || (long_arg >= 16 && !(opcode == MMCO_LONG2UNUSED && FIELD_PICTURE))){
                         av_log(h->s.avctx, AV_LOG_ERROR, "illegal long ref in memory management control operation %d\n", opcode);
                         return -1;
                     }
-                    h->mmco[i].long_arg= long_index;
+                    h->mmco[i].long_arg= long_arg;
                 }
 
                 if(opcode > (unsigned)MMCO_LONG){
                     av_log(h->s.avctx, AV_LOG_ERROR, "illegal memory management control operation %d\n", opcode);
                     return -1;
@@ -3345,14 +3659,21 @@
             }
             h->mmco_index= i;
         }else{
             assert(h->long_ref_count + h->short_ref_count <= h->sps.ref_frame_count);
 
-            if(h->long_ref_count + h->short_ref_count == h->sps.ref_frame_count){ //FIXME fields
+            if(h->long_ref_count + h->short_ref_count == h->sps.ref_frame_count &&
+                    !(FIELD_PICTURE && !s->first_field && s->current_picture_ptr->valid_structure)) {
                 h->mmco[0].opcode= MMCO_SHORT2UNUSED;
                 h->mmco[0].short_pic_num= h->short_ref[ h->short_ref_count - 1 ]->frame_num;
                 h->mmco_index= 1;
+                if (FIELD_PICTURE) {
+                    h->mmco[0].short_pic_num *= 2;
+                    h->mmco[1].opcode= MMCO_SHORT2UNUSED;
+                    h->mmco[1].short_pic_num= h->mmco[0].short_pic_num + 1;
+                    h->mmco_index= 2;
+                }
             }else
                 h->mmco_index= 0;
         }
     }
 
@@ -3436,15 +3757,19 @@
         }
         field_poc[0]= poc;
         field_poc[1]= poc;
     }
 
-    if(s->picture_structure != PICT_BOTTOM_FIELD)
+    if(s->picture_structure != PICT_BOTTOM_FIELD) {
         s->current_picture_ptr->field_poc[0]= field_poc[0];
-    if(s->picture_structure != PICT_TOP_FIELD)
+        s->current_picture_ptr->poc = field_poc[0];
+    }
+    if(s->picture_structure != PICT_TOP_FIELD) {
         s->current_picture_ptr->field_poc[1]= field_poc[1];
-    if(s->picture_structure == PICT_FRAME) // FIXME field pix?
+        s->current_picture_ptr->poc = field_poc[1];
+    }
+    if(!FIELD_PICTURE || !s->first_field)
         s->current_picture_ptr->poc= FFMIN(field_poc[0], field_poc[1]);
 
     return 0;
 }
 
@@ -3506,10 +3831,11 @@
     memcpy(dst->block_offset,     src->block_offset, sizeof(dst->block_offset));
     dst->s.current_picture_ptr  = src->s.current_picture_ptr;
     dst->s.current_picture      = src->s.current_picture;
     dst->s.linesize             = src->s.linesize;
     dst->s.uvlinesize           = src->s.uvlinesize;
+    dst->s.first_field          = src->s.first_field;
 
     dst->prev_poc_msb           = src->prev_poc_msb;
     dst->prev_poc_lsb           = src->prev_poc_lsb;
     dst->prev_frame_num_offset  = src->prev_frame_num_offset;
     dst->prev_frame_num         = src->prev_frame_num;
@@ -3539,19 +3865,21 @@
     unsigned int pps_id;
     int num_ref_idx_active_override_flag;
     static const uint8_t slice_type_map[5]= {P_TYPE, B_TYPE, I_TYPE, SP_TYPE, SI_TYPE};
     unsigned int slice_type, tmp, i;
     int default_ref_list_done = 0;
+    int last_pic_structure;
 
     s->current_picture.reference= h->nal_ref_idc != 0;
     s->dropable= h->nal_ref_idc == 0;
 
     first_mb_in_slice= get_ue_golomb(&s->gb);
 
     if((s->flags2 & CODEC_FLAG2_CHUNKS) && first_mb_in_slice == 0){
         h0->current_slice = 0;
-        s->current_picture_ptr= NULL;
+        if (!s->first_slice)
+            s->current_picture_ptr= NULL;
     }
 
     slice_type= get_ue_golomb(&s->gb);
     if(slice_type > 9){
         av_log(h->s.avctx, AV_LOG_ERROR, "slice type too large (%d) at %d %d\n", h->slice_type, s->mb_x, s->mb_y);
@@ -3616,10 +3944,11 @@
     if (!s->context_initialized) {
         if(h != h0)
             return -1;  // we cant (re-)initialize context during parallel decoding
         if (MPV_common_init(s) < 0)
             return -1;
+        s->first_field = 0;
 
         init_scan_tables(h);
         alloc_tables(h);
 
         for(i = 1; i < s->avctx->thread_count; i++) {
@@ -3650,48 +3979,100 @@
             av_reduce(&s->avctx->time_base.num, &s->avctx->time_base.den,
                       s->avctx->time_base.num, s->avctx->time_base.den, 1<<30);
         }
     }
 
-    if(h0->current_slice == 0){
-        if(frame_start(h) < 0)
-            return -1;
-    }
-    if(h != h0)
-        clone_slice(h, h0);
-
-    s->current_picture_ptr->frame_num= //FIXME frame_num cleanup
     h->frame_num= get_bits(&s->gb, h->sps.log2_max_frame_num);
 
     h->mb_mbaff = 0;
     h->mb_aff_frame = 0;
+    last_pic_structure = s->picture_structure;
     if(h->sps.frame_mbs_only_flag){
         s->picture_structure= PICT_FRAME;
     }else{
         if(get_bits1(&s->gb)) { //field_pic_flag
             s->picture_structure= PICT_TOP_FIELD + get_bits1(&s->gb); //bottom_field_flag
-            av_log(h->s.avctx, AV_LOG_ERROR, "PAFF interlacing is not implemented\n");
         } else {
             s->picture_structure= PICT_FRAME;
             h->mb_aff_frame = h->sps.mb_aff;
         }
     }
+
+    if(h0->current_slice == 0){
+        /* figure out if we have a complementary field pair */
+        if (s->first_field) {
+            assert(s->current_picture_ptr && s->current_picture_ptr->data[0]);
+
+            if (s->picture_structure == PICT_FRAME ||
+                    s->picture_structure == last_pic_structure) {
+                /*
+                 * Unmatched field pair. Don't display it, but use
+                 * for reference if so marked
+                 */
+                if (s->current_picture_ptr->reference == 1)
+                    s->current_picture_ptr->reference = 0;
+                s->current_picture_ptr = NULL;
+
+                s->first_field = FIELD_PICTURE;
+
+            } else {
+                if (s->current_picture.reference &&
+                        s->current_picture_ptr->reference == 3 &&
+                        s->current_picture_ptr->frame_num != h->frame_num) {
+                    /*
+                     * This and previous field were reference, but had
+                     * different frame_nums.  Dump first field (keep in
+                     * reference list) and consider this as first field.
+                     */
+                    s->first_field = 1;
+                    s->current_picture_ptr = NULL;
+
+                } else {
+                    /* Second field in complementary pair */
+                    if (s->current_picture.reference)
+                        s->current_picture_ptr->reference = 3;
+                    s->first_field = 0;
+                }
+            }
+
+        } else {
+            /* Frame or first field in a potentially complementary pair */
+            assert(!s->current_picture_ptr);
+            s->first_field = FIELD_PICTURE;
+        }
+
+        if((!FIELD_PICTURE || s->first_field) && frame_start(h) < 0) {
+            s->first_field = 0;
+            return -1;
+        }
+    }
+
+    if(h != h0)
+        clone_slice(h, h0);
+
+    s->current_picture_ptr->frame_num= h->frame_num; //FIXME frame_num cleanup
+
     assert(s->mb_num == s->mb_width * s->mb_height);
     if(first_mb_in_slice << h->mb_aff_frame >= s->mb_num ||
        first_mb_in_slice                    >= s->mb_num){
         av_log(h->s.avctx, AV_LOG_ERROR, "first_mb_in_slice overflow\n");
         return -1;
     }
     s->resync_mb_x = s->mb_x = first_mb_in_slice % s->mb_width;
     s->resync_mb_y = s->mb_y = (first_mb_in_slice / s->mb_width) << h->mb_aff_frame;
+    if (FIELD_PICTURE) {
+        s->resync_mb_y = s->mb_y = s->mb_y * 2;
+        if (s->picture_structure == PICT_BOTTOM_FIELD)
+            s->resync_mb_y = s->mb_y = s->mb_y + 1;
+    }
     assert(s->mb_y < s->mb_height);
 
     if(s->picture_structure==PICT_FRAME){
         h->curr_pic_num=   h->frame_num;
         h->max_pic_num= 1<< h->sps.log2_max_frame_num;
     }else{
-        h->curr_pic_num= 2*h->frame_num;
+        h->curr_pic_num= 2*h->frame_num + 1;
         h->max_pic_num= 1<<(h->sps.log2_max_frame_num + 1);
     }
 
     if(h->nal_unit_type == NAL_IDR_SLICE){
         get_ue_golomb(&s->gb); /* idr_pic_id */
@@ -3723,11 +4104,11 @@
     h->ref_count[1]= h->pps.ref_count[1];
 
     if(h->slice_type == P_TYPE || h->slice_type == SP_TYPE || h->slice_type == B_TYPE){
         if(h->slice_type == B_TYPE){
             h->direct_spatial_mv_pred= get_bits1(&s->gb);
-            if(h->sps.mb_aff && h->direct_spatial_mv_pred)
+            if(h->mb_aff_frame && h->direct_spatial_mv_pred)
                 av_log(h->s.avctx, AV_LOG_ERROR, "MBAFF + spatial direct mode is not implemented\n");
         }
         num_ref_idx_active_override_flag= get_bits1(&s->gb);
 
         if(num_ref_idx_active_override_flag){
@@ -3843,11 +4224,11 @@
 
     h0->last_slice_type = slice_type;
     h->slice_num = ++h0->current_slice;
 
     h->emu_edge_width= (s->flags&CODEC_FLAG_EMU_EDGE) ? 0 : 16;
-    h->emu_edge_height= FRAME_MBAFF ? 0 : h->emu_edge_width;
+    h->emu_edge_height= (FRAME_MBAFF || FIELD_PICTURE) ? 0 : h->emu_edge_width;
 
     if(s->avctx->debug&FF_DEBUG_PICT_INFO){
         av_log(h->s.avctx, AV_LOG_DEBUG, "slice:%d %s mb:%d %c pps:%u frame:%d poc:%d/%d ref:%d/%d qp:%d loop:%d:%d:%d weight:%d%s\n",
                h->slice_num,
                (s->picture_structure==PICT_FRAME ? "F" : s->picture_structure==PICT_TOP_FIELD ? "T" : "B"),
@@ -4741,10 +5122,12 @@
             mbb_xy = mb_x + (mb_y-1)*s->mb_stride;
     }else{
         int mb_xy = mb_x + mb_y*s->mb_stride;
         mba_xy = mb_xy - 1;
         mbb_xy = mb_xy - s->mb_stride;
+        if (FIELD_PICTURE)
+            mbb_xy -= s->mb_stride;
     }
 
     if( h->slice_table[mba_xy] == h->slice_num && !IS_SKIP( s->current_picture.mb_type[mba_xy] ))
         ctx++;
     if( h->slice_table[mbb_xy] == h->slice_num && !IS_SKIP( s->current_picture.mb_type[mbb_xy] ))
@@ -4873,19 +5256,13 @@
     if( cbp_a == 2 ) ctx++;
     if( cbp_b == 2 ) ctx += 2;
     return 1 + get_cabac_noinline( &h->cabac, &h->cabac_state[77 + ctx] );
 }
 static int decode_cabac_mb_dqp( H264Context *h) {
-    MpegEncContext * const s = &h->s;
-    int mbn_xy;
     int   ctx = 0;
     int   val = 0;
 
-    if( s->mb_x > 0 )
-        mbn_xy = s->mb_x + s->mb_y*s->mb_stride - 1;
-    else
-        mbn_xy = s->mb_width - 1 + (s->mb_y-1)*s->mb_stride;
 
     if( h->last_qscale_diff != 0 )
         ctx++;
 
     while( get_cabac_noinline( &h->cabac, &h->cabac_state[60 + ctx] ) ) {
@@ -5236,10 +5613,12 @@
             h->top_mb_xy -= s->mb_stride;
         }
         if (left_mb_frame_flag != curr_mb_frame_flag) {
             h->left_mb_xy[0] = pair_xy - 1;
         }
+    } else if (FIELD_PICTURE) {
+        h->top_mb_xy -= s->mb_stride;
     }
     return;
 }
 
 /**
@@ -6467,11 +6846,11 @@
 
             if( ++s->mb_x >= s->mb_width ) {
                 s->mb_x = 0;
                 ff_draw_horiz_band(s, 16*s->mb_y, 16);
                 ++s->mb_y;
-                if(FRAME_MBAFF) {
+                if(FRAME_MBAFF || FIELD_PICTURE) {
                     ++s->mb_y;
                 }
             }
 
             if( eos || s->mb_y >= s->mb_height ) {
@@ -6504,11 +6883,11 @@
 
             if(++s->mb_x >= s->mb_width){
                 s->mb_x=0;
                 ff_draw_horiz_band(s, 16*s->mb_y, 16);
                 ++s->mb_y;
-                if(FRAME_MBAFF) {
+                if(FRAME_MBAFF || FIELD_PICTURE) {
                     ++s->mb_y;
                 }
                 if(s->mb_y >= s->mb_height){
                     tprintf(s->avctx, "slice end %d %d\n", get_bits_count(&s->gb), s->gb.size_in_bits);
 
@@ -7103,11 +7482,12 @@
         av_log(NULL, AV_LOG_ERROR,"%02X ", buf[i]);
     }
 #endif
     if(!(s->flags2 & CODEC_FLAG2_CHUNKS)){
         h->current_slice = 0;
-        s->current_picture_ptr= NULL;
+        if (!s->first_field)
+            s->current_picture_ptr= NULL;
     }
 
     for(;;){
         int consumed;
         int dst_length;
@@ -7400,18 +7780,39 @@
         s->current_picture_ptr->qscale_type= FF_QSCALE_TYPE_H264;
         s->current_picture_ptr->pict_type= s->pict_type;
 
         h->prev_frame_num_offset= h->frame_num_offset;
         h->prev_frame_num= h->frame_num;
-        if(s->current_picture_ptr->reference){
+        if(s->current_picture.reference){
+            /*
+             * The above tests current_picture (not _ptr) to get reference
+             * mark of the actual picture. For fields, this may be different
+             * than the field pair (_ptr)
+             */
             h->prev_poc_msb= h->poc_msb;
             h->prev_poc_lsb= h->poc_lsb;
-        }
-        if(s->current_picture_ptr->reference)
             execute_ref_pic_marking(h, h->mmco, h->mmco_index);
+        }
 
-        ff_er_frame_end(s);
+        if (s->first_field) {
+            /* Wait for second field. */
+            *data_size = 0;
+        } else {
+            /*
+             * FIXME: Error handling code does not seem to support interlaced
+             * when slices span multiple rows
+             * The ff_er_add_slice calls don't work right for bottom
+             * fields; they cause massive erroneous error concealing
+             * Error marking covers both fields (top and bottom).
+             * This causes a mismatched s->error_count
+             * and a bad error table. Further, the error count goes to
+             * INT_MAX when called for bottom field, because mb_y is
+             * past end by one (callers fault) and resync_mb_y != 0
+             * causes problems for the first MB line, too.
+             */
+            if (!FIELD_PICTURE)
+                ff_er_frame_end(s);
 
         MPV_frame_end(s);
 
     //FIXME do something with unavailable reference frames
 
@@ -7481,10 +7882,11 @@
 
         if(out)
             *pict= *(AVFrame*)out;
         else
             av_log(avctx, AV_LOG_DEBUG, "no picture\n");
+        }
     }
 
     assert(pict->data[0] || !*data_size);
     ff_print_debug_info(s, pict);
 //printf("out %d\n", (int)pict->data[0]);
--- ../ffmpeg-cosmetics/libavcodec/h264.h	2007-09-18 17:35:33.000000000 -0400
+++ libavcodec/h264.h	2007-09-20 13:23:44.000000000 -0400
@@ -57,14 +57,16 @@
 
 #ifdef ALLOW_INTERLACE
 #define MB_MBAFF h->mb_mbaff
 #define MB_FIELD h->mb_field_decoding_flag
 #define FRAME_MBAFF h->mb_aff_frame
+#define FIELD_PICTURE (s->picture_structure != PICT_FRAME)
 #else
 #define MB_MBAFF 0
 #define MB_FIELD 0
 #define FRAME_MBAFF 0
+#define FIELD_PICTURE 0
 #undef  IS_INTERLACED
 #define IS_INTERLACED(mb_type) 0
 #endif
 
 /**
@@ -281,11 +283,11 @@
     int frame_num_offset;         ///< for POC type 2
     int prev_frame_num_offset;    ///< for POC type 2
     int prev_frame_num;           ///< frame_num of the last pic for POC type 1/2
 
     /**
-     * frame_num for frames or 2*frame_num for field pics.
+     * frame_num for frames or 2*frame_num+1 for field pics.
      */
     int curr_pic_num;
 
     /**
      * max_frame_num or 2*max_frame_num for field pics.
--- ../ffmpeg-cosmetics/libavcodec/mpegvideo.c	2007-09-18 17:35:33.000000000 -0400
+++ libavcodec/mpegvideo.c	2007-09-20 13:23:44.000000000 -0400
@@ -947,11 +947,11 @@
         goto alloc;
     }
 
     assert(s->pict_type == I_TYPE || (s->last_picture_ptr && s->last_picture_ptr->data[0]));
 
-    if(s->picture_structure!=PICT_FRAME){
+    if(s->picture_structure!=PICT_FRAME && s->out_format != FMT_H264){
         int i;
         for(i=0; i<4; i++){
             if(s->picture_structure == PICT_BOTTOM_FIELD){
                  s->current_picture.data[i] += s->current_picture.linesize[i];
             }
--- ../ffmpeg-cosmetics/libavcodec/mpegvideo.h	2007-09-18 17:35:33.000000000 -0400
+++ libavcodec/mpegvideo.h	2007-09-20 13:23:44.000000000 -0400
@@ -136,10 +136,11 @@
 #define USES_LIST(a, list) ((a) & ((MB_TYPE_P0L0|MB_TYPE_P1L0)<<(2*(list)))) ///< does this mb use listX, note does not work if subMBs
 #define HAS_CBP(a)        ((a)&MB_TYPE_CBP)
 
     int field_poc[2];           ///< h264 top/bottom POC
     int poc;                    ///< h264 frame POC
+    int valid_structure;        ///< h264 one of PICT_XXXX, stating which fields are referenced
     int frame_num;              ///< h264 frame_num (raw frame_num from slice header)
     int pic_id;                 ///< h264 pic_num (short or long term)
     int long_ref;               ///< 1->long term reference 0->short term reference
     int ref_poc[2][16];         ///< h264 POCs of the frames used as reference
     int ref_count[2];           ///< number of entries in ref_poc



More information about the ffmpeg-devel mailing list