[FFmpeg-cvslog] avcodec/h264: be more tolerant on what pixel format changes trigger reinits

[Michael Niedermayer]

ffmpeg | branch: release/2.2 | Michael Niedermayer <michaelni at gmx.at> | Mon Mar 17 02:25:20 2014 +0100| [d8fe6957796a95f936d7b16db559a164db7e9de4] | committer: Carl Eugen Hoyos

avcodec/h264: be more tolerant on what pixel format changes trigger reinits

Fixes Ticket3260

Signed-off-by: Michael Niedermayer <michaelni at gmx.at>
(cherry picked from commit 8e92ff25469f75f5c1fcbb9ba5721cea341ca34a)

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=d8fe6957796a95f936d7b16db559a164db7e9de4
---

 libavcodec/h264.c |   13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/libavcodec/h264.c b/libavcodec/h264.c
index 9379b2f..ec3601f 100644
--- a/libavcodec/h264.c
+++ b/libavcodec/h264.c
@@ -3444,6 +3444,17 @@ int ff_set_ref_count(H264Context *h)
     return 0;
 }
 
+static enum AVPixelFormat non_j_pixfmt(enum AVPixelFormat a)
+{
+    switch (a) {
+    case AV_PIX_FMT_YUVJ420P: return AV_PIX_FMT_YUV420P;
+    case AV_PIX_FMT_YUVJ422P: return AV_PIX_FMT_YUV422P;
+    case AV_PIX_FMT_YUVJ444P: return AV_PIX_FMT_YUV444P;
+    default:
+        return a;
+    }
+}
+
 /**
  * Decode a slice header.
  * This will (re)intialize the decoder and call h264_frame_start() as needed.
@@ -3573,7 +3584,7 @@ static int decode_slice_header(H264Context *h, H264Context *h0)
                      || h->mb_width  != h->sps.mb_width
                      || h->mb_height != h->sps.mb_height * (2 - h->sps.frame_mbs_only_flag)
                     ));
-    if (h0->avctx->pix_fmt != get_pixel_format(h0, 0))
+    if (non_j_pixfmt(h0->avctx->pix_fmt) != non_j_pixfmt(get_pixel_format(h0, 0)))
         must_reinit = 1;
 
     h->mb_width  = h->sps.mb_width;