[FFmpeg-cvslog] avcodec/h264: be more tolerant on what pixel format changes trigger reinits

[Michael Niedermayer]

ffmpeg | branch: release/2.1 | Michael Niedermayer <michaelni at gmx.at> | Mon Mar 17 02:25:20 2014 +0100| [9098da032975e9792971125ff93e550d853b9397] | committer: Carl Eugen Hoyos

avcodec/h264: be more tolerant on what pixel format changes trigger reinits

Fixes Ticket3260

Signed-off-by: Michael Niedermayer <michaelni at gmx.at>
(cherry picked from commit 8e92ff25469f75f5c1fcbb9ba5721cea341ca34a)

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=9098da032975e9792971125ff93e550d853b9397
---

 libavcodec/h264.c |   13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/libavcodec/h264.c b/libavcodec/h264.c
index 6c2efba..b3593c6 100644
--- a/libavcodec/h264.c
+++ b/libavcodec/h264.c
@@ -3370,6 +3370,17 @@ int ff_set_ref_count(H264Context *h)
     return 0;
 }
 
+static enum AVPixelFormat non_j_pixfmt(enum AVPixelFormat a)
+{
+    switch (a) {
+    case AV_PIX_FMT_YUVJ420P: return AV_PIX_FMT_YUV420P;
+    case AV_PIX_FMT_YUVJ422P: return AV_PIX_FMT_YUV422P;
+    case AV_PIX_FMT_YUVJ444P: return AV_PIX_FMT_YUV444P;
+    default:
+        return a;
+    }
+}
+
 /**
  * Decode a slice header.
  * This will also call ff_MPV_common_init() and frame_start() as needed.
@@ -3492,7 +3503,7 @@ static int decode_slice_header(H264Context *h, H264Context *h0)
                      || h->mb_width  != h->sps.mb_width
                      || h->mb_height != h->sps.mb_height * (2 - h->sps.frame_mbs_only_flag)
                     ));
-    if (h0->avctx->pix_fmt != get_pixel_format(h0, 0))
+    if (non_j_pixfmt(h0->avctx->pix_fmt) != non_j_pixfmt(get_pixel_format(h0, 0)))
         must_reinit = 1;
 
     h->mb_width  = h->sps.mb_width;