[FFmpeg-cvslog] avcodec/h264: be more tolerant on what pixel format changes trigger reinits

[Michael Niedermayer]

ffmpeg | branch: release/1.2 | Michael Niedermayer <michaelni at gmx.at> | Mon Mar 17 02:25:20 2014 +0100| [e3ce3d373caf29d457d0eb02c707799d141cbd4c] | committer: Carl Eugen Hoyos

avcodec/h264: be more tolerant on what pixel format changes trigger reinits

Fixes Ticket3260

Signed-off-by: Michael Niedermayer <michaelni at gmx.at>
(cherry picked from commit 8e92ff25469f75f5c1fcbb9ba5721cea341ca34a)

Conflicts:
	libavcodec/h264.c

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=e3ce3d373caf29d457d0eb02c707799d141cbd4c
---

 libavcodec/h264.c |   14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)

diff --git a/libavcodec/h264.c b/libavcodec/h264.c
index 001882b..8c0544a 100644
--- a/libavcodec/h264.c
+++ b/libavcodec/h264.c
@@ -3094,6 +3094,18 @@ static int h264_slice_header_init(H264Context *h, int reinit)
     return 0;
 }
 
+static enum AVPixelFormat non_j_pixfmt(enum AVPixelFormat a)
+{
+    switch (a) {
+    case AV_PIX_FMT_YUVJ420P: return AV_PIX_FMT_YUV420P;
+    case AV_PIX_FMT_YUVJ422P: return AV_PIX_FMT_YUV422P;
+    case AV_PIX_FMT_YUVJ444P: return AV_PIX_FMT_YUV444P;
+    default:
+        return a;
+    }
+}
+
+
 /**
  * Decode a slice header.
  * This will also call ff_MPV_common_init() and frame_start() as needed.
@@ -3212,7 +3224,7 @@ static int decode_slice_header(H264Context *h, H264Context *h0)
                      || h->mb_width  != h->sps.mb_width
                      || h->mb_height != h->sps.mb_height * (2 - h->sps.frame_mbs_only_flag)
                     ));
-    if (h0->avctx->pix_fmt != get_pixel_format(h0, 0))
+    if (non_j_pixfmt(h0->avctx->pix_fmt) != non_j_pixfmt(get_pixel_format(h0, 0)))
         must_reinit = 1;
 
     h->mb_width  = h->sps.mb_width;