[FFmpeg-cvslog] libavformat/avienc: Check bits per sample for PAL8

[Michael Niedermayer]

ffmpeg | branch: master | Michael Niedermayer <michael at niedermayer.cc> | Sun Jan 26 22:19:48 2020 +0100| [35958782819c00211e247332ab18fbf2f28267e1] | committer: Michael Niedermayer

libavformat/avienc: Check bits per sample for PAL8

Fixes: assertion failure
Fixes: Ticket 8172

Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=35958782819c00211e247332ab18fbf2f28267e1
---

 libavformat/avienc.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/libavformat/avienc.c b/libavformat/avienc.c
index d6cfb1b783..07e1c3319e 100644
--- a/libavformat/avienc.c
+++ b/libavformat/avienc.c
@@ -458,6 +458,14 @@ static int avi_write_header(AVFormatContext *s)
                     && par->format != AV_PIX_FMT_NONE)
                     av_log(s, AV_LOG_ERROR, "%s rawvideo cannot be written to avi, output file will be unreadable\n",
                           av_get_pix_fmt_name(par->format));
+
+                if (par->format == AV_PIX_FMT_PAL8) {
+                    if (par->bits_per_coded_sample < 0 || par->bits_per_coded_sample > 8) {
+                        av_log(s, AV_LOG_ERROR, "PAL8 with %d bps is not allowed\n", par->bits_per_coded_sample);
+                        return AVERROR(EINVAL);
+                    }
+                }
+
                 break;
             case AVMEDIA_TYPE_AUDIO:
                 flags = (avi->write_channel_mask == 0) ? FF_PUT_WAV_HEADER_SKIP_CHANNELMASK : 0;