[FFmpeg-cvslog] asfdec_o: reject size > INT64_MAX in asf_read_unknown

[Andreas Cadhalpun]

ffmpeg | branch: release/2.8 | Andreas Cadhalpun <Andreas.Cadhalpun at googlemail.com> | Tue Jan  5 13:20:11 2016 +0100| [407ab167c07c9afb4aa3518f850241fad0ac0df9] | committer: Andreas Cadhalpun

asfdec_o: reject size > INT64_MAX in asf_read_unknown

Both avio_skip and detect_unknown_subobject use int64_t for the size
parameter.

This fixes a segmentation fault due to infinite recursion.

Reviewed-by: Alexandra Hájková <alexandra.khirnova at gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun at googlemail.com>
(cherry picked from commit aa180169961b46cf0d2bcc23cb686f93c079b256)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun at googlemail.com>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=407ab167c07c9afb4aa3518f850241fad0ac0df9
---

 libavformat/asfdec_o.c |    3 +++
 1 file changed, 3 insertions(+)

diff --git a/libavformat/asfdec_o.c b/libavformat/asfdec_o.c
index 3221253..2ccfe3a 100644
--- a/libavformat/asfdec_o.c
+++ b/libavformat/asfdec_o.c
@@ -178,6 +178,9 @@ static int asf_read_unknown(AVFormatContext *s, const GUIDParseTable *g)
     uint64_t size   = avio_rl64(pb);
     int ret;
 
+    if (size > INT64_MAX)
+        return AVERROR_INVALIDDATA;
+
     if (asf->is_header)
         asf->unknown_size = size;
     asf->is_header = 0;