[FFmpeg-devel] [PATCH 4/9] avformat/rmdec: Actually return value < 0 on read_header failure

[Andreas Rheinhardt]

The RealMedia demuxer's read_header function initially initializes ret,
the variable designated for the return variable to -1. Afterwards, chunks
of the file are parsed in a loop until an error happens or until the actual
frame data is encountered. If the first function whose return
value is put into ret doesn't fail, then ret contains a value >= 0
(actually == 0) and this is what will be returned if an error is
encountered afterwards.

This is a regression since 35bbc1955a58ba74552c50d9161084644f00bbd3.
Before that, ret had never been overwritten with a nonnegative value.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt at gmail.com>
---
 libavformat/rmdec.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/libavformat/rmdec.c b/libavformat/rmdec.c
index a36e693ab2..6851b7e1f4 100644
--- a/libavformat/rmdec.c
+++ b/libavformat/rmdec.c
@@ -538,7 +538,7 @@ static int rm_read_header(AVFormatContext *s)
     unsigned int data_off = 0, indx_off = 0;
     char buf[128], mime[128];
     int flags = 0;
-    int ret = -1;
+    int ret;
     unsigned size, v;
     int64_t codec_pos;
 
@@ -554,6 +554,7 @@ static int rm_read_header(AVFormatContext *s)
     avio_skip(pb, tag_size - 8);
 
     for(;;) {
+        ret = AVERROR_INVALIDDATA;
         if (avio_feof(pb))
             goto fail;
         tag = avio_rl32(pb);
@@ -619,8 +620,9 @@ static int rm_read_header(AVFormatContext *s)
                 avio_seek(pb, codec_pos + size, SEEK_SET);
             } else {
                 avio_skip(pb, -4);
-                if (ff_rm_read_mdpr_codecdata(s, s->pb, st, st->priv_data,
-                                              size, mime) < 0)
+                ret = ff_rm_read_mdpr_codecdata(s, s->pb, st, st->priv_data,
+                                                size, mime);
+                if (ret < 0)
                     goto fail;
             }
 
-- 
2.20.1