[FFmpeg-trac] #10500(undetermined:new): Constant memory usage increase when doing 24/7 transcoding

[FFmpeg]

#10500: Constant memory usage increase when doing 24/7 transcoding
-------------------------------------+-------------------------------------
             Reporter:  Igor         |                    Owner:  (none)
  Serganov                           |
                 Type:  defect       |                   Status:  new
             Priority:  normal       |                Component:
                                     |  undetermined
              Version:  6.0          |               Resolution:
             Keywords:               |               Blocked By:
             Blocking:               |  Reproduced by developer:  0
Analyzed by developer:  0            |
-------------------------------------+-------------------------------------
Description changed by Igor Serganov:

Old description:

> I am doing 24/7 live transcoding of various live sources (aac/mp3/hls) to
> different output formats (mostly mp3 and aac/adts).
>
> In all cases (regardless of input/output format) memory consumed by
> application slowly grows up (with approximate speed of 1Mb/hour). That is
> quite ok if there is a finite input as resources are freed when LibAV is
> done. But when doing 24/7 transcoding at some point (depending on the
> memory limit) the app container runs into OOM error and shuts down not
> gracefully.
>
> I am doing `av_packet_free` and `av_frame_free` of allocated
> packet/frames during main transcoding loop (see attached file).
>
> I cannot call it a 'memory leak' as all the resources are being freed in
> case if application processes finite input, exits transcoding loop and
> deallocates all the resources. Though if I do everything correctly in my
> code, I still consider this as a serious issue since with every read of
> decoded packet I can see increase of memory usage which is not reduced in
> the following iteration.
>
> Attaching the file with minimal code that reproduces the issue.
> After compiling the app can be run with command `./live -i <your-input>`.
> For the sake of simplicity, the application writes decoded stream to a
> file, however the same issue can be observed if output stream is written
> to http response instead of a file.

New description:

 I am doing 24/7 live transcoding of various live sources (aac/mp3/hls) to
 different output formats (mostly mp3 and aac/adts).

 In all cases (regardless of input/output format) memory consumed by
 application slowly grows up (with approximate speed of 1Mb/hour). That is
 quite ok if there is a finite input as resources are freed when LibAV is
 done. But when doing 24/7 transcoding at some point (depending on the
 memory limit) the app container runs into OOM error and shuts down not
 gracefully.

 I am doing `av_packet_free` and `av_frame_free` of allocated packet/frames
 during main transcoding loop (see attached file).


 The actual transcoding loop that runs 24/7:


 {{{
 while (av_read_frame(ctx, input_packet) == 0)
     {
         int response = avcodec_send_packet(decctx, input_packet);
         while (response >= 0)
         {
             response = avcodec_receive_frame(decctx, input_frame);
             if (response == AVERROR(EAGAIN) || response == AVERROR_EOF)
             {
                 av_frame_unref(input_frame);
                 av_packet_unref(input_packet);
                 goto read_input;
             }
             else if (response < 0)
             {
                 return response;
             }
             if (response >= 0)
             {
                 input_frame->pts = in_pts;
                 in_pts += input_frame->nb_samples;
                 response = av_buffersrc_add_frame(src, input_frame);
                 if (response < 0)
                 {
                     fprintf(stderr, "Cannot send frame to afilter:");
                     av_frame_unref(input_frame);
                     av_packet_unref(input_packet);
                     goto read_input;
                 }
                 while ((response = av_buffersink_get_frame(sink,
 out_frame)) >= 0)
                 {
                     out_frame->pts = out_pts;
                     out_pts += out_frame->nb_samples;
                     encode(ofmt_ctx, istream, ostream, encctx,
 audio_stream_index, out_frame);
                     av_frame_unref(out_frame);
                 }
                 av_frame_unref(input_frame);
             }
         }
         av_packet_unref(input_packet);
     }
 }}}


 I cannot call it a 'memory leak' as all the resources are being freed in
 case if application processes finite input, exits transcoding loop and
 deallocates all the resources. Though if I do everything correctly in my
 code, I still consider this as a serious issue since with every read of
 decoded packet I can see increase of memory usage which is not reduced in
 the following iteration.

 Attaching the file with minimal code that reproduces the issue.
 After compiling the app can be run with command `./live -i <your-input>`.
 For the sake of simplicity, the application writes decoded stream to a
 file, however the same issue can be observed if output stream is written
 to http response instead of a file.

--
-- 
Ticket URL: <https://trac.ffmpeg.org/ticket/10500#comment:1>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker