<div dir="ltr">Hi all,<div><br></div><div>I am developing a C++ class that stands between two media libraries, namely OpenCV and the very ffmpeg. The aim is simple: to read uncompressed .avi videos with ffmpeg and send frames to OpenCV format in order that OpenCV might be able to process these frames.</div>
<div><br></div><div>The reason why I am doing such a class rather than doing everything using OpenCV is that there is a bug with OpenCV still unresolved (as the best of my knowledge tells me). This bug unfortunately makes the application using OpenCV crash when trying to read an uncompressed video like those I have to read.</div>
<div><br></div><div>So, I learned the basics of ffmpeg and was able to develop, compile and run an application that reads these videos and saves the frames into hard disk, if I want. The question is that I tested my functions in a loop and figured out that there is a huge leak of memory: the memory space allocated increases very fast. </div>
<div><br></div><div>The structure of the code inside the loop is that:</div><div><br></div><div>1) Open file and initialize ffmpeg structures and variables</div><div>2) Read 10 frames from the file</div><div>3) Close file and release ffmpeg structures and variables</div>
<div><br></div><div>I developed Open() and Close() functions specially to avoid memory accumulation but for some reason I am not being able to release properly all ffmpeg resources. I tested the loop without Open() and Close() and it works properly, so the leak is inside these functions and not inside reading routine.</div>
<div><br></div><div>Here follows my Open() and Close() routines:</div><div><br></div><div style>Notice: you will not see the function av_register_all() in my open() code. It is called inside the class constructor.</div><div style>
<br></div><div style>Open():</div><div style><br></div><div style><div>void VideoInterface::open(string filePath) {</div><div> // Check if it is already open</div><div> if (openSuccess) {</div><div> throw GeneralException("There is an already open video in this interface.");</div>
<div> }</div><div> </div><div> // Open video file</div><div> if (avformat_open_input(&formatCtx, filePath.c_str(), NULL, NULL) != 0) {</div><div> throw OpenVideoException("Video could not be open."); </div>
<div> }</div><div> </div><div> //Retrieve stream information</div><div> if (avformat_find_stream_info(formatCtx, NULL) < 0) {</div><div> throw OpenVideoException("Could not ind stream information for the given video.");</div>
<div> }</div><div> </div><div> // Find first video stream</div><div> vStreamIndex = -1;</div><div> for (int i = 0; i < formatCtx->nb_streams; ++i) {</div><div> if (formatCtx->streams[i]->codec->codec_type == AVMEDIA_TYPE_VIDEO) {</div>
<div> vStreamIndex = i;</div><div> break;</div><div> }</div><div> }</div><div> if (vStreamIndex == -1) {</div><div> throw OpenVideoException("Could not find a video stream.");</div>
<div> }</div><div> </div><div> // Get a pointer to the codec context for the video stream</div><div> codecCtx = formatCtx->streams[vStreamIndex]->codec;</div><div> </div><div> /*------------------------------*/</div>
<div> //Grabbing useful information</div><div> double duration = ((double) formatCtx->duration)/AV_TIME_BASE;</div><div> double fps = 1/av_q2d(codecCtx->time_base);</div><div> int frameCount = duration * fps;</div>
<div> /*cout << "Numerator:\t" << codecCtx->time_base.num << endl;</div><div> cout << "Denominator:\t" << codecCtx->time_base.den << endl;</div><div> cout << "FPS:\t\t" << fps << endl;</div>
<div> cout << "Duration:\t" << duration << " secs." << endl;</div><div> cout << "Frame count:\t" << frameCount << endl;*/</div><div> </div>
<div> // Fill VideoInformation structure</div><div> vinfo->duration = duration;</div><div> vinfo->fps = fps;</div><div> vinfo->frameCount = frameCount;</div><div> /*------------------------------*/</div>
<div> </div><div> // Find the decoder for the video stream</div><div> codec = avcodec_find_decoder(codecCtx->codec_id);</div><div> if (codec == NULL) {</div><div> throw OpenVideoException("Unsuported codec!");</div>
<div> }</div><div> </div><div> // Open codec</div><div> AVDictionary *options = NULL;</div><div> if (avcodec_open2(codecCtx, codec, &options) < 0) {</div><div> throw OpenVideoException("Could not open codec!");</div>
<div> } </div><div> </div><div> /*------- Prepare interface for first reading -------*/</div><div> </div><div> // Allocate an AVFrame structure</div><div> frame = avcodec_alloc_frame();</div><div> frameRGB = avcodec_alloc_frame();</div>
<div> if (frameRGB == NULL or frame == NULL) {</div><div> throw OpenVideoException("Not able to allocate frame space.");</div><div> }</div><div> </div><div> // Determine required buffer size and allocate buffer</div>
<div> int numBytes = avpicture_get_size(PIX_FMT_RGB24, codecCtx->width, codecCtx->height);</div><div> rawData = (uint8_t *) av_malloc(numBytes * sizeof(uint8_t));</div><div><br></div><div> sws_ctx = sws_getContext(codecCtx->width, codecCtx->height, </div>
<div> codecCtx->pix_fmt, codecCtx->width, codecCtx->height, PIX_FMT_RGB24, </div><div> SWS_BILINEAR, NULL, NULL, NULL);</div><div> </div><div> // Assign appropriate parts of buffer to image planes in pFrameRGB</div>
<div> // Note that pFrameRGB is an AVFrame, but AVFrame is a superset</div><div> // of AVPicture</div><div> avpicture_fill((AVPicture*) frameRGB, rawData, PIX_FMT_RGB24, </div><div> codecCtx->width, codecCtx->height); </div>
<div> </div><div> openSuccess = true;</div><div>}</div></div><div><br></div><div>Close():</div><div><br></div><div><div>void VideoInterface::close() {</div><div> // Free buffer</div><div> av_free(rawData);</div>
<div> </div><div> // Free the RGB image</div><div> av_free(frameRGB);</div><div><br></div><div> // Free the YUV frame</div><div> av_free(frame);</div><div> </div><div> // Free packet</div><div>
av_free_packet(&packet);</div><div> </div><div> // Close Codec context</div><div> avcodec_close(codecCtx);</div><div> </div><div> // Close Format context</div><div> avformat_close_input(&formatCtx);</div>
<div> </div><div> // Reinit class variables (not ffmpeg at all, and not any dynamically allocate memory as well)</div><div> init();</div><div>}</div><div><br></div><div style>One possible reason for my problem is that I found out that avcodec_close(AvCodecContext*) only frees AVCodecContext content, and not the variable itself. After it I tried to call av_free(codecCtx) but it generated an error. So, a first approach would be: how can one completely release AVCodecContext variables?</div>
<div style><br></div><div style>Thank you all!</div><div><br></div>-- <br><div><div>--</div><div>Matheus Henrique Klem Galvez</div></div><div>The Catholic University of America, Washington, DC</div><div>Universidade Federal do Rio de Janeiro, Rio de Janeiro, Brasil</div>
</div></div>