<html><head><meta http-equiv="Content-Type" content="text/html charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">On Jan 23, 2017, at 5:38 PM, Cesareo Fernandez <<a href="mailto:cesareof@gmail.com" class="">cesareof@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class=""><div class="">I have been attempting to use the ffmpeg libraries to take as input a RTMP stream consisting of a video stream and an audio stream, and decode the respective streams in order to push the contents out to a website via a websocket. The video portion of this task is complete and works fine without issue. Converting frames of h.264 video to single motion jpeg images and pushing those on without a problem. I am attempting to do something similar with audio, takes frames of aac audio and passing those out through a websocket to be played back in chunks, this portion of the task is proving to be a lot more difficult than I thought it would be. </div><div class=""><br class=""></div><div class="">The way I am testing this is simply learning the way to decode an encode audio (if necessary) to a file before trying to implement it practically via websockets and even this is proving more difficult than I expected, I found some examples via google, but I can't get any of these to work: </div><div class=""> </div><div class=""><br class=""></div><div class=""><div class="">int PullAudioFrames( char input[], char output[] )</div><div class="">{</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>const int MAX_AUDIO_FRAME_SIZE = 192000;</div><div class=""> bool bRet = false;</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>bool bFileOpened = false;</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>bool bAudioCodecOpen = false;</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>bool abort = false;</div><div class=""><br class=""></div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>int nVideoStream = -1;</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>int nAudioStream = -1;</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>AVDictionary *optionsDict = NULL;</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span></div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>//-- Register all formats and codecs and network tools</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>avcodec_register_all();</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>av_register_all();<span class="gmail-Apple-tab-span" style="white-space:pre"> </span></div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>avformat_network_init();</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>//--</div><div class=""><br class=""></div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>AVCodecContext *pIn_AudioCodecCtx = NULL;</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>AVCodec *pIn_AudioCodec = NULL;</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>struct SwrContext *au_convert_ctx;</div><div class=""><br class=""></div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>AVFormatContext *pFormatCtx = avformat_alloc_context();</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>if(avformat_open_input(&pFormatCtx, input, NULL, &optionsDict) == 0 )<span class="gmail-Apple-tab-span" style="white-space:pre"> </span>//Open Stream<span class="gmail-Apple-tab-span" style="white-space:pre"> </span></div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>{</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>bFileOpened = true;<span class="gmail-Apple-tab-span" style="white-space:pre"> </span></div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>if(avformat_find_stream_info(pFormatCtx, &optionsDict) >= 0 )<span class="gmail-Apple-tab-span" style="white-space:pre"> </span>//Get Stream information (video, audio, subtitle)</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>{</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>for(int i=0; i<(int)pFormatCtx->nb_streams; i++)<span class="gmail-Apple-tab-span" style="white-space:pre"> </span>//Find the video stream</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>{</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>if(pFormatCtx->streams[i]->codecpar->codec_type==AVMEDIA_TYPE_AUDIO)</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>nAudioStream=i;</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span></div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>if( nAudioStream >= 0 ) </div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>break;</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>bool bCleanUp = false;</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span></div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>if( nAudioStream >= 0 )</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>{</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>pIn_AudioCodec = avcodec_find_decoder( pFormatCtx->streams[nAudioStream]->codecpar->codec_id );</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>if(pIn_AudioCodec != NULL)</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>{</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>pIn_AudioCodecCtx = avcodec_alloc_context3(pIn_AudioCodec);</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span></div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>// Fill the codecCtx with the parameters of the codec used in the read file.</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>if( avcodec_parameters_to_context(pIn_AudioCodecCtx, pFormatCtx->streams[nAudioStream]->codecpar) == 0 )</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>{</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>if( avcodec_open2(pIn_AudioCodecCtx, pIn_AudioCodec, NULL) >= 0 )<span class="gmail-Apple-tab-span" style="white-space:pre"> </span>//Open Audio codec<span class="gmail-Apple-tab-span" style="white-space:pre"> </span></div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>bAudioCodecOpen = true;</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>else</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>bCleanUp = true;</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>else</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>bCleanUp = true;</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>if( bCleanUp ) </div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>{</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>avcodec_close( pIn_AudioCodecCtx );</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>avcodec_free_context( &pIn_AudioCodecCtx );</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><br class=""></div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>if( bAudioCodecOpen )</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>{</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>au_convert_ctx = swr_alloc();</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>au_convert_ctx=swr_alloc_set_opts(au_convert_ctx, AV_CH_LAYOUT_STEREO, AV_SAMPLE_FMT_S16, 44100, av_get_default_channel_layout(pIn_AudioCodecCtx->channels), pIn_AudioCodecCtx->sample_fmt , pIn_AudioCodecCtx->sample_rate,0, NULL);</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>swr_init(au_convert_ctx);</div><div class=""><br class=""></div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>AVFrame *pFrame=av_frame_alloc();</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>AVPacket packet;</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>av_init_packet(&packet);</div><div class=""><br class=""></div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>while( !abort ) </div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>{</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>av_packet_unref(&packet);</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>av_frame_unref(pFrame);</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>int result = av_read_frame( pFormatCtx, &packet);<span class="gmail-Apple-tab-span" style="white-space:pre"> </span></div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>if( result < 0 )</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>{</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>if( result == AVERROR(EAGAIN) )<span class="gmail-Apple-tab-span" style="white-space:pre"> </span></div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>continue;</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>else if( result == AVERROR(EPIPE) )</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>printf( "[\"Error\", {\"message\" : \"Unable to read Frame pipe error. [%d] [%d]\"}]", result, AVERROR(result) );<span class="gmail-Apple-tab-span" style="white-space:pre"> </span></div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>else if (result == AVERROR_EOF)</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>printf( "[\"Error\", {\"message\" : \"Unable to read Frame end of File. [%d] [%d]\"}]", result, AVERROR(result) );</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>else</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>printf( "[\"Error\", {\"message\" : \"Unable to read Frame. [%d] [%d]\"}] ", result, AVERROR(result) );</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>abort = true;</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>else</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>{<span class="gmail-Apple-tab-span" style="white-space:pre"> </span> </div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>if(packet.stream_index==nAudioStream)<span class="gmail-Apple-tab-span" style="white-space:pre"> </span></div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>{</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>int frameFinished=0;</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>int nResult = avcodec_send_packet(pIn_AudioCodecCtx, &packet);</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>if( nResult == 0 )</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>nResult = avcodec_receive_frame( pIn_AudioCodecCtx, pFrame );</div><div class=""><br class=""></div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>if( nResult == 0 )</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>{</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>uint8_t *out_buffer = (uint8_t *)av_malloc(MAX_AUDIO_FRAME_SIZE*2);</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>int data_size = av_samples_get_buffer_size(NULL, pIn_AudioCodecCtx->channels, pFrame->nb_samples, pIn_AudioCodecCtx->sample_fmt, 1);</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>swr_convert(au_convert_ctx,&out_buffer, MAX_AUDIO_FRAME_SIZE,(const uint8_t **)pFrame->data , pFrame->nb_samples);</div><div class=""><br class=""></div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>FILE *fOut;</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>if( fopen_s( &fOut, output, "a+" ) == 0 )</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>{</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>fwrite(out_buffer, 1, data_size, fOut);</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>fclose(fOut);</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>}<span class="gmail-Apple-tab-span" style="white-space:pre"> </span></div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>av_packet_unref(&packet);</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>av_frame_free(&pFrame);</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>else </div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>printf( "Unable to open Codec in %s", input );<span class="gmail-Apple-tab-span" style="white-space:pre"> </span></div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>else</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>printf( "No stream information found in %s", input );<span class="gmail-Apple-tab-span" style="white-space:pre"> </span></div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>else</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>printf("Unable to open live video stream %s", input );</div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>if( bAudioCodecOpen )</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>{</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>avcodec_close(pIn_AudioCodecCtx);</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>avcodec_free_context(&pIn_AudioCodecCtx);</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>swr_free(&au_convert_ctx);</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>bAudioCodecOpen = false;</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><br class=""></div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>if(bFileOpened)</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>{</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>if( pFormatCtx )</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>avformat_close_input(&pFormatCtx);<span class="gmail-Apple-tab-span" style="white-space:pre"> </span></div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>bFileOpened = false;</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><span class="gmail-Apple-tab-span" style="white-space:pre"> </span>return 0;</div><div class="">}</div></div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">The resulting file is not playable and passing it back through CLI ffmpeg.exe doesn't give a whole lot of useful information why that is. Any help, or point in the right direction would be greatly appreciated. </div><div class=""><br class=""></div><div class=""><br class=""></div></div>
_______________________________________________<br class="">Libav-user mailing list<br class=""><a href="mailto:Libav-user@ffmpeg.org" class="">Libav-user@ffmpeg.org</a><br class=""><a href="http://ffmpeg.org/mailman/listinfo/libav-user" class="">http://ffmpeg.org/mailman/listinfo/libav-user</a><br class=""></div></blockquote><br class=""></div><div><div class=""><span class="gmail-Apple-tab-span" style="white-space: pre;"> </span>uint8_t *out_buffer = (uint8_t *)av_malloc(MAX_AUDIO_FRAME_SIZE*2);</div><div class=""><span class="gmail-Apple-tab-span" style="white-space: pre;"> </span>int data_size = av_samples_get_buffer_size(NULL, pIn_AudioCodecCtx->channels, pFrame->nb_samples, pIn_AudioCodecCtx->sample_fmt, 1);</div><div class=""><span class="gmail-Apple-tab-span" style="white-space: pre;"> </span>swr_convert(au_convert_ctx,&out_buffer, MAX_AUDIO_FRAME_SIZE,(const uint8_t **)pFrame->data , pFrame->nb_samples);</div><div class=""><br class=""></div><div class=""><span class="gmail-Apple-tab-span" style="white-space: pre;"> </span>FILE *fOut;</div><div class=""><span class="gmail-Apple-tab-span" style="white-space: pre;"> </span>if( fopen_s( &fOut, output, "a+" ) == 0 )</div><div class=""><span class="gmail-Apple-tab-span" style="white-space: pre;"> </span>{</div><div class=""><span class="gmail-Apple-tab-span" style="white-space: pre;"> </span>fwrite(out_buffer, 1, data_size, fOut);</div><div class=""><span class="gmail-Apple-tab-span" style="white-space: pre;"> </span>fclose(fOut);</div><div class=""><span class="gmail-Apple-tab-span" style="white-space: pre;"> </span>}</div><div class=""><span class="gmail-Apple-tab-span" style="white-space: pre;"> </span></div><div class=""><span class="gmail-Apple-tab-span" style="white-space: pre;">This looks wrong, you should be using the return of swr_convert to base the number of bytes to write. Also, why allocate out_buffer, and open and close the file on every frame?</span></div></div><br class=""></body></html>