<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Hi Brad, </div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
I tried your example but I could not see the data that I added as side data. This is my code ( simplified and only relevant parts ), please let me know if you need more. The image class has a OpenCV Mat and some members. In the encoder I see the char that
I added as side_data to the frame, but it can't be retrieved when I decode it. I want to mention again that encode/decoding works like a charm.... but not the custom side data
<span contenteditable="false" id="🙁">🙁</span></div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
=====================================================</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
void MovieEncoder::process(const std::vector< Image >* images )
<div>{</div>
<div>TimeStamp encodingTimer("EncodingTimer", true );</div>
<div><br>
</div>
<div> for( auto image : *images )</div>
<div> {</div>
<div> if( m_setStartTime )</div>
<div> {</div>
<div> m_startTime = image.timeStamp();</div>
<div> m_setStartTime = false;</div>
<div> }</div>
<div><br>
</div>
<div><br>
</div>
<div> if (!m_stream )</div>
<div> return;</div>
<div><br>
</div>
<div> AVFrame* frame = av_frame_alloc();</div>
<div><br>
</div>
<div> createFrame( image, frame );</div>
<div><br>
</div>
<div> writeFrame( frame );</div>
<div><br>
</div>
<div> if( frame )</div>
<div> {</div>
<div> av_free( frame->data[0]);</div>
<div> av_frame_free( &frame );</div>
<div> }</div>
<div> }</div>
<span>}</span><br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span style="background-color:rgb(255, 255, 255);display:inline !important">=====================================================</span></div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
void MovieEncoder::createFrame( const Image& image, AVFrame* frame )
<div>{</div>
<div> frame->format = outputPixelFormat;<br>
</div>
<div> frame->width = image.width();</div>
<div> frame->height = image.height();</div>
<div> frame->pict_type = AV_PICTURE_TYPE_P;</div>
<div><br>
</div>
<div> int ret = av_image_alloc( frame->data, frame->linesize, frame->width, frame->height, AV_PIX_FMT_BGR24, 1);</div>
<div><br>
</div>
<div> if (ret < 0)</div>
<div> return;</div>
<div><br>
</div>
<div> struct SwsContext* sws_ctx = sws_getContext((int)image.width(), (int)image.height(), AV_PIX_FMT_BGR24,</div>
<div> (int)image.width(), (int)image.height(), outputPixelFormat,</div>
<div> 0, NULL, NULL, NULL);</div>
<div><br>
</div>
<div> const uint8_t* rgbData[1] = { (uint8_t* )image.getData() };</div>
<div> int rgbLineSize[1] = { 3 * (int)image.width() };</div>
<div><br>
</div>
<div> sws_scale(sws_ctx, rgbData, rgbLineSize, 0, image.height(), frame->data, frame->linesize);</div>
<div><br>
</div>
<div> // @Brad, just adding a char 'H' here </div>
<div> char seiMessage[] = {'H'};</div>
<div> qDebug() << "encode " << seiMessage;</div>
<div> AVFrameSideData* sideData = av_frame_new_side_data( frame, AV_FRAME_DATA_SEI_UNREGISTERED, sizeof( seiMessage ));</div>
<div> if( sideData == nullptr )</div>
<div> return;</div>
<div> std::memcpy( sideData->data, seiMessage, sideData->size );</div>
<div> qDebug() << "encode side " << *( char*)( sideData->data );</div>
}</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span style="background-color:rgb(255, 255, 255);display:inline !important">=====================================================</span><br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
void MovieEncoder::encodingFrame( AVFrame* frame )
<div>{</div>
<div> int result;</div>
<div><br>
</div>
<div> if ( s_frameCount >= streamNumberOfFrames)</div>
<div> {</div>
<div> /**</div>
<div> * @note No more frames to compress.</div>
<div> */</div>
<div> s_frameCount = streamNumberOfFrames;</div>
<div> }</div>
<div><br>
</div>
<div> frame->pts = s_frameCount;</div>
<div><br>
</div>
<div><br>
</div>
<div> if (m_formatCtx->oformat->flags & AVFMT_NOFILE )</div>
<div> {</div>
<div> /**</div>
<div> * @todo this is experimental for other examinations than OSV/OSD</div>
<div> * @note Raw video case - directly store the picture in the packet</div>
<div> */</div>
<div> AVPacket pkt;</div>
<div> av_init_packet(&pkt);</div>
<div><br>
</div>
<div> pkt.flags |= AV_PKT_FLAG_KEY;</div>
<div> pkt.stream_index = m_stream->index;</div>
<div> pkt.data = frame->data[0];</div>
<div> pkt.size = sizeof(AVPicture);</div>
<div><br>
</div>
<div> result = av_write_frame( m_formatCtx, &pkt );</div>
<div> }</div>
<div> else</div>
<div> {</div>
<div> AVPacket pkt;</div>
<div> av_init_packet(&pkt);</div>
<div><br>
</div>
<div> result = avcodec_send_frame(m_codecCtx, frame);</div>
<div><br>
</div>
<div> checkError( result, "Error encoding video frame: ");</div>
<div><br>
</div>
<div> while( result >= 0 )</div>
<div> {</div>
<div> result = avcodec_receive_packet(m_codecCtx, &pkt);</div>
<div><br>
</div>
<div> if (result == AVERROR(EAGAIN) /*|| ret == AVERROR_EOF*/)</div>
<div> { </div>
<div> Log::printLine("No more packet to write", Log::Debug );</div>
<div> return;</div>
<div> }</div>
<div> else if (result < 0)</div>
<div> {</div>
<div> Log::printLine("Codec context of the package is not correct", Log::Debug);</div>
<div> return;</div>
<div> }</div>
<div><br>
</div>
<div> pkt.stream_index = m_stream->index;</div>
<div><br>
</div>
<div> pkt.pts = av_rescale_q( pkt.pts, m_codecCtx->time_base, m_stream->time_base );</div>
<div> pkt.dts = av_rescale_q( pkt.dts, m_codecCtx->time_base, m_stream->time_base );</div>
<div> pkt.duration = int64( 1 / streamFrameRate );</div>
<div><br>
</div>
<div> result = av_write_frame( m_formatCtx, &pkt );</div>
<div><br>
</div>
<div> s_frameCount++;</div>
<div> }</div>
<div> }</div>
<span>}</span><br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span><span style="background-color:rgb(255, 255, 255);display:inline !important">=====================================================</span><br>
</span></div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span><span style="background-color:rgb(255, 255, 255);display:inline !important">void MovieDecoder::process()
<div>{</div>
<div> int retCode( 0 );</div>
<div><br>
</div>
<div> AVFrame *frame = av_frame_alloc();</div>
<div> AVFrame* rgbFrame = av_frame_alloc();</div>
<div> AVPacket* avpkt = av_packet_alloc();</div>
<div><br>
</div>
<div> uint8_t* imageBuffer = NULL;</div>
<div><br>
</div>
<div> // Read all the frames</div>
<div> while( av_read_frame( m_formatCtx, avpkt ) >= 0)</div>
<div> {</div>
<div> if (avpkt->size == 0)</div>
<div> break;</div>
<div><br>
</div>
<div> // This function might fail because of parameter set packets, just ignore and continue</div>
<div> retCode = avcodec_send_packet( m_codecCtx, avpkt);</div>
<div> checkError( retCode, "avcodec_send_packet ret < 0" );</div>
<div><br>
</div>
<div> if (retCode < 0)</div>
<div> continue;</div>
<div><br>
</div>
<div><br>
</div>
<div> // Receive the uncompressed frame back</div>
<div> retCode = avcodec_receive_frame( m_codecCtx, frame);</div>
<div><br>
</div>
<div><span style="background-color:rgb(255, 255, 255);display:inline !important"> // @Brad, no char 'H' found over here
<span contenteditable="false" id="🙁">🙁</span></span><br>
</div>
<div> AVFrameSideData* sideData = av_frame_get_side_data( frame, AV_FRAME_DATA_SEI_UNREGISTERED );</div>
<div> qDebug() << "decode " << *(char*)sideData->data;</div>
<div><br>
</div>
<div> if (retCode < 0)</div>
<div> {</div>
<div> // Sometimes we cannot get a new frame, continue in this case</div>
<div> if (retCode == AVERROR(EAGAIN))</div>
<div> continue;</div>
<div><br>
</div>
<div> break;</div>
<div> }</div>
<div><br>
</div>
<div> // Calculate output buffer requirements</div>
<div> uint32_t bufferSize = av_image_get_buffer_size(AVPixelFormat(frame->format),</div>
<div> frame->width, frame->height, 1);</div>
<div><br>
</div>
<div> // Use temp buffer for the video data</div>
<div> if( imageBuffer == NULL )</div>
<div> imageBuffer = new uint8_t[bufferSize];</div>
<div><br>
</div>
<div><br>
</div>
<div> struct SwsContext* sws_ctx = sws_getContext((int)frame->width, (int)frame->height, AV_PIX_FMT_GBRP,</div>
<div> (int)frame->width, (int)frame->height, inputPixelFormat,</div>
<div> NULL, NULL, NULL, NULL);</div>
<div><br>
</div>
<div> av_image_fill_arrays( rgbFrame->data, rgbFrame->linesize, imageBuffer,</div>
<div> inputPixelFormat, m_codecCtx->width, m_codecCtx->height, 1);</div>
<div><br>
</div>
<div> rgbFrame->width = m_codecCtx->width;</div>
<div> rgbFrame->height = m_codecCtx->height;</div>
<div> rgbFrame->format = inputPixelFormat;</div>
<div><br>
</div>
<div> sws_scale(sws_ctx, frame->data, frame->linesize, 0, frame->height, rgbFrame->data, rgbFrame->linesize );</div>
<div><br>
</div>
<div> cv::Mat rgbMat( frame->height, frame->width, CV_8UC3, rgbFrame->data[0] );</div>
<div><br>
</div>
<div> av_packet_unref( avpkt );</div>
<div> }</div>
<div><br>
</div>
<div> delete imageBuffer;</div>
<div><br>
</div>
<div> avcodec_close( m_codecCtx );</div>
<div> av_free( m_codecCtx );</div>
<div> av_dict_free(&frame->metadata);</div>
<div> av_frame_free( &frame );</div>
<div> av_frame_free( &rgbFrame );</div>
<div> av_packet_free( &avpkt );</div>
}<br>
</span></span></div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>Van:</b> Libav-user <libav-user-bounces@ffmpeg.org> namens laddoe <xyfix@hotmail.com><br>
<b>Verzonden:</b> woensdag 16 februari 2022 12:24<br>
<b>Aan:</b> This list is about using libavcodec, libavformat, libavutil, libavdevice and libavfilter. <libav-user@ffmpeg.org><br>
<b>Onderwerp:</b> Re: [Libav-user] Custom data in AVPacket or AVFrame</font>
<div> </div>
</div>
<style type="text/css" style="display:none">
<!--
p
{margin-top:0;
margin-bottom:0}
-->
</style>
<div dir="ltr">
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Sorry that was my fault but I use avcodec-58.dll, avformat-58.dll, avutil-56.dll and libx264-161.dll. I will try your example code and come back to you. </div>
<div id="x_appendonsend"></div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="x_divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>Van:</b> Libav-user <libav-user-bounces@ffmpeg.org> namens Brad Hards <bradh@frogmouth.net><br>
<b>Verzonden:</b> woensdag 16 februari 2022 10:28<br>
<b>Aan:</b> This list is about using libavcodec, libavformat, libavutil, libavdevice and libavfilter. <libav-user@ffmpeg.org><br>
<b>Onderwerp:</b> Re: [Libav-user] Custom data in AVPacket or AVFrame</font>
<div> </div>
</div>
<div class="x_BodyFragment"><font size="2"><span style="font-size:11pt">
<div class="x_PlainText">On Wednesday, 16 February 2022 8:16:17 PM AEDT laddoe wrote:<br>
> Correction, I meant av_frame_new_side_data but it looks like the "side data"<br>
> is not meant for that. I couldn't find the right documentation that would<br>
> explain what the purpose is for the "side data" ( the same for<br>
> AVPacketSideData). I also tried the metadata struct (AVDictionary) in the<br>
> AVFrame struct but it did not work either.<br>
We know it didn't work - you wouldn't be posting if it did.<br>
However we have no idea what actually happened instead. We still don't know <br>
what version of the libraries you are using.<br>
<br>
Here is an extract from a sample I used:<br>
<br>
AVFrame *frame;<br>
AVFrameSideData *side_data;<br>
// This is from MISB ST 2101<br>
char sei_message[] = {0xa5, 0x50, 0x52, 0xaf, 0x52, 0x16, 0x5f, 0x45,<br>
0xa3, 0x18, 0x1c, 0xfc, 0x7a, 0xbb, 0xc2, 0x67,<br>
0x01, 0x70, 0xF5, 0x92, 0xF0, 0x23, 0x73, 0x36,<br>
0x4A, 0xF8, 0xAA, 0x91, 0x62, 0xC0, 0x0F, 0x2E,<br>
0xB2, 0xDA, 0x16, 0xB7, 0x43, 0x41, 0x00, 0x08,<br>
0x41, 0xA0, 0xBE, 0x36, 0x5B, 0x5A, 0xB9, 0x6A,<br>
0x36, 0x45};<br>
<br>
side_data = av_frame_new_side_data(frame, AV_FRAME_DATA_SEI_UNREGISTERED, <br>
sizeof(sei_message));<br>
if (!side_data)<br>
{<br>
fprintf(stderr, "Could not allocate the video frame side data\n");<br>
exit(1);<br>
}<br>
memcpy(side_data->data, sei_message, side_data->size);<br>
<br>
If that doesn't work for you, please make a SSCCE (<a href="http://sscce.org/">http://sscce.org/</a>) of what
<br>
you are doing, what you expected that code to do, and what happened instead.<br>
<br>
Brad<br>
<br>
<br>
<br>
_______________________________________________<br>
Libav-user mailing list<br>
Libav-user@ffmpeg.org<br>
<a href="https://ffmpeg.org/mailman/listinfo/libav-user">https://ffmpeg.org/mailman/listinfo/libav-user</a><br>
<br>
To unsubscribe, visit link above, or email<br>
libav-user-request@ffmpeg.org with subject "unsubscribe".<br>
</div>
</span></font></div>
</div>
</body>
</html>