<div dir="ltr"><div><div><div><div><div><div><div><div>For your first question, it seems that some info about the stream (in stream->codec->extradata) needs to be passed on to the avCodecContext, so these three lines do exactly that (but I never used it <span style="font-family:arial,helvetica,sans-serif">personnally, so I'm not sure)<br>
</span></div><span style="font-family:arial,helvetica,sans-serif">For your second question, it seems that you did not see that <code>packetToSend.data is a pointer and is not increased but moved,but it also seems that there is a small mistake : when updating packet, offsetInData must be reset to 0<br>
</code></span></div><span style="font-family:arial,helvetica,sans-serif">The code is to take in its entirety :<br></span></div><span style="font-family:arial,helvetica,sans-serif">- offsetInData is the amount of data in packet already processed by <code>avcodec_decode_video2 : if you look at the beginning of the loop, the structure Packet is only updated when all its data has been processed. However, there is no guaranty that </code><code>avcodec_decode_video2 will use all data in the Packet. What the whole structure means is :<br>
</code></span></div><span style="font-family:arial,helvetica,sans-serif"><code>- process data from the packet through a secondary structure packetToSend, return amount of data processed<br></code></span></div><span style="font-family:arial,helvetica,sans-serif"><code>- if the amount of data processed is >= to the amount of data in Packet, update packet<br>
</code></span></div><span style="font-family:arial,helvetica,sans-serif"><code>- otherwise, copy in packetToSend data not yet processed (at address </code><code>packet.packet.data + offsetInData and of size </code><code>packet.packet.size - offsetInData) and process this data<br>
</code></span></div><span style="font-family:arial,helvetica,sans-serif"><code>Repeat last step until an update of Packet becomes necessary (and reset offsetInData to 0)<br><br></code></span></div><code><span style="font-family:arial,helvetica,sans-serif">The data in packetToSend is thus re-addressed at each step, not increased.</span><br>
</code></div><div class="gmail_extra"><br><br><div class="gmail_quote">2014-07-22 22:40 GMT+02:00 Matt Orlando <span dir="ltr"><<a href="mailto:morlando616@gmail.com" target="_blank">morlando616@gmail.com</a>></span>:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">So this tutorial looks like it's covering 2 things that I don't do within my plugin, which is great news since the answer is somewhere within! Unfortunately, I'm a bit confused with some things from the tutorial.<div>
<br></div><div><br><div>Can anyone explain this to me, and/or perhaps provide a more legible code example (c++ / c#)?</div><div><br></div><div><pre style="border:0px;font-family:Menlo,Consolas,'DejaVu Sans Mono',Monaco,monospace;font-size:16px;margin-top:0px;margin-bottom:1.5em;outline:0px;padding:0.25em;vertical-align:baseline;line-height:24px;white-space:pre-wrap;word-wrap:break-word;max-width:100%;color:rgb(51,51,51);background:rgb(245,245,245)">
<code style="border:0px;font-family:Menlo,Consolas,'DejaVu Sans Mono',Monaco,monospace;font-size:1em;font-style:inherit;font-weight:inherit;margin:0px 0px 1.5em;outline:0px;padding:0.25em;vertical-align:baseline;line-height:1.5;max-width:100%;background-image:initial;background-repeat:initial">// we need to make a copy of videoStream->codec->extradata and give it to the context
// make sure that this vector exists as long as the avVideoCodec exists
std::vector codecContextExtraData(stream->codec->extradata, stream->codec->extradata + stream->codec->extradata_size);
avVideoCodec->extradata = reinterpret_cast(codecContextExtraData.data());
avVideoCodec->extradata_size = codecContextExtraData.size();</code></pre></div><div>The other part I don't understand is what he's doing with the offsetInData variable. I'm almost definitely misunderstanding his usage, but it looks as though he's increasing packet.packet.Data by this ever-growing offsetInData. So hypothetically, if he had a 3 hour movie, that offsetInData would be enormous! How could his packet.packet.Data array be large enough to shift the pointer by that amount? I also don't understand why he's only reading (through the reset function of his packet) if that offsetInData is larger than the packet size.</div>
</div><div><br></div><div>Both of these things confuse me...and both of these things I need to implement. If anyone can explain any/all of this in as much detail as possible, I'd be forever grateful. I've got a terrible deadline to hit and I simply need this to work. Thank you sincerely for the help, folks!</div>
<span class="HOEnZb"><font color="#888888">
<div><br></div><div>-Matt</div><div><br></div></font></span></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Jul 22, 2014 at 2:59 PM, Matt Orlando <span dir="ltr"><<a href="mailto:morlando616@gmail.com" target="_blank">morlando616@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Thank you very much, I'll take some time and look at this tutorial.</div><div><div>
<div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Jul 22, 2014 at 11:22 AM, cyril poulet <span dir="ltr"><<a href="mailto:cyril.poulet@centraliens.net" target="_blank">cyril.poulet@centraliens.net</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div>I think "<i style="color:rgb(0,0,0);font-family:Roboto,sans-serif;font-size:14px;line-height:19px">For video, the packet contains exactly one frame.</i><font color="#000000" face="Roboto, sans-serif"><span style="font-size:14px;line-height:19px">" is not true for all codecs, but is for most of the usual ones.<br>
</span></font></div><font color="#000000" face="Roboto, sans-serif"><span style="font-size:14px;line-height:19px">try this one : <a href="http://blog.tomaka17.com/2012/03/libavcodeclibavformat-tutorial/" target="_blank">http://blog.tomaka17.com/2012/03/libavcodeclibavformat-tutorial/</a><br>
</span></font></div><font color="#000000" face="Roboto, sans-serif"><span style="font-size:14px;line-height:19px">It may be that avcodec_decode_video2 recognizes that the first packet data is header and discards it as irrelevant for image decoding...<br>
</span></font></div><div class="gmail_extra"><br><br><div class="gmail_quote">2014-07-22 17:14 GMT+02:00 Matt Orlando <span dir="ltr"><<a href="mailto:morlando616@gmail.com" target="_blank">morlando616@gmail.com</a>></span>:<div>
<div><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">The <a href="https://www.ffmpeg.org/doxygen/trunk/group__lavf__decoding.html#ga4fdb3084415a82e3810de6ee60e46a61" target="_blank">documentation</a> says "<i style="color:rgb(0,0,0);font-family:Roboto,sans-serif;font-size:14px;line-height:19px">For video, the packet contains exactly one frame.</i><font color="#000000" face="Roboto, sans-serif"><span style="font-size:14px;line-height:19px">" I used </span></font><a href="http://dranger.com/ffmpeg/" style="color:rgb(0,0,0);font-family:Roboto,sans-serif;font-size:14px;line-height:19px" target="_blank">this</a><font color="#000000" face="Roboto, sans-serif"><span style="font-size:14px;line-height:19px"> tutorial to understand the flow of what needs to be done. Some of the calls were deprecated so a few google searches got me the replacements. What appears to be happening is those lost frames are just gone...the data isn't being accumulated. Is it possible my av_free_packet call is in a bad spot?</span></font></div>
<div><div>
<div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Jul 22, 2014 at 10:58 AM, cyril poulet <span dir="ltr"><<a href="mailto:cyril.poulet@centraliens.net" target="_blank">cyril.poulet@centraliens.net</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div>what av_read_frame does is take some data and fill your packet with it. There is no guaranty that a full frame is in the packet, or even image data.<br>
</div>avcodec_decode_video2 is the only way to know if you have decoded an image : you feed it your successive packets, it accumulates data, and tells you when the data allowed him to successfully decode an image (through frameFinished)<br>
</div>You should look online for basic examples of how to decode safely vids, there is in particular a tutorial which introduces the structure SafePacket (first version has been updated for latest versions of the libs)<br>
</div><div class="gmail_extra"><br><br><div class="gmail_quote">2014-07-22 16:49 GMT+02:00 Matt Orlando <span dir="ltr"><<a href="mailto:morlando616@gmail.com" target="_blank">morlando616@gmail.com</a>></span>:<div>
<div><br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">Thanks for the response. That's very interesting, though I wonder if it could actually be header related. Based on the videos, I know for sure there are exactly 64 frames in the first movie, for instance. My m_FrameRead is only increased if avcodec_decode_video2 says it was able to get a picture. My thinking is that if it was a header that was read, that function wouldn't set frameFinished to "true". So av_read_frame hits the EOF when my m_FrameRead counter is only 56.<div>
<br></div><div>Are you saying that perhaps the av_read_frame is reading a bunch of data (header + frames) and my avcodec_decode_video2 says there's no picture because there's header data in there. Then a few frames later (or maybe next frame), av_read_frame reads more data, now including the 9th 'frame' and finally avcodec_decode_video2 decodes and I increment my counter (which would be now 1 when I'd want it to be 9).<br>
<div><br></div><div>Thanks for the help!</div><span><font color="#888888"><div>-Matt</div></font></span></div></div><div><div><div class="gmail_extra"><br><br><div class="gmail_quote">
On Tue, Jul 22, 2014 at 8:30 AM, cyril poulet <span dir="ltr"><<a href="mailto:cyril.poulet@centraliens.net" target="_blank">cyril.poulet@centraliens.net</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div>according to the doc, av_read_packet does not check wether the data in the decoded frame is a valid image or not.<br>
</div>As it is always the 8 first "frames" missing, I would assume it is the header that gets decoded first.<br>
</div>Have you noticed missing images when comparing the first real decoded image and your stream in a player ?<br></div><div class="gmail_extra"><br><br><div class="gmail_quote">2014-07-21 18:37 GMT+02:00 Matt Orlando <span dir="ltr"><<a href="mailto:morlando616@gmail.com" target="_blank">morlando616@gmail.com</a>></span>:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div><div dir="ltr">Hi folks,<div><br></div><div>I'm having some trouble with the decoding of my h.264 movies. I've tested with different videos, one 64 frames long and another 52 frames long. In both cases, the first 8 frames (exactly 8) weren't decoding. In other words, the call to avcodec_decode_video2 returns 0 for the got_picture variable. I track the frames read and received for our system and I never get notice that all the frames were read because my counter doesn't increase for the first 8...however all of the subsequent calls to av_read_frame return EOF. To prevent any confusion (or perhaps just to add some context), I need the raw data from each decoded frame since I don't treat them as a frame to a movie but rather a frame of x,y,z positional data.</div>
<div><br></div><div>I work in Unity but made a plugin to communicate with the libraries to read and decode the h.264 movies. This is our update function:</div><div><br></div><div><span style="color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px">bool H264Stream::Update(int desiredFrame)</span><br style="margin:0px;padding:0px;color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px">
<span style="color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px">{</span><br style="margin:0px;padding:0px;color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px">
<span style="color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px"> if( desiredFrame <= m_FrameRead )</span><br style="margin:0px;padding:0px;color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px">
<span style="color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px"> return true;</span><br style="margin:0px;padding:0px;color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px">
<br style="margin:0px;padding:0px;color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px"><span style="color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px"> int frameFinished = 0;</span><br style="margin:0px;padding:0px;color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px">
<br style="margin:0px;padding:0px;color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px"><span style="color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px"> if( av_read_frame( m_AVFormat, &m_Packet ) >= 0 )</span><br style="margin:0px;padding:0px;color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px">
<span style="color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px"> {</span><br style="margin:0px;padding:0px;color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px">
<span style="color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px"> if( m_Packet.stream_index == m_AVStream->index )</span><br style="margin:0px;padding:0px;color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px">
<span style="color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px"> {</span><br style="margin:0px;padding:0px;color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px">
<span style="color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px"> av_frame_unref( m_AVFrame );</span><br style="margin:0px;padding:0px;color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px">
<span style="color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px"> avcodec_decode_video2( m_AVContext, m_AVFrame, &frameFinished, &m_Packet );</span><br style="margin:0px;padding:0px;color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px">
<br style="margin:0px;padding:0px;color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px"><span style="color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px"> if( frameFinished != 0 )</span><br style="margin:0px;padding:0px;color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px">
<span style="color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px"> {</span><br style="margin:0px;padding:0px;color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px">
<span style="color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px"> sws_scale( m_SwsContext, m_AVFrame->data, m_AVFrame->linesize, 0, m_AVContext->height, m_Picture->data, m_Picture->linesize );</span></div>
<div><br style="margin:0px;padding:0px;color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px"><span style="color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px"> m_FrameRead++;</span><br style="margin:0px;padding:0px;color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px">
<br style="margin:0px;padding:0px;color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px"><span style="color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px"> av_free_packet( &m_Packet );</span><br style="margin:0px;padding:0px;color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px">
<span style="color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px"> }</span><br style="margin:0px;padding:0px;color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px">
<span style="color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px"> }</span><br style="margin:0px;padding:0px;color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px">
<span style="color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px"> }</span><br style="margin:0px;padding:0px;color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px">
<br style="margin:0px;padding:0px;color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px"><span style="color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px"> return m_FrameRead >= desiredFrame;</span><br style="margin:0px;padding:0px;color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px">
<span style="color:rgb(46,139,87);font-family:Monaco,'Andale Mono','Courier New',Courier,mono;font-size:12px;line-height:15.210000038146973px">}</span><br></div><div><br></div><div><br></div><div>Oh, quick note, this is my first time using these libraries so if there's something blatantly wrong with how I'm doing this, please let me know. :) Just keep in mind, it's the full frame of 'raw' data I use (post-decode) that I pass back into my Unity c# scripts.</div>
<div><br></div><div>Thanks for the help everyone!</div><span><font color="#888888"><div>-Matt</div></font></span></div>
<br></div></div>_______________________________________________<br>
Libav-user mailing list<br>
<a href="mailto:Libav-user@ffmpeg.org" target="_blank">Libav-user@ffmpeg.org</a><br>
<a href="http://ffmpeg.org/mailman/listinfo/libav-user" target="_blank">http://ffmpeg.org/mailman/listinfo/libav-user</a><br>
<br></blockquote></div><br></div>
<br>_______________________________________________<br>
Libav-user mailing list<br>
<a href="mailto:Libav-user@ffmpeg.org" target="_blank">Libav-user@ffmpeg.org</a><br>
<a href="http://ffmpeg.org/mailman/listinfo/libav-user" target="_blank">http://ffmpeg.org/mailman/listinfo/libav-user</a><br>
<br></blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
Libav-user mailing list<br>
<a href="mailto:Libav-user@ffmpeg.org" target="_blank">Libav-user@ffmpeg.org</a><br>
<a href="http://ffmpeg.org/mailman/listinfo/libav-user" target="_blank">http://ffmpeg.org/mailman/listinfo/libav-user</a><br>
<br></blockquote></div></div></div><br></div>
<br>_______________________________________________<br>
Libav-user mailing list<br>
<a href="mailto:Libav-user@ffmpeg.org" target="_blank">Libav-user@ffmpeg.org</a><br>
<a href="http://ffmpeg.org/mailman/listinfo/libav-user" target="_blank">http://ffmpeg.org/mailman/listinfo/libav-user</a><br>
<br></blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
Libav-user mailing list<br>
<a href="mailto:Libav-user@ffmpeg.org" target="_blank">Libav-user@ffmpeg.org</a><br>
<a href="http://ffmpeg.org/mailman/listinfo/libav-user" target="_blank">http://ffmpeg.org/mailman/listinfo/libav-user</a><br>
<br></blockquote></div></div></div><br></div>
<br>_______________________________________________<br>
Libav-user mailing list<br>
<a href="mailto:Libav-user@ffmpeg.org" target="_blank">Libav-user@ffmpeg.org</a><br>
<a href="http://ffmpeg.org/mailman/listinfo/libav-user" target="_blank">http://ffmpeg.org/mailman/listinfo/libav-user</a><br>
<br></blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
Libav-user mailing list<br>
<a href="mailto:Libav-user@ffmpeg.org">Libav-user@ffmpeg.org</a><br>
<a href="http://ffmpeg.org/mailman/listinfo/libav-user" target="_blank">http://ffmpeg.org/mailman/listinfo/libav-user</a><br>
<br></blockquote></div><br></div>