[FFmpeg-devel] mpegts encoder not interleaving audio/video packets well

[Michael Niedermayer]

On Tue, Jun 14, 2011 at 01:48:14PM -0400, Tony Strauss wrote:
> This issue exists in the most recent versions of both ffmpeg and libav.
>  Here's how to reproduce:
> 
> wget http://s3.amazonaws.com/tony-strauss-public/180p_d736.mp4
> ffmpeg -y -i 180p_d736.mp4 -acodec copy -vcodec copy -vbsf h264_mp4toannexb
> 180p.ts
> ffprobe -show_packets 180p.ts | grep pts_time | less -i
> 
> Scan through the output; at some point, you should see a jump like:
> pts_time=23.800000
> pts_time=23.866667
> pts_time=23.933333
> pts_time=24.000000
> pts_time=24.066667
> pts_time=20.231378 # JUMP back 4 seconds!
> pts_time=20.254589
> pts_time=20.277800
> pts_time=20.301011
> pts_time=20.324222
> pts_time=20.347433
> 
> It turns out that video frames go up to pts 24.066667, and then audio frames
> starting at 20.231378 are encountered.  This large jump is causing issues in
> a downstream player.
> 
> Why aren't the audio and video frames interleaved better?  The cause seems
> to be the following code in libavformat/mpegtsenc.c (around line 950):
> 
>     if (st->codec->codec_type != AVMEDIA_TYPE_AUDIO) {
>         // for video and subtitle, write a single pes packet
>         mpegts_write_pes(s, st, buf, size, pts, dts);
>         av_free(data);
>         return 0;
>     }
> 
>     if (ts_st->payload_index + size > DEFAULT_PES_PAYLOAD_SIZE) {
>         mpegts_write_pes(s, st, ts_st->payload, ts_st->payload_index,
>                          ts_st->payload_pts, ts_st->payload_dts);
>         ts_st->payload_index = 0;
>     }
> 
> Basically, it seems as if the mpegts encoder writes video frames as soon as
> they're encountered but buffers audio frames up to DEFAULT_PES_PAYLOAD_SIZE.
>  I can fix my issue by simply changing DEFAULT_PES_PAYLOAD_SIZE to 0 in the
> above conditional (always write audio frames).  Does the encoder buffer
> audio frames because audio frames in general are smaller than video frames
> and so PES overhead is more of an issue?  I did see that my file size
> increased a bit with my "fix".
> 
> I'd like to create a long-term fix for this, but I'm not sure of the right
> move.  One thought I have is to keep track of the pts of the first audio
> frame in the buffer; if the pts of a later audio frame is greater than the
> pts of the first audio frame by some threshold (say .5 sec), then flush the
> buffer.

The mpeg-ts muxer should generate spec compliant streams and have
low overhead.

You could take a look at how our mpeg-ps muxer achives that, as the
problem is similar


[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

No snowflake in an avalanche ever feels responsible. -- Voltaire
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20110614/f4f53759/attachment.asc>