<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"><html>
<head>
<meta name="Generator" content="Zarafa WebAccess v7.0.2-29470">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<title>how to handle missing pts values to keep audio sync (mpegts->mp4)</title>
<style type="text/css">
body
{
font-family: Arial, Verdana, Sans-Serif ! important;
font-size: 12px;
padding: 5px 5px 5px 5px;
margin: 0px;
border-style: none;
background-color: #ffffff;
}
p, ul, li
{
margin-top: 0px;
margin-bottom: 0px;
}
</style>
</head>
<body>
<div><p>Hello!</p> <p> </p> <p>This is my first mail on this list. So i hope i didn't got the wrong mailinglist.</p> <p> </p> <p>Im writing an program which received a mpeg ts udp stream and convert it to mp4 on the fly.</p> <p>Sometimes i have a problem with async audio.</p> <p> </p> <p>[mpegts @ 0x35fd580] PES packet size mismatch<br />[mpeg2video @ 0x3604200] skipped MB in I frame at 25 32<br />[mpeg2video @ 0x3604200] skipped MB in I frame at 1 33<br />[mpeg2video @ 0x3604200] ac-tex damaged at 0 34<br />[mpeg2video @ 0x3604200] ac-tex damaged at 0 35<br />[mpeg2video @ 0x3604200] Warning MVs not available<br />[mpeg2video @ 0x3604200] concealing 176 DC, 176 AC, 176 MV errors in I frame<br />[mp2 @ 0x3649fa0] Header missing<br />[err]failed to decode audio packet</p> <p> </p> <p>I use udp so it is possible to loose a packages. I have to handle it.</p> <p> </p> <p>my video_push and audio_push are appended.</p> <p>I dump out the PTS before i push it to the encoder.</p> <p>So you can see that 67 PTS Values are missing.</p> <p>With a 25fps video it is a delay of 2,68 sec. That's exactly what i see if i play the mp4 file via vlc.<br /> I have tried to skip some video frames, but din't have any effect.<br /> How to bring back sync audio?<br /> skip audio or video frames?<br /> recalc pts values?<br /> add dummy frames?<br /> all together?</p> <p> </p> <p>push audio frame to 689152 (673)<br />push audio frame to 690176 (674)<br />[mpegts @ 0x2163580] PES packet size mismatch<br />push video frame to 392<br />[mpeg2video @ 0x216a200] skipped MB in I frame at 25 32<br />[mpeg2video @ 0x216a200] skipped MB in I frame at 1 33<br />[mpeg2video @ 0x216a200] ac-tex damaged at 0 34<br />[mpeg2video @ 0x216a200] ac-tex damaged at 0 35<br />[mpeg2video @ 0x216a200] Warning MVs not available<br />[mpeg2video @ 0x216a200] concealing 176 DC, 176 AC, 176 MV errors in I frame<br />push video frame to 393<br />push video frame to 394<----------------------------- 1 PTS Values is missing!!!!!!!<br />push video frame to 396<br />push video frame to 397<br />[mp2 @ 0x21affa0] Header missing<br />[err]failed to decode audio packet<br />push audio frame to 691200 (675)<br />push audio frame to 692224 (676)<br />push audio frame to 693248 (677)<br />push audio frame to 694272 (678)<br />push audio frame to 695296 (679)<br />push audio frame to 696320 (680)<br />push audio frame to 697344 (681)<br />push audio frame to 698368 (682)<br />push audio frame to 699392 (683)<br />push audio frame to 700416 (684)<br />push audio frame to 701440 (685)<br />push audio frame to 702464 (686)<br />push audio frame to 703488 (687)<br />push audio frame to 704512 (688)<br />push audio frame to 705536 (689)<br />push audio frame to 706560 (690)<br />push audio frame to 707584 (691)<br />push audio frame to 708608 (692)<br />push video frame to 398<br />push video frame to 465 <----------------------------- 67 PTS Values are missing!!!!!!!<br />push video frame to 466<br /> </p> <p> </p> <p> </p> <p> </p> <p>my video_push look something like that:</p> <p>int enc_audio_frame (my_enc_t *enc,AVFrame *audio_frame){<br /><br /> if ( enc->first_video_pts == AV_NOPTS_VALUE)<br /> return 0;<br /><br /> if (enc->first_audio_pts == AV_NOPTS_VALUE) {<br /> enc->first_audio_pts = audio_frame->pts;<br /> }<br /><br /> if (audio_frame->pts != AV_NOPTS_VALUE) {<br /> audio_frame->pts = av_rescale_q(audio_frame->pts - enc->first_audio_pts, my_decoder_output.a_time_base, enc->conf.a_time_base);<br /> }<br /> AVPacket *enc_packet = (AVPacket *) malloc (sizeof (AVPacket));<br /> av_init_packet(enc_packet);<br /> enc_packet->size = 0;<br /> enc_packet->data=NULL;<br /> int packet_done = 0;<br /> if ( avcodec_encode_audio2 (enc->audio_codec_ctx,enc_packet,audio_frame,&packet_done) < 0) {<br /> fprintf (stderr,"[err]%s failed to encode audio samples\n", __FUNCTION__);<br /> return -1;<br /> }<br /> if ( packet_done ) {<br /> enc_packet->stream_index= 1;<br /> fprintf (stderr,"push audio frame to %"PRId64" (%"PRId64")\n", audio_frame->pts, audio_frame->pts/1024);<br /> enc->push(enc->push_priv,enc_packet);<br /> }<br /> av_free_packet(enc_packet);<br /> av_free (enc_packet);<br /><br /> return 0;</p> <p>}</p> <p> </p> <p>my audio_push look something like that:</p> <p>int enc_audio_frame (my_enc_t *enc,AVFrame *audio_frame){<br /><br /> if ( enc->first_video_pts == AV_NOPTS_VALUE)<br /> return 0;<br /><br /> if (enc->first_audio_pts == AV_NOPTS_VALUE) {<br /> enc->first_audio_pts = audio_frame->pts;<br /> }<br /><br /> if (audio_frame->pts != AV_NOPTS_VALUE) {<br /> audio_frame->pts = av_rescale_q(audio_frame->pts - enc->first_audio_pts, my_decoder_output.a_time_base, enc->conf.a_time_base);<br /> }<br /> AVPacket *enc_packet = (AVPacket *) malloc (sizeof (AVPacket));<br /> av_init_packet(enc_packet);<br /> enc_packet->size = 0;<br /> enc_packet->data=NULL;<br /> int packet_done = 0;<br /> if ( avcodec_encode_audio2 (enc->audio_codec_ctx,enc_packet,audio_frame,&packet_done) < 0) {<br /> fprintf (stderr,"[err]%s failed to encode audio samples\n", __FUNCTION__);<br /> return -1;<br /> }<br /> if ( packet_done ) {<br /> enc_packet->stream_index= 1;<br /> fprintf (stderr,"push audio frame to %"PRId64" (%"PRId64")\n", audio_frame->pts, audio_frame->pts/1024);</p> <p> enc->push(enc->push_priv,enc_packet);<br /> }<br /> av_free_packet(enc_packet);<br /> av_free (enc_packet);<br /><br /> return 0;</p> <p> </p></div><p> </p><p> </p><p> </p><p> </p><p>If i use the application ffmpeg i got also an async mp4:</p><p> </p><p>ffmpeg -i ~/Videos/defekte_streams/asyncron_1.ts test.mp4<br />ffmpeg version 0.8.6-6:0.8.6-0ubuntu0.12.10.1, Copyright (c) 2000-2013 the Libav developers<br /> built on Apr 2 2013 17:02:16 with gcc 4.7.2<br />*** THIS PROGRAM IS DEPRECATED ***<br />This program is only provided for compatibility and will be removed in a future release. Please use avconv instead.<br />[mp3 @ 0xe71700] Header missing<br />[mpegts @ 0xe6a180] max_analyze_duration reached<br />[NULL @ 0xe8dbc0] start time is not set in estimate_timings_from_pts<br />[NULL @ 0xe8fbe0] start time is not set in estimate_timings_from_pts<br />[mpegts @ 0xe6a180] PES packet size mismatch<br />Input #0, mpegts, from '/home/patrick/Videos/defekte_streams/asyncron_1.ts':<br /> Duration: 00:03:53.69, start: 94926.112233, bitrate: 3423 kb/s<br /> Program 130 <br /> Stream #0.0[0x1411]: Video: mpeg2video (Main), yuv420p, 704x576 [PAR 16:11 DAR 16:9], 10000 kb/s, 25 fps, 25 tbr, 90k tbn, 50 tbc<br /> Stream #0.1[0x1412](deu): Audio: mp2, 48000 Hz, stereo, s16, 192 kb/s<br /> Stream #0.2[0x1414](deu): Subtitle: [6][0][0][0] / 0x0006<br /> Stream #0.3[0x87c]: Data: [11][0][0][0] / 0x000B<br /> Stream #0.4[0xa6e]: Data: [5][0][0][0] / 0x0005<br />File 'test.mp4' already exists. Overwrite ? [y/N] y<br />[buffer @ 0xe9c480] w:704 h:576 pixfmt:yuv420p<br />[libx264 @ 0xe70560] using SAR=16/11<br />[libx264 @ 0xe70560] using cpu capabilities: MMX2 SSE2Fast SSSE3 FastShuffle SSE4.2<br />[libx264 @ 0xe70560] profile Main, level 3.0<br />[libx264 @ 0xe70560] 264 - core 123 r2189 35cf912 - H.264/MPEG-4 AVC codec - Copyleft 2003-2012 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x1:0x111 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=1 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=0 b_adapt=1 b_bias=0 direct=1 weightb=0 open_gop=1 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.25 aq=1:1.00<br />Output #0, mp4, to 'test.mp4':<br /> Metadata:<br /> encoder : Lavf53.21.1<br /> Stream #0.0: Video: libx264, yuv420p, 704x576 [PAR 16:11 DAR 16:9], q=-1--1, 25 tbn, 25 tbc<br /> Stream #0.1(deu): Audio: libvo_aacenc, 48000 Hz, stereo, s16, 200 kb/s<br />Stream mapping:<br /> Stream #0.0 -> #0.0<br /> Stream #0.1 -> #0.1<br />Press ctrl-c to stop encoding<br />[mp2 @ 0xe71700] Header missing<br />Error while decoding stream #0.1<br />[mpegts @ 0xe6a180] Continuity check failed for pid 5137 expected 6 got 8/s dup=21 drop=0 <br />[mpegts @ 0xe6a180] Continuity check failed for pid 18 expected 15 got 7<br />[mpegts @ 0xe6a180] Continuity check failed for pid 5138 expected 7 got 1<br />[mpegts @ 0xe6a180] PES packet size mismatch<br />[mpegts @ 0xe6a180] Continuity check failed for pid 2670 expected 15 got 12<br />[mpegts @ 0xe6a180] Continuity check failed for pid 2172 expected 6 got 9<br />[mpegts @ 0xe6a180] Continuity check failed for pid 0 expected 15 got 3<br />[mpeg2video @ 0xe6f6a0] skipped MB in I frame at 25 32<br />[mpeg2video @ 0xe6f6a0] skipped MB in I frame at 1 33<br />[mpeg2video @ 0xe6f6a0] ac-tex damaged at 0 34<br />[mpeg2video @ 0xe6f6a0] ac-tex damaged at 0 35<br />[mpeg2video @ 0xe6f6a0] Warning MVs not available<br />[mpeg2video @ 0xe6f6a0] concealing 176 DC, 176 AC, 176 MV errors<br />[mp2 @ 0xe71700] Header missing= 1410kB time=14.92 bitrate= 774.4kbits/s dup=21 drop=0 <br />Error while decoding stream #0.1<br />[mpegts @ 0xe6a180] PES packet size mismatchime=230.12 bitrate=1401.1kbits/s dup=88 drop=0 <br />[mpeg2video @ 0xe6f6a0] ac-tex damaged at 36 13<br />[mpeg2video @ 0xe6f6a0] Warning MVs not available<br />[mpeg2video @ 0xe6f6a0] concealing 1012 DC, 1012 AC, 1012 MV errors<br />[mp2 @ 0xe71700] incomplete frame<br />Error while decoding stream #0.1<br />frame= 5851 fps= 28 q=28.0 Lsize= 40194kB time=230.66 bitrate=1427.5kbits/s dup=88 drop=0 <br />video:34406kB audio:5631kB global headers:0kB muxing overhead 0.391637%<br />frame I:58 Avg QP:21.87 size: 27346<br />[libx264 @ 0xe70560] frame P:3250 Avg QP:24.10 size: 8797<br />[libx264 @ 0xe70560] frame B:2543 Avg QP:26.71 size: 1988<br />[libx264 @ 0xe70560] consecutive B-frames: 31.4% 26.7% 15.4% 26.5%<br />[libx264 @ 0xe70560] mb I I16..4: 44.1% 0.0% 55.9%<br />[libx264 @ 0xe70560] mb P I16..4: 11.1% 0.0% 5.9% P16..4: 45.6% 9.3% 4.3% 0.0% 0.0% skip:23.7%<br />[libx264 @ 0xe70560] mb B I16..4: 0.6% 0.0% 0.4% B16..8: 32.4% 2.8% 0.6% direct: 1.9% skip:61.3% L0:36.1% L1:56.3% BI: 7.6%<br />[libx264 @ 0xe70560] coded y,uvDC,uvAC intra: 38.9% 54.4% 16.7% inter: 16.6% 20.4% 1.4%<br />[libx264 @ 0xe70560] i16 v,h,dc,p: 45% 28% 12% 14%<br />[libx264 @ 0xe70560] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 25% 19% 21% 5% 6% 7% 5% 6% 5%<br />[libx264 @ 0xe70560] i8c dc,h,v,p: 51% 24% 20% 4%<br />[libx264 @ 0xe70560] Weighted P-Frames: Y:10.6% UV:6.0%<br />[libx264 @ 0xe70560] ref P L0: 65.7% 11.6% 15.1% 7.2% 0.4%<br />[libx264 @ 0xe70560] ref B L0: 84.6% 15.4%<br />[libx264 @ 0xe70560] kb/s:1204.27</p>
</body>
</html>