[Libav-user] Please help me about transcoding to TS stream: H.264 bitstream malformed, no startcode found, use the h264_mp4toannexb bitstream filter (-bsf h264_mp4toannexb)
Li Zhang
lizhang at utelisys.com
Fri Mar 9 16:20:39 CET 2012
Hi everyone,
I am a new user to ffmpeg API. I want to transcode mpeg2 video stream to H.264+AAC and then put them into TS container again.
I have implemented transcoding from mpeg2 TS to AVI, MKV, MP4 local files. Now the problem is when I used the same code put those H.264+AAC data packet into TS container, the video can not be played and only the audio can be played. Actually, there was video packet data in the .ts file which was derived from trandcoding.
And I got the information from ffmpeg that "H.264 bitstream malformed, no startcode found, use the h264_mp4toannexb bitstream filter (-bsf h264_mp4toannexb)". So I initiated the h264_mp4toannexb bitstreamfilter and use av_bitstream_filter_filter to process the packet before write them to output file. However, av_bitstream_filter_filter returns 0 and the final output file still the same situation and the warning information still happen.
There is another warning information about DTS, it only happen every 29 frames. That is so strange. Please help me. I used the latest version of ffmpeg.
When I used the Elecard Stream Analyzer to check my transcoded file, I found the file missed SPS and SEI information for the video stream. There is only SPS and SEI information for audio stream. I do not know why the TS container can not produce these information for me.
Here is the information about ffmpeg:
===============================================================================================================================
ffmpeg version git-2012-03-09-d07de6d Copyright (c) 2000-2012 the FFmpeg developers
built on Mar 9 2012 14:57:55 with gcc 4.5.2
configuration: --enable-gpl --enable-libfaac --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libtheora --enable-libvorbis --enable-libx264 --enable-nonfree --enable-version3 --enable-x11grab --enable-shared
libavutil 51. 42.100 / 51. 42.100
libavcodec 54. 10.100 / 54. 10.100
libavformat 54. 2.100 / 54. 2.100
libavdevice 53. 4.100 / 53. 4.100
libavfilter 2. 63.100 / 2. 63.100
libswscale 2. 1.100 / 2. 1.100
libswresample 0. 7.100 / 0. 7.100
libpostproc 52. 0.100 / 52. 0.100
Hyper fast Audio and Video encoder
Here is the information I got from ffmpeg during transcoding:
==============================================================================================================================
Format mpegts probed with size=2048 and score=100
stream=0 stream_type=6 pid=29 prog_reg_desc=
stream=1 stream_type=3 pid=5c prog_reg_desc=
stream=2 stream_type=2 pid=a3 prog_reg_desc=
parser not found for codec dvb_teletext, packets or times may be invalid.
mpeg_decode_postinit() failure
mpeg_decode_postinit() failure
mpeg_decode_postinit() failure
mpeg_decode_postinit() failure
mpeg_decode_postinit() failure
max_analyze_duration 5000000 reached at 5016000
Continuity check failed for pid 163 expected 0 got 13
Continuity check failed for pid 163 expected 3 got 15
Continuity check failed for pid 92 expected 12 got 14
PES packet size mismatch
Continuity check failed for pid 163 expected 11 got 2
PES packet size mismatch
PES packet size mismatch
Input #0, mpegts, from 'bbc.mpg':
Duration: 00:02:52.68, start: 20886.724667, bitrate: 3801 kb/s
Program 10050
Metadata:
service_name : BBC World
service_provider: BBC
Stream #0:0[0x29](eng), 129, 1/90000: Subtitle: dvb_teletext ([6][0][0][0] / 0x0006)
Stream #0:1[0x5c](eng), 211, 1/90000: Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, 2 channels, s16, 256 kb/s
Stream #0:2[0xa3], 130, 1/90000: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p, 720x576 [SAR 64:45 DAR 16:9], 1/50, 15000 kb/s, 25.60 fps, 25 tbr, 90k tbn, 50 tbc
detected 8 logical cores
Output #0, mpegts, to 'my.ts':
Stream #0:0, 0, 1/90000: Video: h264 (hq), yuv420p, 1280x720, 1/25, q=10-51, 1500 kb/s, 90k tbn, 25 tbc
Stream #0:1, 0, 1/90000: Audio: aac (LC), 48000 Hz, 2 channels, s16, 96 kb/s
frame MB size (80x45) > level limit (1620)
DPB size (4 frames, 5529600 bytes) > level limit (2 frames, 3110400 bytes)
MB rate (90000) > level limit (40500)
using cpu capabilities: MMX2 SSE2Fast SSSE3 FastShuffle SSE4.2 AVX
profile High, level 3.0
264 - core 122 r2183 c522ad1 - H.264/MPEG-4 AVC codec - Copyleft 2003-2012 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=umh subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=2 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=2 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=12 keyint_min=7 scenecut=40 intra_refresh=0 rc_lookahead=25 rc=cbr mbtree=1 bitrate=1500 ratetol=1.0 qcomp=0.60 qpmin=10 qpmax=51 qpstep=4 vbv_maxrate=1500 vbv_bufsize=1500 nal_hrd=none ip_ratio=1.40 aq=1:1.00
muxrate VBR, pcr every 2 pkts, sdt every 200, pat/pmt every 40 pkts
frame= 0 QP=27.00 NAL=3 Slice:I Poc:0 I:3600 P:0 SKIP:0 size=32702 bytes
H.264 bitstream malformed, no startcode found, use the h264_mp4toannexb bitstream filter (-bsf h264_mp4toannexb)
non-strictly-monotonic PTS
frame= 1 QP=32.31 NAL=2 Slice:P Poc:6 I:41 P:605 SKIP:2954 size=1969 bytes
frame= 2 QP=40.12 NAL=2 Slice:B Poc:4 I:0 P:302 SKIP:3287 size=608 bytes
frame= 3 QP=39.80 NAL=0 Slice:B Poc:2 I:0 P:308 SKIP:3284 size=564 bytes
frame= 4 QP=27.67 NAL=2 Slice:P Poc:12 I:85 P:1246 SKIP:2269 size=5319 bytes
frame= 5 QP=39.93 NAL=2 Slice:B Poc:10 I:0 P:459 SKIP:3139 size=559 bytes
frame= 6 QP=40.25 NAL=0 Slice:B Poc:8 I:0 P:490 SKIP:3102 size=717 bytes
frame= 7 QP=25.05 NAL=2 Slice:P Poc:20 I:162 P:1835 SKIP:1603 size=8799 bytes
frame= 8 QP=32.99 NAL=2 Slice:B Poc:16 I:1 P:1005 SKIP:2580 size=1377 bytes
frame= 9 QP=36.74 NAL=0 Slice:B Poc:14 I:0 P:710 SKIP:2886 size=831 bytes
frame= 10 QP=36.07 NAL=0 Slice:B Poc:18 I:1 P:543 SKIP:3053 size=721 bytes
frame= 11 QP=23.64 NAL=2 Slice:P Poc:22 I:126 P:1552 SKIP:1922 size=7150 bytes
frame= 12 QP=22.30 NAL=3 Slice:I Poc:0 I:3600 P:0 SKIP:0 size=53773 bytes
frame= 13 QP=21.03 NAL=2 Slice:P Poc:6 I:179 P:2275 SKIP:1146 size=12912 bytes
frame= 14 QP=23.29 NAL=2 Slice:B Poc:4 I:3 P:1043 SKIP:2521 size=2953 bytes
frame= 15 QP=26.52 NAL=0 Slice:B Poc:2 I:0 P:1096 SKIP:2492 size=1467 bytes
frame= 16 QP=23.99 NAL=2 Slice:P Poc:14 I:59 P:1311 SKIP:2230 size=5514 bytes
frame= 17 QP=28.52 NAL=2 Slice:B Poc:10 I:1 P:772 SKIP:2804 size=1684 bytes
frame= 18 QP=31.24 NAL=0 Slice:B Poc:8 I:1 P:541 SKIP:3051 size=902 bytes
frame= 19 QP=29.40 NAL=0 Slice:B Poc:12 I:0 P:594 SKIP:3002 size=962 bytes
frame= 20 QP=23.63 NAL=2 Slice:P Poc:22 I:104 P:1514 SKIP:1982 size=7512 bytes
frame= 21 QP=28.40 NAL=2 Slice:B Poc:18 I:0 P:927 SKIP:2658 size=2009 bytes
frame= 22 QP=31.98 NAL=0 Slice:B Poc:16 I:0 P:641 SKIP:2952 size=1072 bytes
frame= 23 QP=30.60 NAL=0 Slice:B Poc:20 I:1 P:623 SKIP:2968 size=1133 bytes
frame= 24 QP=21.10 NAL=3 Slice:I Poc:0 I:3600 P:0 SKIP:0 size=60984 bytes
frame= 25 QP=22.74 NAL=2 Slice:P Poc:6 I:124 P:1511 SKIP:1965 size=6791 bytes
frame= 26 QP=30.00 NAL=2 Slice:B Poc:4 I:0 P:97 SKIP:3503 size=151 bytes
frame= 27 QP=51.00 NAL=0 Slice:B Poc:2 I:0 P:66 SKIP:3534 size=141 bytes
frame= 28 QP=23.64 NAL=2 Slice:P Poc:14 I:83 P:1392 SKIP:2125 size=7147 bytes
frame= 29 QP=29.88 NAL=2 Slice:B Poc:10 I:3 P:794 SKIP:2784 size=1703 bytes
Application provided invalid, non monotonically increasing dts to muxer in stream 0: 93600 >= 93600
non-strictly-monotonic PTS
===================================================================================================================
Here is the main code which I used:
===================================================================================================================
bsfc = av_bitstream_filter_init("h264_mp4toannexb");
if (!bsfc) {
printf("Cannot open the h264_mp4toannexb BSF!\n");
return 1;
}
int write_packet(AVFormatContext *oc, AVPacket *pkt, AVCodecContext *avctx, AVStream *ost) {
if (bsfc) {
AVPacket new_pkt = *pkt;
int a = av_bitstream_filter_filter(bsfc, avctx, NULL,
&new_pkt.data, &new_pkt.size,
pkt->data, pkt->size,
pkt->flags & AV_PKT_FLAG_KEY);
if (a > 0) {
av_free_packet(pkt);
new_pkt.destruct = av_destruct_packet;
} else if (a < 0) {
printf("Failed to open bitstream filter %s for stream %d with codec %s",
bsfc->filter->name, pkt->stream_index,
avctx->codec ? avctx->codec->name : "copy");
}
*pkt = new_pkt;
//bsfc = bsfc->next;
}
pkt->stream_index = ost->index;
if(av_write_frame(oc, pkt) != 0) {
printf("write has finished\n");
return -1;
}
return 0;
}
... ...
while(av_read_frame(ic, &packet)>=0 && frame_index < 7000) {
if(packet.stream_index == videoindex) {
avcodec_get_frame_defaults(oVFrame);
len = avcodec_decode_video2(vCodecCtx, oVFrame, &frameFinished, &packet);
if(len<0) {
printf("Error while decoding\n");
continue;
}
else if(0 == frameFinished ) {
printf("Error decoding and frameFinished is 0\n");
continue;
}
else if(frameFinished) {
//if it is needed, rescale the frame
AVFrame encode_frame;
AVFrame temp_frame;
if(scale_need && video_scale_context) {
if(avpicture_alloc((AVPicture*)&temp_frame, oVcc->pix_fmt, oVcc->width, oVcc->height)) {
printf("the tempory frame can not be allocated\n");
return 1;
}
//scale the video size for the encoder
sws_scale(video_scale_context, oVFrame->data,oVFrame->linesize,
0, vCodecCtx->height,
temp_frame.data,
temp_frame.linesize);
encode_frame = temp_frame;
printf("the frame is scaled !!!\n" );
}
else {
encode_frame = *oVFrame;
}
encode_frame.interlaced_frame = oVFrame->interlaced_frame;
encode_frame.pict_type = AV_PICTURE_TYPE_NONE;
encode_frame.quality = oVFrame->quality;
encode_frame.pts = frame_index;
video_outbuf = (unsigned char *) av_malloc(video_outbuf_size);
memset(video_outbuf, 0, video_outbuf_size);
//out_size = avcodec_encode_video2(oVcc, &output_video_packet, oVFrame, &is_valid_encode);
out_size = avcodec_encode_video3(oVcc, video_outbuf, video_outbuf_size, &encode_frame);
//if (is_valid_encode)
if(out_size > 0) {
//encapsue the data into packet and configure some parameters
AVPacket output_video_packet;
av_init_packet(&output_video_packet);
output_video_packet.data = video_outbuf;
output_video_packet.size = out_size;
//configure the key frame
if(oVcc->coded_frame && oVcc->coded_frame->key_frame)
output_video_packet.flags |= AV_PKT_FLAG_KEY;
output_video_packet.stream_index = video_st->index;
//configure the pts and dts
if(oVcc->coded_frame && oVcc->coded_frame->pts != AV_NOPTS_VALUE) {
output_video_packet.pts = av_rescale_q(oVcc->coded_frame->pts, oVcc->time_base,video_st->time_base);
}
output_video_packet.dts = AV_NOPTS_VALUE;
//output_video_packet.dts = 3600 * (frame_index - 25);
int ret = write_packet(oc, &output_video_packet, oVcc, video_st);
if(ret!=0) {
printf("while write video frame error\n");
continue;
}
if (oVcc->coded_frame->pict_type == AV_PICTURE_TYPE_B)
printf("AV_PICTURE_TYPE_B\n");
else if (oVcc->coded_frame->pict_type == AV_PICTURE_TYPE_I)
printf("AV_PICTURE_TYPE_I\n");
else if (oVcc->coded_frame->pict_type == AV_PICTURE_TYPE_P)
printf("AV_PICTURE_TYPE_p\n");
printf("packet: pts is:%d, dts is: %d\n", packet.pts, packet.dts);
printf("the pts is: %d, and the dts is: %d, oVcc->coded_frame->pts is: %d\n", output_video_packet.pts, output_video_packet.dts, oVcc->coded_frame->pts);
//printf("video: the decoded packet and has been encoded has the packet size is: %d\n", packet.size);
printf("video pacekt is written and frame_index is %d\n", frame_index);
//printf(" video: the packet size is: %d\n", output_video_packet.size);
}
else {
printf("video: encoding is failed, is_valid_encode is: %d\n", is_valid_encode);
printf("video: encoding is failed, out_size is: %d\n", out_size);
}
frame_index++;
av_free(video_outbuf);
avpicture_free((AVPicture*)&temp_frame); //bug ??
}
else {
printf("video:frameFinished is %d\n", frameFinished);
}
}
else if(packet.stream_index==audioindex) {
int ret=0;
int is_audio_valid = 0;
avcodec_get_frame_defaults(oAFrame);
ret=avcodec_decode_audio4(aCodecCtx,oAFrame,&is_audio_valid, &packet);
if(ret<0 || is_audio_valid < 0) {
printf("audio: while decode audio failure\n");
continue;
}
else if(0 == is_audio_valid) {
// Audio was not decoded
printf("audio: Error decoiding audio frame result_bytes=0 bytes_decoded= %d\n", ret);
}
else {
//fifo the audio data if input codec context->frame_size not equal output codec context->frame_size------>
uint8_t *buf = oAFrame->data[0];
//decoded audio frame size
int tmp_size = aCodecCtx->frame_size * aCodecCtx->channels * av_get_bytes_per_sample(aCodecCtx->sample_fmt);
//encoding audio frame size
int frame_bytes = oAcc->frame_size * oAcc->channels * av_get_bytes_per_sample(oAcc->sample_fmt);
uint8_t *buftmp = NULL;
buftmp = (uint8_t*) av_realloc(buftmp, frame_bytes);
// write into the fifo buffer
av_fifo_generic_write(fifo_buf, buf, tmp_size, NULL);
while(av_fifo_size(fifo_buf)>=frame_bytes) { // loop if there are enough audio data
av_fifo_generic_read(fifo_buf, buftmp, frame_bytes, NULL);
//initialize encode_audio_frame
AVFrame *encode_audio_frame = NULL;
encode_audio_frame = avcodec_alloc_frame();
if(!encode_audio_frame) {
printf("Can not allocate encode_audio_frame.\n");
continue;
}
avcodec_get_frame_defaults(encode_audio_frame);
//configure the filling length for encode_audio_frame
encode_audio_frame->nb_samples = 1024;
//fill the data in tmp into the encode_audio_frame
if(avcodec_fill_audio_frame(encode_audio_frame,
oAcc->channels,
oAcc->sample_fmt,
buftmp, frame_bytes, 1) < 0) {
printf("audio: encoding_audio_frame is failed.\n");
continue;
}
AVPacket pkt;
av_init_packet(&pkt);
audio_outbuf = (uint8_t*)av_malloc(audio_outbuf_size);
memset(audio_outbuf, 0, audio_outbuf_size);
pkt.size = audio_outbuf_size;
pkt.data = audio_outbuf;
oVFrame->pts = AV_NOPTS_VALUE;
audio_out_size= avcodec_encode_audio2(oAcc, &pkt, encode_audio_frame ,&is_audio_valid);
if(0 > audio_out_size){
printf("audio: the encoding is error \n");
continue;
}
else if(0 == is_audio_valid) {
printf("audio: Error encoding audio, the is_audio_valid is 0\n");
continue;
}
else if(is_audio_valid) {
if(oAcc->coded_frame && (oAcc->coded_frame->pts != AV_NOPTS_VALUE)) {
//pkt.pts = av_rescale_q(oAcc->coded_frame->pts, oAcc->time_base, audio_st->time_base);
}
pkt.flags |= AV_PKT_FLAG_KEY;
pkt.stream_index= audio_st->index;
pkt.dts = AV_NOPTS_VALUE;
#if 1
if (write_packet(oc, &pkt,oVcc,audio_st) != 0) {
fprintf(stderr, "audio: Error while writing audio frame\n");
continue;
}
#endif
printf("audio: audio packet is written.\n");
}
av_free(pkt.data);
pkt.size = 0;
audio_outbuf = NULL;
av_free(encode_audio_frame);
}// the end of while
av_free(buftmp);
buftmp = NULL;
}
}
av_free_packet(&packet);
}
if(0 != av_write_trailer(oc)) {
printf("the trailer can not be written!");
}
=============================================================================================================================================
Best regards,
Li
More information about the Libav-user
mailing list