[Libav-user] Syncing a seperate audio and video stream

Ludwig Ertl ert at csp.at
Tue Mar 20 19:17:43 CET 2012


I want to use ffmpeg to decode Audio and video streams by a
Trendnet TV-IP512P IP Camera in order to feed it to ffserver for streaming.
I found a document on the Internet which is describing the video and
audio format container used by the IP camera:
See Section 8.2 (Advanced ip-Camera Stream(ACS) Header.

Unfortunately, Audio and Video streams are served seperately under different
URLs (See Section 4.1.5 and 4.1.6): /video/ACVS.cgi and /audio/ACAS.cgi

Now I wrote 2 decoding plugins for libavformat (which I'd like to
contribute to libavformat once they are working as expected), which I have
attached in this message.
They basically work fine, but as I'm completely new to libav, I don't have
an idea how to sync those 2 streams together.
Currently, what I'm doing for testing is:

ffmpeg -i http://admin:xxx@ -i
http://admin:xxx@ test.mpg

So I have 2 seperate streams which are of course out of sync.
There is a timestamp field in the frame-Header of each audio/videoframe,
is just a unix-timestamp with msec precision, both from the same clock
So this information could be used to sync the streams, but I have no clue
this could possibly work, as Audio- and Videodecoder plugins don't know any-
thing from each other and even if they would (via an external variable or
ugly hack like that), I don't have a clue how to sync them. I suspect that
may have something to do with PTS and DTS timestamps, but I don't know how
they are used for audio and video sync in seperate streams.

Can you give me some hints on how to solve this problem? I can provide
data from the camera, if you want.
As you may see from the code, I'm also quite unsure how to correctly
PTS (and possibly DTS) values, but the current code at least somehow works,
if I don't use any seeking or offsets (i.e. ffmpeg -ss parameter).

I hope that there is a possibility for synchronisation.
What I have already tried was using the clock as PTS for both audio and

av_set_pts_info(st, 64, 1, 1000000);  /* 64 bits pts in us */
pkt->pts = ac->hdr.ulTimeSec * 1000000LL + ac->hdr.ulTimeUSec;

But this just resulted in a totally garbled video stream.

DI (FH) Ludwig Ertl
Software Engineering
Computer Software Production GmbH
Fockygasse 4
1120 Wien
Web: http://www.csp.at
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: acsdec.c
URL: <http://ffmpeg.org/pipermail/libav-user/attachments/20120320/0024d022/attachment.c>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: acsvdec.c
URL: <http://ffmpeg.org/pipermail/libav-user/attachments/20120320/0024d022/attachment-0001.c>

More information about the Libav-user mailing list