[FFmpeg-user] A53 Captions Garbled If Frame Duplication Occurs

[Devin Heitmueller]

Hi Shane,

On Fri, May 3, 2024 at 5:48 PM Shane Warren <shanew at innovsys.com> wrote:
>
> I ingest some multicast video that has a53 captions embedded in h264 video, the video claims to have a framerate of 29.97 but it must not quite be that because ffmpeg duplicates frames when transcoding this stream. The output stream has a53 captions, but they appear to be somewhat garbled, like some letters are duplicated, sometimes.
>
> I decided to git bisect since I knew this worked in a much older ffmpeg binary I had. I finally found the commit that broke this:
>
> a11ab647304307d0340fd052e04cf638b6bef309 : fftools/ffmpeg: share the code encoding a single frame between video and audio
>
> Call do_video_stats() for every video packet produced by the encoder,
> rather than for every frame sent to the encoder.
>
> ------------------------------------------------------------------------
>
> The key part that was missed in this commit was calling this directly after avcodec_send_frame:
>
> // Make sure Closed Captions will not be duplicated
> av_frame_remove_side_data(in_picture, AV_FRAME_DATA_A53_CC);
>
> I fixed this in that commit, and captions worked again. I tried to fix it the same way in a the current ffmpeg source by changing:
>
> ffmpeg_enc.c encode_frame:
>
>     ret = avcodec_send_frame(enc, frame);
>     if (ret < 0 && !(ret == AVERROR_EOF && !frame)) {
>         av_log(ost, AV_LOG_ERROR, "Error submitting %s frame to the encoder\n",
>                type_desc);
>         return ret;
>     }
>
>     +if (frame != NULL)
>     +{
>     +   av_frame_remove_side_data(frame, AV_FRAME_DATA_A53_CC);
>     +}
>
> But that didn't fix the problem, I suspect this is needed when a frame is duplicated, but I can't figure out the place to put that.
>
> Anyone have any ideas what I need to change to get captions working in the current ffmpeg source?
> _______________________________________________
> ffmpeg-user mailing list
> ffmpeg-user at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
> To unsubscribe, visit link above, or email
> ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".

This was discussed last May in the following thread:

https://ffmpeg.org/pipermail/ffmpeg-devel/2023-June/310534.html

The easiest thing for you to do is use the "fps" filter (i.e. -vf
fps=60000/1001) rather than using "-r".  That makes use of the ccfifo
framework to ensure that the data is properly preserved.  The hack to
do it with "-r" actually doesn't work correctly in a number of cases
(e.g. reducing the framerate), and in fact even with upconverting to
59.94 the data doesn't actually conform to the spec, although it works
in some decoders (specifically you should have every frame with a
cc_count of 10, when in fact you have it alternating between 20 and
being absent).

Devin

-- 
Devin Heitmueller, Senior Software Engineer
LTN Global Communications
o: +1 (301) 363-1001
w: https://ltnglobal.com  e: devin.heitmueller at ltnglobal.com