<div dir="ltr"><div>Hello,</div><div><br></div><div>I am interested in storing custom metadata on a per frame level in a mov or mp4 file. It looks like using <span class="gmail-blob-code-inner gmail-blob-code-marker gmail-js-code-nav-pass gmail-js-skip-tagsearch">user data unregistered (UDU) supplemental enhancement information (SEI) may be a suitable approach for this. I am however struggling to get this to work as I am always reading back the custom message I used for the first frame.</span></div><div><span class="gmail-blob-code-inner gmail-blob-code-marker gmail-js-code-nav-pass gmail-js-skip-tagsearch"><br></span></div><div><span class="gmail-blob-code-inner gmail-blob-code-marker gmail-js-code-nav-pass gmail-js-skip-tagsearch">Below is a basic change I made to the mux.c and demux_decode.c examples from ffmpeg that shows my attempt:</span></div><div><a href="https://gist.github.com/tvercaut/94c68e46d0d0321a2c83b88024d9cd69/revisions">https://gist.github.com/tvercaut/94c68e46d0d0321a2c83b88024d9cd69/revisions</a></div><div><br></div><div>Essentially, I have tried to use</div><div>`<span class="gmail-blob-code-inner gmail-blob-code-marker gmail-js-code-nav-pass"><span class="gmail-pl-en">av_frame_new_side_data_from_buf</span>(f<span class="gmail-pl-c1">rame</span>, <span class="gmail-pl-c1">AV_FRAME_DATA_SEI_UNREGISTERED</span>, <span class="gmail-pl-s1">my_sei_buffer</span>);`</span></div><div><span class="gmail-blob-code-inner gmail-blob-code-marker gmail-js-code-nav-pass">at muxing time and</span></div><div><span class="gmail-blob-code-inner gmail-blob-code-marker gmail-js-code-nav-pass">`</span><span class="gmail-blob-code-inner gmail-blob-code-marker gmail-js-code-nav-pass"><span class="gmail-pl-en">av_frame_get_side_data</span>(<span class="gmail-pl-s1">frame</span>, <span class="gmail-pl-c1">AV_FRAME_DATA_SEI_UNREGISTERED</span>);`</span></div><div><span class="gmail-blob-code-inner gmail-blob-code-marker gmail-js-code-nav-pass">at demuxing/decoding time.</span></div><div><span class="gmail-blob-code-inner gmail-blob-code-marker gmail-js-code-nav-pass"><br></span></div><div><span class="gmail-blob-code-inner gmail-blob-code-marker gmail-js-code-nav-pass">I also realised I had to set the `</span><span class="gmail-blob-code-inner gmail-blob-code-marker gmail-js-code-nav-pass"><span class="gmail-pl-s">udu_sei` flag when muxing:<br></span></span></div><div><span class="gmail-blob-code-inner gmail-blob-code-marker gmail-js-code-nav-pass">`</span><span class="gmail-blob-code-inner gmail-blob-code-marker gmail-js-code-nav-pass"><span class="gmail-pl-en">av_dict_set_int</span>(<span class="gmail-pl-c1">&</span><span class="gmail-pl-s1">opt</span>, <span class="gmail-pl-s">"udu_sei"</span>, <span class="gmail-pl-c1">1</span>, <span class="gmail-pl-c1">0</span>);`</span></div><div><span class="gmail-blob-code-inner gmail-blob-code-marker gmail-js-code-nav-pass"><br></span></div><div><span class="gmail-blob-code-inner gmail-blob-code-marker gmail-js-code-nav-pass">I have looked for additional material / documentation on how to do this but couldn't find a lot. Am I missing a key step here?</span></div><div><span class="gmail-blob-code-inner gmail-blob-code-marker gmail-js-code-nav-pass"><br></span></div><div><span class="gmail-blob-code-inner gmail-blob-code-marker gmail-js-code-nav-pass">Also, is there a way to display the UDU SEI messages using ffmpeg or ffprobe command line tools? I tried using</span></div><div><span class="gmail-blob-code-inner gmail-blob-code-marker gmail-js-code-nav-pass">`</span>ffmpeg -i test.mov -vf showinfo -f null -`</div><div>but this doesn't display the content to the UDU SEI messages I provided.<br></div><div><br></div><div>P.S.: This is a bit of a follow-up from an old post of mine but trying a different approach for per-frame metadata: <a href="https://lists.ffmpeg.org/pipermail/libav-user/2022-April/013051.html">https://lists.ffmpeg.org/pipermail/libav-user/2022-April/013051.html</a></div><div><br></div><div>Best wishes,</div><div>Tom<br></div></div>