[FFmpeg-devel] I've written a filter in Rust

[Leandro Santiago]

On 2/20/25 23:49, Michael Niedermayer wrote:
> Hi
>
> On Thu, Feb 20, 2025 at 02:06:47PM +0100, Leandro Santiago wrote:
>> [insert meme here]
> [...]
>> I also recorded a video showing the filter in action [7].
> [...
>> [7] https://youtu.be/U_y4-NnaINg
> cool, it doesnt detect everyone though

Yes, this has two reasons: one is the quality of the detection depends on the model used. I use in the demo a yolov4-tiny that is very fast but not really accurate, leading to many people not being tracked. There can be bugs in the track_sort code as well.

The other reason is that the `track_sort` at the moment ignores all the detected boxes which have a position, but no area (they seem to be detected as simple points by `dnn_detect`), which are useless for now, as the SORT implementation at the moment depends only the boxes dimensions for computing IOU (intersection over union).

I could think though on extending the code to "fake an area" of a box when it's unknown, which might help to improve the tracking.

>
> also i think this shows how useful a plugin framework would be for ffmpeg
>
> with plugins everyone could use,test and contribute to this today.
> without plugins, this needs to be merged in ffmpeg git master. (which
> will take some time i suspect)

Yes, it could even open space for webassembly based plugins, which would be less of a hassle regarding ABI and potentially be cross platform, although I foresee some challenges on getting wasm runtime to share data with the host without copying, especially if the data in the GPU.

>
> Rust support in FFmpeg is an interresting idea though, iam in favor of
> supporting more "safe" languages
Great to hear that!
>
> thx
>
> [...]
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".