[FFmpeg-devel] [GSOC] [PATCH] TensorFlow backend introduction for DNN module

Pedro Arthur bygrandao at gmail.com
Sun Jun 3 21:41:14 EEST 2018


2018-06-03 15:25 GMT-03:00 Sergey Lavrushkin <dualfal at gmail.com>:
> 2018-06-03 19:57 GMT+03:00 Pedro Arthur <bygrandao at gmail.com>:
>>
>> 2018-05-31 12:01 GMT-03:00 Sergey Lavrushkin <dualfal at gmail.com>:
>> > Hello,
>> >
>> > This patch introduces TensorFlow backend for DNN inference module.
>> > This backend uses TensorFlow binary models and requires from model
>> > to have the operation named 'x' as an input operation and the operation
>> > named 'y' as an output operation. Models are executed using
>> > libtensorflow.
>>
>> Hi,
>>
>> You added the tf model in dnn_srcnn.h, it seems the data is being
>> duplicated as it already contains the weights as C float arrays.
>> Is it possible to construct the model graph via C api and set the
>> weights using the ones we already have, eliminating the need for
>> storing the whole tf model?
>
>
> Hi,
>
> I think, it is possible, but it will require to manually create every
> operation
> and specify each of their attributes and inputs in a certain order specified
> by
> operations declaration. Here is that model:
> https://drive.google.com/file/d/1s7bW7QnUfmTaYoMLPdYYTOLujqNgRq0J/view?usp=sharing
> It is just a lot easier to store the whole model and not construct it
> manually.
> Another way, I think of, is to pass weights in placeholders and not save
> them in
> model, but it has to be done when session is already created and not during
> model
> loading. Maybe some init operation can be specified with variables
> assignment to values
> passed through placeholders during model loading, if it is possible. But is
> it really crucial
> to not store the whole tf model? It is not that big.

My concern is when we add more models, currently we have to store 2
models, one for the "native" implementation and one for the TF
backend.
There is also the case were one wants to update the weights for a
model, it will be necessary to update both the native and TF data.
Having duplicated data is much easier to get inconsistencies between
implementations.


More information about the ffmpeg-devel mailing list