[FFmpeg-devel] av_fopen_utf8 and cross-DLL CRT object sharing issue on Windows

[Martin Storsjö]

On Sat, 7 May 2022, Soft Works wrote:

>> -----Original Message-----
>> From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of
>> Martin Storsjö
>> Sent: Wednesday, April 20, 2022 2:48 PM
>> To: ffmpeg-devel at ffmpeg.org
>> Subject: [FFmpeg-devel] av_fopen_utf8 and cross-DLL CRT object sharing
>> issue on Windows
>>
>> Hi,
>>
>> I just became aware of the av_fopen_utf8 function - which was
>> introduced
>> to fix path name translations on Windows - actually has a notable
>> design
>> flaw.
>
> Hi Martin,
>
> I just became aware that somebody would be compiling ffmpeg like
> this on Windows and I'm curious regarding the whereabouts..
>
>> Background:
>>
>> On Windows, a process can contain more than one C runtime (CRT); the
>> system comes with two shared ones (UCRT and msvcrt.dll) and in MSVC
>> builds, each DLL/EXE can have one statically linked in instead of
>> linking
>> against a shared library CRT (and that's actually the default
>> configuration when building with MSVC).
>
> The default configuration for both, EXE and DLL projects is to link
> to the C runtime dynamically (crt dll).

No, that's not true. If you invoke e.g. "cl.exe myprog.c" without 
explicitly passing either -MD or -MT, the default is -MT, which is to 
statically link the CRT.

>> This means that CRT objects (file descriptors from open(), FILE*
>> opened
>> with fopen/fdopen) mustn't be shared across DLLs; such an object must
>> be
>> opened, accessed and closed within the same DLL.
>
> This only happens when you explicitly modify the build configuration
> to statically link to the CRT.

No, this is not a custom build configuration. This is the build 
configuration you get if you configure with "configure --enable-shared 
--toolchain=msvc".

> Why are you compiling it this way?
> Your earlier patch is from 2013, so you seem to be doing so for
> quite a while.

It's not that I'm shipping a production setup built this way. I just spent 
a fair amount of work to make ffmpeg work when built with MSVC; both built 
as static libraries and as DLLs. And I'd like to keep that working. Not 
only on the "seems to work for whatever is covered by fate" level, but 
also on the level of not using constructs that are known to not work.

As av_fopen_utf8 gets duplicated across the libraries by being in the same 
source file as the other functions, it works for all uses across the 
libraries, but doesn't work for uses outside of the libraries (fftools, 
external API users). That's why I'm hesitant against increasing the use of 
this function in fftools until we have resolve this issue.

Also, another fairly common situation where the "different CRTs" scenario 
happens if you'd e.g. build the ffmpeg libraries as DLLs with mingw, but 
then link against those DLLs with a user application built with MSVC. Then 
it's not an issue with the function between the libraries, but it is an 
issue for use of av_fopen_utf8 in the user's code outside of libav*.

> Is the file API the only case where you had any trouble?

As far as I remember, that was the only case of cross-library resource 
sharing issue I ran into at the time.

> I don't have experience with that kind of setup, but I would have 
> thought that with separate CRTs, you could already get into trouble when 
> you would allocate a string in the main application which you pass to 
> any of the DLL's APIs and which might get freed by the DLL at a later 
> time - doesn't that fail?

No, it doesn't (as Andreas explained).

// Martin