[FFmpeg-devel] Debug builds and if (ARCH_...) linking issues

[Aaron Levinson]

On 4/13/2017 11:04 AM, Matt Oliver wrote:
> On 14 April 2017 at 03:31, Hendrik Leppkes <h.leppkes at gmail.com> wrote:
>
>> On Thu, Apr 13, 2017 at 7:16 PM, Matt Oliver <protogonoi at gmail.com> wrote:
>>> On 14 April 2017 at 02:11, Rostislav Pehlivanov <atomnuker at gmail.com>
>> wrote:
>>>
>>>>
>>>>
>>>> On 13 April 2017 at 16:51, wm4 <nfxjfg at googlemail.com> wrote:
>>>>
>>>>> On Thu, 13 Apr 2017 17:39:57 +1000
>>>>> Matt Oliver <protogonoi at gmail.com> wrote:
>>>>>
>>>>>> On 13 April 2017 at 17:20, Aaron Levinson <alevinsn at aracnet.com>
>> wrote:
>>>>>>
>>>>>>> I wanted to build a debug build of ffmpeg using Visual C++ today,
>> one
>>>>>>> without any optimizations.  This implies the use of the -Od
>> compiler
>>>>>>> option.  Unfortunately, I quickly discovered that the build fails
>> soon
>>>>>>> after it starts because it can't find certain architecture-specific
>>>>>>> references.  For example, in libavutil/cpu.c, there is the
>> following:
>>>>>>>
>>>>>>>     if (ARCH_AARCH64)
>>>>>>>         return ff_get_cpu_flags_aarch64();
>>>>>>>
>>>>>>> The linker couldn't find ff_get_cpu_flags_aarch64 (and other
>> similar
>>>>>>> references) and failed.  This isn't an issue when optimizations are
>>>>> turned
>>>>>>> on because the compiler notices that ARCH_AARCH64 is defined as 0
>> and
>>>>>>> eliminates the relevant code.
>>>>>>>
>>>>>>> Effectively, successful builds of ffmpeg depend on this compiler
>>>>>>> optimization.  This appears to have been the standard practice in
>> the
>>>>>>> ffmpeg code base for at least the last few years, but it is unclear
>>>>> to me
>>>>>>> why this approach is being used, since, in addition to depending on
>>>>>>> specific compiler behavior, it prevents fully debug builds from
>>>>> succeeding,
>>>>>>> at least with Visual C++.
>>>>>>>
>>>>>>> If people like the if (ARCH_...) syntax, while it wouldn't look
>> quite
>>>>> as
>>>>>>> nice, what's wrong with doing the following:
>>>>>>>
>>>>>>> #if ARCH_AARCH64
>>>>>>>     if (ARCH_AARCH64)
>>>>>>>         return ff_get_cpu_flags_aarch64();
>>>>>>> #endif
>>>>>>>
>>>>>>> Another, much less desirable option is to use #pragma optimize for
>> the
>>>>>>> relevant functions in ffmpeg to turn optimizations on for specific
>>>>>>> functions.
>>>>>>>
>>>>>>> A third option would be to build only the relevant files with
>>>>>>> optimizations turned on, but this will end up making the Makefiles
>>>>> more
>>>>>>> complicated, and the relative simplicity of the Makefiles is
>>>>> appealing.
>>>>>>>
>>>>>>> For now, I'm using both -Od and -Og with Visual C++ (-Og turns on
>> some
>>>>>>> optimizations, but not as much as -O1 or -O2), but this isn't the
>>>>> same as a
>>>>>>> true debug build.
>>>>>>>
>>>>>>
>>>>>> Similar patches have been submitted before. This is an issue with
>> Dead
>>>>> Code
>>>>>> Elimination (DCE) within FFmpeg and the fact the MSVC doesn't support
>>>>>> removing it in debug builds.
>>>>>>
>>>>>> There have been some discussions on the mailing list in the past
>> about
>>>>>> resolving this but nothing was ever decided.
>>>>>>
>>>>>> As a quick and dirty work around I have a tool that i wrote that
>> scans
>>>>> in
>>>>>> the configure/makefile from a ffmpeg distro and generates a native
>>>>> Visual
>>>>>> Studio project file that can be used to just compile within Visual
>>>>> Studio
>>>>>> itself. You just pass it the desired configure options that you want
>> to
>>>>> use
>>>>>> to build the project and it will make one for you. The main thing is
>>>>> that
>>>>>> it scans the source and automatically generates the missing DCE
>> sections
>>>>>> and adds them so that everything will compile correctly with Debug
>>>>> builds
>>>>>> and you can debug directly in VS. You can find the tool here
>>>>>> http://shiftmediaproject.github.io/1-projects/ (normally i wouldn't
>> put
>>>>>> external references on the mailing list but this may be of use to
>> you).
>>>>>
>>>>> Any chance you could revive your old patches to remove the DCE
>>>>> requirement? (Not sure if there were any patches.)
>>>>>
>>>>> Before you do any real work, make a thread on the ML requesting
>>>>> comments on this. Although I would very much welcome such patches, I'm
>>>>> not fully sure about others.
>>>>>
>>>>> This DCE requirement is pretty painful, and affects debugging on Linux
>>>>> as well.
>>>>>
>>>>
>>> I put up a general discussion a while ago (
>>> http://ffmpeg.org/pipermail/ffmpeg-devel/2016-December/204530.html) but
>>> there were a few people who opposed a direct removal of DCE and no one
>>> really came up with a consensus as to what the acceptable approach would
>> be.
>>>
>>
>> I wouldn't like any weird hacks in the source just to work-around the
>> lack of DCE in debug builds, so we should decide to either keep using
>> it or get rid of it.
>
>
> I agree.

It seems that the DCE requirement affects debug builds in general, and 
not just with Windows builds.  And, because of the DCE requirement, it 
is necessary to introduce some level of optimization in debug builds, 
which makes it harder to debug.  Sure, I can use -Zo with MSVC on 
Windows (see https://msdn.microsoft.com/en-us/library/dn785163.aspx for 
more details) to make it easier to debug release builds, but that's not 
the same as debugging a fully debug build, and something similar may not 
exist with other compilers.

I reviewed the thread mentioned by Matt Oliver above, and one of the 
arguments against removing DCE was that it may make some people less 
likely to work on the project.  Nicolas George wrote:

"Someone's personal preferences affect their willingness to work on the
project. Since the project is perpetually short on manpower, this is a
very serious issue."

The same argument can be used regarding the unwillingness to eliminate 
DCE.  Eliminating DCE would make it easier to debug ffmpeg, and a 
secondary benefit to making it easier to debug ffmpeg would be that 
individuals could be more productive and more likely to contribute to 
ffmpeg.  And, I would argue that easier debugging takes precedence to 
personal preference on whether or not preprocessor directives are ugly.

It seems to me that the quickest approach to eliminating DCE would be 
for someone that is good with regular expressions and 
perl/sed/awk/python etc to write a script that looks for DCE instances 
and adjusts the code properly.  I would think that the simplest approach 
would be to take something like:

     if (ARCH_AARCH64)
         return ff_get_cpu_flags_aarch64();

and replace it with:

#if ARCH_AARCH64
     if (ARCH_AARCH64)
         return ff_get_cpu_flags_aarch64();
#endif

Such a script could be reviewed and checked into the source base, then 
applied to the source base, and possibly reused in the future as necessary.

I also have a compromise approach in mind if no agreement can be 
reached, but I think it is best to put energy into the DCE/no-DCE 
discussion instead at this point.

Aaron Levinson