[FFmpeg-devel] [PATCH] gdigrab: Fix hwnd parameter issues

[James Almer]

On 12/18/2023 5:54 AM, Martin Storsjö wrote:
> Converting from an integer to HWND (which is a pointer) requires
> an explicit cast, otherwise Clang errors out like this:
> 
>      src/libavdevice/gdigrab.c:280:14: error: incompatible integer to pointer conversion assigning to 'HWND' (aka 'struct HWND__ *') from 'long' [-Wint-conversion]
>        280 |         hwnd = strtol(name, &p, 0);
>            |              ^ ~~~~~~~~~~~~~~~~~~~
> 
> (With GCC and MSVC, this was a mere warning, but with recent Clang,
> this is an error.)
> 
> After adding a cast, all compilers also warn something like this:
> 
>      src/libavdevice/gdigrab.c:280:16: warning: cast to 'HWND' (aka 'struct HWND__ *') from smaller integer type 'long' [-Wint-to-pointer-cast]
>        280 |         hwnd = (HWND) strtol(name, &p, 0);
>            |                ^~~~~~~~~~~~~~~~~~~~~~~~~~
> 
> On Windows, long types are 32 bit, so to get a usable pointer, we
> need to use long long. And interpret it as unsigned long long
> while at it - i.e. using strtoull.
> 
> Finally, right above it, the code triggered the following warning:
> 
>      src/libavdevice/gdigrab.c:278:15: warning: mixing declarations and code is incompatible with standards before C99 [-Wdeclaration-after-statement]
>        278 |         char *p;
>            |               ^
> ---
>   libavdevice/gdigrab.c | 4 ++--
>   1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/libavdevice/gdigrab.c b/libavdevice/gdigrab.c
> index 41ef370f2b..b2858ecd89 100644
> --- a/libavdevice/gdigrab.c
> +++ b/libavdevice/gdigrab.c
> @@ -274,10 +274,10 @@ gdigrab_read_header(AVFormatContext *s1)
>       } else if (!strcmp(filename, "desktop")) {
>           hwnd = NULL;
>       } else if (!strncmp(filename, "hwnd=", 5)) {
> -        name = filename + 5;
>           char *p;
> +        name = filename + 5;
>   
> -        hwnd = strtol(name, &p, 0);
> +        hwnd = (HWND) strtoull(name, &p, 0);
>   
>           if (p == NULL || p == name || p[0] == '\0')
>           {

Should be ok.