x11grab BadCursor

[idooley]

Summary of the bug:
ffmpeg exits when calling "xcim = XFixesGetCursorImage(dpy);" from x11grab.c, if X11 cursor has not been set yet. In most X11 servers, a cursor has already been set. But in simple Xvfb instances, the cursor may not yet have been set.

Affected Versions: 0.11.1, 47277c4, f7cba73 on Ubuntu 12.10

How to reproduce:

% Xvfb :10 -screen 0 800x600x24 &

% ./ffmpeg -an -f x11grab -s 800x600 -r 30 -i :10.0 -vcodec libx264  -threads 1 ${HOME}/capturedvideoXvfb.avi
ffmpeg version 0.11.1 Copyright (c) 2000-2012 the FFmpeg developers
  built on Sep 15 2012 14:48:09 with gcc 4.6.3
  configuration: --enable-network --enable-protocol=tcp --enable-libx264 --enable-x11grab --enable-gpl
  libavutil      51. 54.100 / 51. 54.100
  libavcodec     54. 23.100 / 54. 23.100
  libavformat    54.  6.100 / 54.  6.100
  libavdevice    54.  0.100 / 54.  0.100
  libavfilter     2. 77.100 /  2. 77.100
  libswscale      2.  1.100 /  2.  1.100
  libswresample   0. 15.100 /  0. 15.100
  libpostproc    52.  0.100 / 52.  0.100
[x11grab @ 0x1614360] device: :10.0 -> display: :10.0 x: 0 y: 0 width: 800 height: 600
[x11grab @ 0x1614360] shared memory extension found
X Error of failed request:  BadCursor (invalid Cursor parameter)
  Major opcode of failed request:  142 (XFIXES)
  Minor opcode of failed request:  25 (XFixesGetCursorImageAndName)
  Resource id in failed request:  0x200001
  Serial number of failed request:  14
  Current serial number in output stream:  14

or, for a different ffmpeg version:

% Xvfb :10 -screen 0 800x600x24 &

%  ./ffmpeg -an -f x11grab -s 800x600 -r 30 -i :10.0 -vcodec libx264  -threads 1 ${HOME}/capturedvideoXvfb.avi
rm: cannot remove `/home/isaacdooley/capturedvideo*': No such file or directory
ffmpeg version N-42091-gf7cba73 Copyright (c) 2000-2012 the FFmpeg developers
  built on Sep 15 2012 14:38:41 with gcc 4.6.3
  configuration: --enable-network --enable-protocol=tcp --enable-libx264 --enable-x11grab --enable-gpl
  libavutil      51. 63.100 / 51. 63.100
  libavcodec     54. 31.100 / 54. 31.100
  libavformat    54. 14.100 / 54. 14.100
  libavdevice    54.  0.100 / 54.  0.100
  libavfilter     3.  0.100 /  3.  0.100
  libswscale      2.  1.100 /  2.  1.100
  libswresample   0. 15.100 /  0. 15.100
  libpostproc    52.  0.100 / 52.  0.100
[x11grab @ 0x1eeb380] device: :10.0 -> display: :10.0 x: 0 y: 0 width: 800 height: 600
[x11grab @ 0x1eeb380] shared memory extension found
X Error of failed request:  BadCursor (invalid Cursor parameter)
  Major opcode of failed request:  142 (XFIXES)
  Minor opcode of failed request:  25 (XFixesGetCursorImageAndName)
  Resource id in failed request:  0x200001
  Serial number of failed request:  14
  Current serial number in output stream:  14

Fix:
I found a post in the mailing list that appears to correctly identify the problem and reference a fix to a perl module for this same issue: http://ffmpeg.org/pipermail/ffmpeg-user/2012-August/008804.html

I've made some changes to x11grab.c, similar to those in the perl module, namely adding a call to set the cursor immediately before calling XFixesGetCursorImage(). The patch below makes ffmpeg work properly:

$ git diff
diff --git a/libavdevice/x11grab.c b/libavdevice/x11grab.c
index b6bd486..b8271d5 100644
--- a/libavdevice/x11grab.c
+++ b/libavdevice/x11grab.c
@@ -43,6 +43,7 @@
 #include "libavutil/parseutils.h"
 #include "libavutil/time.h"
 #include <time.h>
+#include <X11/cursorfont.h>
 #include <X11/X.h>
 #include <X11/Xlib.h>
 #include <X11/Xlibint.h>
@@ -357,6 +358,12 @@ paint_mouse_pointer(XImage *image, struct x11grab *s)
     if (image->bits_per_pixel != 24 && image->bits_per_pixel != 32)
         return;
 
+    Cursor c = XCreateFontCursor(dpy, XC_left_ptr);
+    Window w = DefaultRootWindow(dpy);
+    XSetWindowAttributes attr;
+    attr.cursor = c;
+    XChangeWindowAttributes(dpy, w, CWCursor, &attr);
+
     xcim = XFixesGetCursorImage(dpy);
 
     x = xcim->x - xcim->xhot;

[jensb]

I had to modify the above patch a bit, so it works with the ffmpeg version 0.8.3, that ships with Ubuntu 12.04.

I'll put it here, so it may be useful for others. Hope you do not mind.

--- libav-0.8.3.orig/libavdevice/x11grab.c
+++ libav-0.8.3/libavdevice/x11grab.c
@@ -43,6 +43,7 @@
 #include "libavutil/opt.h"
 #include "libavutil/parseutils.h"
 #include <time.h>
+#include <X11/cursorfont.h>
 #include <X11/X.h>
 #include <X11/Xlib.h>
 #include <X11/Xlibint.h>
@@ -342,6 +343,7 @@ paint_mouse_pointer(XImage *image, struc
     int x, y;
     int line, column;
     int to_line, to_column;
+    Cursor c; Window w; XSetWindowAttributes attr;
     int pixstride = image->bits_per_pixel >> 3;
     /* Warning: in its insanity, xlib provides unsigned image data through a
      * char* pointer, so we have to make it uint8_t to make things not break.
@@ -353,6 +355,11 @@ paint_mouse_pointer(XImage *image, struc
     if (image->bits_per_pixel != 24 && image->bits_per_pixel != 32)
         return;
 
+    c = XCreateFontCursor(dpy, XC_left_ptr);
+    w = DefaultRootWindow(dpy);
+    attr.cursor = c;
+    XChangeWindowAttributes(dpy, w, CWCursor, &attr);
+
     xcim = XFixesGetCursorImage(dpy);
 
     x = xcim->x - xcim->xhot;

[cehoyos]

Fixed by Isaac Dooley.