[FFmpeg-devel] [WIP] XComposite window capture demuxer (Linux)

[Emanuele Oriani]

Hi Marton,

TL;DR
xcompgrab uses more CPU but produces much better streams than x11grab.

I have the following CPU usage performance report (on my i7-8700k, 
Nvidia 2080 Ti RTX, governor set to 'performance', on Ubuntu 18.04 using 
Gnome shell).
The capture has been a surface area of  1720x1376 for 10 seconds at 30 
FPS, only getting the frames, not trying to encode them into a file:

              real    user   sys
- xcompgrab: 10.13s, 2.06s, 0.085s
- x11grab:   10.05s, 0.20s, 0.035s

from a first outlook x11grab is much more CPU efficient, almost 90% less 
CPU usage, but upon starting to record the video, it feels like it's 
'choppier' and loses frames.

Below videos have been captured at 1920x1080, the encoder is exactly the 
same with same settings and the total CPU time is greater for xcompgrab 
(as expected).

But if you look at both, the xcompgrab doesn't lose frames/is not 
choppier, instead x11grab seems to be much worse.

I've uploaded both 10 secs segments at:
https://send.firefox.com/download/0cbf01f9126519c7/#DPKzIfjemAFm263ZSdMOBw

The code used to grab and encode is https://github.com/Emanem/replayer 
again very simple, one thread gets the packets/frames, the other picks 
the frames and encodes them.

Performance of this run has been:

              real    user   sys
- xcompgrab: 10.42s, 12.5s, 0.22s
- x11grab:   10.21s, 8.90s, 0.31s

As expected x11grab yields better CPU usage but the quality of output is 
somehow lacking.

Let me know your thoughts.

Thanks,
Emanuele

On 07/05/2020 19:52, Marton Balint wrote:
> 
> 
> On Thu, 7 May 2020, Emanuele Oriani wrote:
> 
>> Hi FFMPEG devel,
>>
>> I have been writing a simple XComposite window capture demuxer, 
>> heavily inspired from x11grab sources and OBS Window capture logic/code.
> 
> Have you compared performance to x11grab for various resolutions and 
> frame rates? Do you have some numbers?
> 
> Thanks,
> Marton
> 
>>
>> I would like to give back to the community and share the sources; 
>> before I would formally submit for a review/patch, I would really 
>> appreciate if someone could have a high level pass at the code (it's 
>> relatively simple actually) and point out any major issues/problems/...
>>
>> The code is available as part of a simple test application on github, at:
>>
>> https://github.com/Emanem/replayer/blob/master/src/xcompgrab.c
>>
>> Being the author of the code I would be happy to re-license it to be 
>> used in ffmpeg/libav* mainline once I get the OK from you.
>> In order to compile/execute this code, one needs the following dev 
>> packages:
>>
>> libav*-dev, libx11-dev, libxcomposite-dev, OpenGL headers/libs.
>>
>> Libraries to be linked are:
>>
>> -lavcodec -lavformat -lavdevice -lavutil -lswscale -lX11 -lXcomposite 
>> -lGL
>>
>> I've tested it and it's working fine on both AMD and Nvidia hardware.
>>
>> Let me know if this is of interest, otherwise no worries and thanks 
>> again for the fantastic library/software.
>>
>> Emanuele
>>
>> _______________________________________________
>> ffmpeg-devel mailing list
>> ffmpeg-devel at ffmpeg.org
>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>>
>> To unsubscribe, visit link above, or email
>> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".