[FFmpeg-devel] [WIP] XComposite window capture demuxer (Linux)

[Emanuele Oriani]

Hi Marton,

Capturing at 60 FPS 1920x1080, the difference is even more noticeable: 
x11grab is choppy, xcompgrab is smooth (this time I used H264 ultrafast 
to encode).

https://send.firefox.com/download/17001ef60837a5ec/#FaQTa-dP4MB28YfPNMXxuw

CPU performance:

              real    user   sys
- xcompgrab: 10.32s, 18.2s, 0.38s
- x11grab:   10.20s, 11.6s, 0.22s

Definitely much more CPU usage, but the output quality can't be compared.
Please note no frames were lost/skipped, everything was processed on 
time for both tests.

Thanks,
Emanuele

On 08/05/2020 11:05, Emanuele Oriani wrote:
> Hi Marton,
> 
> TL;DR
> xcompgrab uses more CPU but produces much better streams than x11grab.
> 
> I have the following CPU usage performance report (on my i7-8700k, 
> Nvidia 2080 Ti RTX, governor set to 'performance', on Ubuntu 18.04 using 
> Gnome shell).
> The capture has been a surface area of  1720x1376 for 10 seconds at 30 
> FPS, only getting the frames, not trying to encode them into a file:
> 
>               real    user   sys
> - xcompgrab: 10.13s, 2.06s, 0.085s
> - x11grab:   10.05s, 0.20s, 0.035s
> 
> from a first outlook x11grab is much more CPU efficient, almost 90% less 
> CPU usage, but upon starting to record the video, it feels like it's 
> 'choppier' and loses frames.
> 
> Below videos have been captured at 1920x1080, the encoder is exactly the 
> same with same settings and the total CPU time is greater for xcompgrab 
> (as expected).
> 
> But if you look at both, the xcompgrab doesn't lose frames/is not 
> choppier, instead x11grab seems to be much worse.
> 
> I've uploaded both 10 secs segments at:
> https://send.firefox.com/download/0cbf01f9126519c7/#DPKzIfjemAFm263ZSdMOBw
> 
> The code used to grab and encode is https://github.com/Emanem/replayer 
> again very simple, one thread gets the packets/frames, the other picks 
> the frames and encodes them.
> 
> Performance of this run has been:
> 
>               real    user   sys
> - xcompgrab: 10.42s, 12.5s, 0.22s
> - x11grab:   10.21s, 8.90s, 0.31s
> 
> As expected x11grab yields better CPU usage but the quality of output is 
> somehow lacking.
> 
> Let me know your thoughts.
> 
> Thanks,
> Emanuele
> 
> On 07/05/2020 19:52, Marton Balint wrote:
>>
>>
>> On Thu, 7 May 2020, Emanuele Oriani wrote:
>>
>>> Hi FFMPEG devel,
>>>
>>> I have been writing a simple XComposite window capture demuxer, 
>>> heavily inspired from x11grab sources and OBS Window capture logic/code.
>>
>> Have you compared performance to x11grab for various resolutions and 
>> frame rates? Do you have some numbers?
>>
>> Thanks,
>> Marton
>>
>>>
>>> I would like to give back to the community and share the sources; 
>>> before I would formally submit for a review/patch, I would really 
>>> appreciate if someone could have a high level pass at the code (it's 
>>> relatively simple actually) and point out any major issues/problems/...
>>>
>>> The code is available as part of a simple test application on github, 
>>> at:
>>>
>>> https://github.com/Emanem/replayer/blob/master/src/xcompgrab.c
>>>
>>> Being the author of the code I would be happy to re-license it to be 
>>> used in ffmpeg/libav* mainline once I get the OK from you.
>>> In order to compile/execute this code, one needs the following dev 
>>> packages:
>>>
>>> libav*-dev, libx11-dev, libxcomposite-dev, OpenGL headers/libs.
>>>
>>> Libraries to be linked are:
>>>
>>> -lavcodec -lavformat -lavdevice -lavutil -lswscale -lX11 -lXcomposite 
>>> -lGL
>>>
>>> I've tested it and it's working fine on both AMD and Nvidia hardware.
>>>
>>> Let me know if this is of interest, otherwise no worries and thanks 
>>> again for the fantastic library/software.
>>>
>>> Emanuele
>>>
>>> _______________________________________________
>>> ffmpeg-devel mailing list
>>> ffmpeg-devel at ffmpeg.org
>>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>>>
>>> To unsubscribe, visit link above, or email
>>> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>> _______________________________________________
>> ffmpeg-devel mailing list
>> ffmpeg-devel at ffmpeg.org
>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>>
>> To unsubscribe, visit link above, or email
>> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".