[FFmpeg-trac] #7649(avutil:new): FFmpeg QSV initialization fails on KBL HadesCanyon (= both Intel & AMD GPUs) when DISPLAY is set

[FFmpeg]

#7649: FFmpeg QSV initialization fails on KBL HadesCanyon (= both Intel & AMD
GPUs) when DISPLAY is set
--------------------------------+--------------------------------------
             Reporter:  eero-t  |                     Type:  defect
               Status:  new     |                 Priority:  normal
            Component:  avutil  |                  Version:  git-master
             Keywords:          |               Blocked By:
             Blocking:          |  Reproduced by developer:  0
Analyzed by developer:  0       |
--------------------------------+--------------------------------------
 Setup:
 * Ubuntu 18.04
 * KBL i7-8809G: https://ark.intel.com/products/130409/Intel-Core-i7-8809G-
 Processor-with-Radeon-RX-Vega-M-GH-graphics-8M-Cache-up-to-4-20-GHz-
 * Latest FFmpeg, MediaSDK, libva, intel-driver and gmmlib built from
 sources at GitHub
 * drm-git kernel (e.g. v4.19 or v4.20)


 To reproduce, use FFmpeg QSV backend.  For example:
 {{{
 ffmpeg -hwaccel qsv -c:v mpeg2_qsv -i 1920x1080i_29.97_20mb_mpeg2_high.mpv
 -c:v h264_qsv -b:v 6M 0024_HD17i7_1.0.h264
 ...
 Stream mapping:
   Stream #0:0 -> #0:0 (mpeg2video (mpeg2_qsv) -> h264 (h264_qsv))
 Press [q] to stop, [?] for help
 DRM_IOCTL_I915_GEM_APERTURE failed: Invalid argument
 Assuming 131072kB available aperture size.
 May lead to reduced performance or incorrect rendering.
 get chip id failed: -1 [2]
 param: 4, val: 0
 [AVHWDeviceContext @ 0x55a0ad0ee600] Failed to initialise VAAPI
 connection: 1 (operation failed).
 Error creating a QSV device
 qsv hwaccel requested for input stream #0:0, but cannot be initialized.
 Error while decoding stream #0:0: Operation not permitted
 [mpeg2_qsv @ 0x55a0ad0f8e80] video_get_buffer: image parameters invalid
 [mpeg2_qsv @ 0x55a0ad0f8e80] get_buffer() failed
 Error while decoding stream #0:0: Invalid argument
 [mpeg2_qsv @ 0x55a0ad0f8e80] video_get_buffer: image parameters invalid
 [mpeg2_qsv @ 0x55a0ad0f8e80] get_buffer() failed
 ...
 [mpeg2_qsv @ 0x55a0ad0f8e80] Too many errors when draining, this is a bug.
 Stop draining and force EOF.
 Error while decoding stream #0:0: Internal bug, should not have happened
 [h264_qsv @ 0x55a0ad0fa2c0] Selected ratecontrol mode is unsupported
 [h264_qsv @ 0x55a0ad0fa2c0] Current frame rate is unsupported
 [h264_qsv @ 0x55a0ad0fa2c0] Current picture structure is unsupported
 [h264_qsv @ 0x55a0ad0fa2c0] Current resolution is unsupported
 [h264_qsv @ 0x55a0ad0fa2c0] Current pixel format is unsupported
 [h264_qsv @ 0x55a0ad0fa2c0] some encoding parameters are not supported by
 the QSV runtime. Please double check the input parameters.
 Error initializing output stream 0:0 -- Error while opening encoder for
 output stream #0:0 - maybe incorrect parameters such as bit_rate, rate,
 width or height
 Conversion failed!
 }}}

 It's expected to work, because it works:
 * on other KBL devices (GT2, GT3e)
 * using FFmpeg with VA-API acceleration instead of QSV with same drivers
 * using MediaSDK sample transcode application to do the same thing

 QSV backend works if DISPLAY environment variable is unset.

 From "strace -f -e openat" output I can see following files to have been
 opened, when DISPLAY is not set:
 {{{
 ...
 <input video>
 <output video>
 /dev/dri/renderD128
 /path/to/iHD_drv_video.so
 }}}

 Whereas with DISPLAY set, following are opened instead:
 {{{
 ...
 <input video>
 <output video>
 /home/user/.Xauthority
 /dev/dri/card1
 /path/to/iHD_drv_video.so
 }}}


 When using "latrace" to trace library calls, without DISPLAY I see
 following:
 {{{
 ...
 XOpenDisplay(nil)
 XDisplayName(0, , , 4720)
 vaGetDisplayDRM(5, , 2, 0)
 vaSetErrorCallback(, , , )
 vaSetInfoCallback(, , , )
 vaInitialize(, , , ) = 0
 }}}

 Whereas with DISPLAY set, it goes:
 {{{
 ...
 XOpenDisplay(nil)
 vaGetDisplay(, 85, , 0)
 XDisplayName(0, 0, , )
 vaSetErrorCallback(, , , )
 vaSetInfoCallback(, , , )
 vaInitialize(, , , ) = 1
 }}}

 Above differences seem to come from "vaapi_device_create":
 https://github.com/FFmpeg/FFmpeg/blob/master/libavutil/hwcontext_vaapi.c#L1482

 Even if user specifies correct device for FFmpeg with {{{-hwaccel_device
 /dev/dri/renderD128}}}, same thing is done when DISPLAY is set, although X
 doesn't have anything to do with video transcoding.

--
Ticket URL: <https://trac.ffmpeg.org/ticket/7649>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker