[FFmpeg-trac] #8902(avcodec:new): VAAPI pixel format support regression between FFmpeg 3.x and 4.x

[FFmpeg]

#8902: VAAPI pixel format support regression between FFmpeg 3.x and 4.x
-------------------------------------+-------------------------------------
             Reporter:  bmegli       |                    Owner:
                 Type:  defect       |                   Status:  new
             Priority:  normal       |                Component:  avcodec
              Version:  git-master   |               Resolution:
             Keywords:  vaapi        |               Blocked By:
  regression                         |
             Blocking:               |  Reproduced by developer:  1
Analyzed by developer:  0            |
-------------------------------------+-------------------------------------

Comment (by bmegli):

 I have just made some tests.

 I first went with vaapi_scale e.g.

 {{{
 "format=vaapi,scale_vaapi=w=%d:h=%d:format=nv12"
 }}}

 But it occured that the critical difference (scale_vaapi is not necessary)
 lies in


 {{{
 AVHWFramesContext->sw_format
 }}}

 I got working bgr0 with both 3.4 (Ubuntu 18.04) and 4.2 (Ubuntu 20.04) by
 setting


 {{{
 frames_ctx->sw_format = AV_PIX_FMT_NV12;
 }}}

 In other test with 3.4 where I use sw_format as yuyv422 I see in the log:

 {{{
 [h264_vaapi @ 0x558bdfa854e0] Using nv12 as format of reconstructed
 frames.
 }}}

 From what I remember setting yuyv422 sw_format is what would cause error
 in 4.2, at least with H.264 and my Kaby Lake.


 This means that for H.264 I may set NV12 as sw_format for 8 bit data and
 P010LE for 10 bit data and it is ok, at least on my Kaby Lake.

 When looking at larger picture, beyond Kaby Lake it begins to look more
 complex.

 https://github.com/intel/media-driver#decodingencoding-features

 Starting from Ice Lake there is support for HEVC 422 and 444 data
 I wonder if setting frames_ctx->sw_format to AV_PIX_FMT_NV12 would result
 in loss of data in such case.

--
Ticket URL: <https://trac.ffmpeg.org/ticket/8902#comment:8>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker