[FFmpeg-trac] #11515(avcodec:new): Consider NV12 / P010 output pixel format support

[FFmpeg]

#11515: Consider NV12 / P010 output pixel format support
--------------------------------------+-----------------------------------
             Reporter:  Robert Mader  |                    Owner:  (none)
                 Type:  defect        |                   Status:  new
             Priority:  normal        |                Component:  avcodec
              Version:  unspecified   |               Resolution:
             Keywords:  nv12, p010    |               Blocked By:
             Blocking:                |  Reproduced by developer:  0
Analyzed by developer:  0             |
--------------------------------------+-----------------------------------
Comment (by Robert Mader):

 Replying to [comment:7 Balling]:
 > I still do not get this issue.

 The central points are:
 1. The pixel-format / pixel-representation is not dictated by the codec
 and decoders are free to choose whatever format they see fit.
 2. Unfortunately, for whatever reason, there is a mismatch between the
 formats chosen by most SW decoders like libav, dav1d, vpx etc. and what
 most HW decoders chose. Display engines in turn usually follow the HW
 decoders.
 3. Software can be changed while existing hardware can't. There are
 billions of devices from various vendors out there with display engines
 that support P010 (as defined in Linux DRM, which is compatible Vulkans
 `VK_FORMAT_G10X6_B10X6R10X6_2PLANE_420_UNORM_3PACK16` and, to my
 knowledge, with what Microsoft recommends for Windows, see
 https://learn.microsoft.com/en-us/windows/win32/medfound/10-bit-and-16
 -bit-yuv-video-formats).
 4. The SW decoder >= 10bit formats in turn don't seem to be supported by
 any display engines and, from what I can see, haven't even been defined in
 graphics APIs (the later I'm hoping to change for VK and Linux DRM).
 5. That's why I'm asking to **consider** supporting the most common format
 natively - if that is considered feasible. If the conclusion is that it
 would require too much work, that's totally fine.

 > GPUs adding more garbage formats is also stupid since it adds even more
 transistors in silicon budget. Which will have to be converted to YCbCr or
 RGB in the HDMI anyway...

 Again, existing display engines (which do this job) were for some reason
 build for formats like P010 and not what ffmpeg decided to use.

 > Moroever nvenc supports p010 input and it makes sense to use HW encoder
 more less energy and less problems.

 I don't see what this has to do with the topic. This is about SW decoding,
 not HW encoding on Nvidia hardware.

 > Finally, as I said it is impossible to decode 4k60 VP9 8 bit already in
 software

 I have various older / low power test devices here that can decode 4k60
 VP9 8 bit videos as shipped by youtube fine - AFAICS it mainly depends on
 bitrate and features. In various cases they struggle to get the pixels on
 screen though because doing several buffer-size copies exhausts their
 memory bandwidth. This issue about helping with that in common scenarios
 where buffers can be passed through.

 > so it does not even matter, because with Hardware overlays (color
 management of which Intel will fix soon, I hope) output to display
 directly, CPU just draws the overlay and menus.

 Err, I don't know where this is coming from but it sounds pretty out-
 dated. Proper clients - on Linux/Wayland this includes GTK4-based players
 (see https://blog.gtk.org/2023/11/15/introducing-graphics-offload/),
 Chromium and Firefox (in development / not enabled by default yet) - draw
 overlays on the GPU and arrange things so that the system compositor can
 pass through video content to the display engines.
-- 
Ticket URL: <https://trac.ffmpeg.org/ticket/11515#comment:8>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker