[FFmpeg-trac] #6736(avfilter:new): vidstab fails with yuv444 input

FFmpeg trac at avcodec.org
Sat Dec 23 10:00:36 EET 2017

#6736: vidstab fails with yuv444 input
             Reporter:  ochi         |                    Owner:
                 Type:  defect       |                   Status:  new
             Priority:  normal       |                Component:  avfilter
              Version:  unspecified  |               Resolution:
             Keywords:  libvidstab   |               Blocked By:
             Blocking:               |  Reproduced by developer:  0
Analyzed by developer:  0            |

Comment (by TouchMonkey):

 The bit-depth check that is being done in vf_vidstabdetect.c and
 vf_vidstabtransform.c is completely broken because of how FFMPEG and
 livvidstab report how many bits are in a pixel.

 Here's the check from vf_vidstabdetect.c:

  if (fi.bytesPerPixel != av_get_bits_per_pixel(desc)/8) {

 fi.bytesPerPixel is populated by libvidstab by calling vsFrameInfoInit in
 frameinfo.c in the external library. For this function, bytesPerPixel is
 PER PLANE, and is set to 1 for all YUV formats.

 On the other hand, av_get_bits_per_pixel reports the average number of
 bits for all planes. The average part is important since many YUV formats
 share components across multiple pixels.

 For example, for the YUV444P pixel format libvidstab reports that there is
 1 byte per pixel. FFMPEG says there are 24 bits per pixel since it counts
 all planes. This gets divided by 8 for a final value of 3. The check sees
 that 1 != 3 so an error gets thrown.

 Things get even crazier when you look at YUV422, since that format shares
 UV data between pixels. Again, libvidstab sets fi.bytesPerPixel to 1 since
 from libvidstab's perspective that values is how many actual bytes the
 data is taking up after it has been unpacked into the frame buffer.
 libvidstab is tracking that each YUV channel is using a single byte. It's
 all about memory layout. Contrast this to FFMPEG's av_get_bits_per_pixel
 which returns the theoretical, average pixel value of 12 bits. That has
 nothing to do with memory layout, it's about describing the ideal pixel
 format. To make things worse, that 12 gets divided by 8 to become 1 byte.
 This actually passes the check but is actually really broken since the
 value is getting truncated.

 The difference in purpose for the values means there isn't an easy fix for
 this check.

 You can't just multiply fi.bytesPerPixel by fi.planes to compare total
 number of bytes for two reasons. The first is that libvidstab sets
 fi.planes to 0 for RGB and RGBA, so the total number of bytes would be 0.
 It uses a planes value of 0 to mark the format as non-planer. In addition,
 if we look at YUV422 again we get a bytes per pixel value of 3 since there
 are 3 channels of 1 full byte each in memory. This still fails the check
 since 12/8=1 on the FFMPEG side and 1!=3. You can't remove the division on
 the FFMPEG side and multiple the bytes by 8 on the libvidstab side either
 because 3*8=24 bits instead of the correct 12 and 12!=24.

 It really all comes down to the fact that the FFMPEG data and the
 libvidstab data are used for two completely different things. FFMPEG is
 reporting on the ideal, average pixel which is why you get values like 12
 bpp, which isn't evenly divisible by 8. libvidstab is using the data to
 describe how the pixels are laid out in memory. There are no partial
 pixels when the data is unpacked, every YUV sample is contained in 1 byte.

Ticket URL: <https://trac.ffmpeg.org/ticket/6736#comment:2>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker

More information about the FFmpeg-trac mailing list