[FFmpeg-trac] #8747(swscale:new): libswscale 4.3 crash if output buffer is not 16 bytes aligned for yuv2rgb conversion

[FFmpeg]

#8747: libswscale 4.3 crash if output buffer is not 16 bytes aligned for yuv2rgb
conversion
------------------------------------+-----------------------------------
             Reporter:  melanconj   |                    Owner:
                 Type:  defect      |                   Status:  new
             Priority:  important   |                Component:  swscale
              Version:  git-master  |               Resolution:
             Keywords:  regression  |               Blocked By:
             Blocking:              |  Reproduced by developer:  0
Analyzed by developer:  0           |
------------------------------------+-----------------------------------

Comment (by darbyjohnston):

 Hi, this might be related, I was trying to upgrade my software to FFmpeg
 4.3 and am also running into a number of crashes in libswscale with movies
 files that worked fine with 4.2.2. I've made this code snippet to
 reproduce:

 {{{
 #include <libswscale/swscale.h>
 #include <libavutil/frame.h>
 #include <libavutil/imgutils.h>
 #include <stdio.h>

 void scale(int w, int h)
 {
     const enum AVPixelFormat inFmt = AV_PIX_FMT_YUV420P;
     const enum AVPixelFormat outFmt = AV_PIX_FMT_RGBA;
     struct AVFrame* inFrame = NULL;
     struct AVFrame* outFrame = NULL;
     uint8_t* outData = NULL;
     struct SwsContext* context = NULL;

     printf("Scale: %dx%d\n", w, h);

     inFrame = av_frame_alloc();
     inFrame->width = w;
     inFrame->height = h;
     inFrame->format = inFmt;
     av_frame_get_buffer(inFrame, 0);

     outFrame = av_frame_alloc();
     outData = (uint8_t*)malloc(w * h * 4);
     av_image_fill_arrays(outFrame->data, outFrame->linesize, outData,
 outFmt, w, h, 1);

     context = sws_getContext(w, h, inFmt, w, h, outFmt, SWS_BILINEAR,
 NULL, NULL, NULL);
     sws_scale(context, (uint8_t const* const*)inFrame->data,
 inFrame->linesize, 0, h,
         outFrame->data, outFrame->linesize);

     av_frame_free(&inFrame);
     av_frame_free(&outFrame);
     free(outData);
 }

 int main(int argc, char** argv)
 {
     scale(640, 480);
     scale(1920, 1080);
     scale(716, 574); // Valgrind: Invalid write of size 8
     scale(702, 478); // Valgrind: Process terminating with default action
 of signal 11 (SIGSEGV)
     return 0;
 }
 }}}

 Running this with Valgrind on an Ubuntu 20.04 x64 system gives the
 following output:

 {{{
 ==167716== Memcheck, a memory error detector
 ==167716== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
 ==167716== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright
 info
 ==167716== Command: ./a.out
 ==167716==
 Scale: 640x480
 Scale: 1920x1080
 Scale: 716x574
 ==167716== Invalid write of size 8
 ==167716==    at 0x4B73C68: ??? (in /home/darby/dev/DJV-install-
 Debug/lib/libswscale.so.5.7.100)
 ==167716==    by 0x518C1DF: ???
 ==167716==    by 0x51A71FF: ???
 ==167716==  Address 0x60995e0 is 0 bytes after a block of size 1,643,936
 alloc'd
 ==167716==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind
 /vgpreload_memcheck-amd64-linux.so)
 ==167716==    by 0x109325: scale (in /home/darby/dev/DJV-Debug/a.out)
 ==167716==    by 0x109451: main (in /home/darby/dev/DJV-Debug/a.out)
 ==167716==
 Scale: 702x478
 ==167716==
 ==167716== Process terminating with default action of signal 11 (SIGSEGV)
 ==167716==  General Protection Fault
 ==167716==    at 0x4B73C5A: ??? (in /home/darby/dev/DJV-install-
 Debug/lib/libswscale.so.5.7.100)
 ==167716==    by 0x5233FDF: ???
 ==167716==    by 0x52489FF: ???
 ==167716==
 ==167716== HEAP SUMMARY:
 ==167716==     in use at exit: 2,349,684 bytes in 66 blocks
 ==167716==   total heap usage: 133 allocs, 67 frees, 18,277,268 bytes
 allocated
 ==167716==
 ==167716== LEAK SUMMARY:
 ==167716==    definitely lost: 159,024 bytes in 3 blocks
 ==167716==    indirectly lost: 229,026 bytes in 42 blocks
 ==167716==      possibly lost: 0 bytes in 0 blocks
 ==167716==    still reachable: 1,961,634 bytes in 21 blocks
 ==167716==         suppressed: 0 bytes in 0 blocks
 ==167716== Rerun with --leak-check=full to see details of leaked memory
 ==167716==
 ==167716== For lists of detected and suppressed errors, rerun with: -s
 ==167716== ERROR SUMMARY: 2 errors from 1 contexts (suppressed: 0 from 0)
 Segmentation fault (core dumped)
 }}}

--
Ticket URL: <https://trac.ffmpeg.org/ticket/8747#comment:4>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker