[FFmpeg-devel] H.264 decoder calls fill_rectangle() with unaligned buffers

[Michael Niedermayer]

On Mon, Jul 14, 2008 at 02:58:18PM +0100, M?ns Rullg?rd wrote:
> 
> Michael Niedermayer wrote:
> > On Mon, Jul 14, 2008 at 12:36:47AM +0100, M?ns Rullg?rd wrote:
> >> If STRIDE_ALIGN is set to 16, the H.264 decoder will call
> >> fill_rectangle() with buffer addresses failing the alignment assert()
> >> at the top of the function.  This obviously does no harm to this
> >> function, so I'm not sure which is incorrect, the assert() or the
> >> calling code.
> >
> > And i am not sure without knowing what that calling code is.
> 
> Sorry, I forgot to mention.  It can happen with any of the
> fill_rectangle(h->mv_cache[list][ scan8[0] ], 4, 4, 8, val, 4)
> and similar calls where width*size > 8, for instance the one
> on line 4784 of h264.c.

I suspect you are missing a
#define DECLARE_ALIGNED_8(t, v) DECLARE_ALIGNED(16, t, v)
in dsputil.h where STRIDE_ALIGN is set to 16

(yes DECLARE_ALIGNED_8 is not a good name i know, it should be
renamed to DECLARE_ALIGNED_NATIVE or _MAX or so)

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

There will always be a question for which you do not know the correct awnser.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20080714/168dc27d/attachment.pgp>