[FFmpeg-devel] [PATCH 2/5] lavu/common.h: Fix UB in av_clip_intp2_c()

[Tomas Härdin]

tor 2024-05-30 klockan 00:24 +0200 skrev Andreas Rheinhardt:
> Tomas Härdin:
> >  static av_always_inline av_const int av_clip_intp2_c(int a, int p)
> >  {
> > -    if (((unsigned)a + (1 << p)) & ~((2 << p) - 1))
> > +    if (((unsigned)a + (1U << p)) & ~((2U << p) - 1))
> >          return (a >> 31) ^ ((1 << p) - 1);
> >      else
> >          return a;
> 
> This will support p == 30 (but not 31); but the first change is not
> UB
> in this range.

p=31 is most definitely UB before this change. 1<<31 is signed overflow
with 32-bit int. The compiler has therefore been allowed to do whatever
for p=31 up until this point. To me it seems the intent of the code is
preserved

Personally I dislike bithacks because they are difficult to verify. A
good enough compiler will gain peephole optimizations for them sooner
or later anyway

/Tomas