[FFmpeg-devel] [PATCH]Do not store more than 30 bit for r10k

[Reimar Döffinger]

On Sat, Dec 27, 2014 at 12:41:12AM +0100, Carl-Eugen Hoyos wrote:
> diff --git a/libavcodec/r210enc.c b/libavcodec/r210enc.c
> index d61cd75..4cbebd7 100644
> --- a/libavcodec/r210enc.c
> +++ b/libavcodec/r210enc.c
> @@ -62,7 +62,7 @@ static int encode_frame(AVCodecContext *avctx, AVPacket *pkt,
>              if (avctx->codec_id == AV_CODEC_ID_R210)
>                  pixel = (r << 20) | (g << 10) | b >> 2;
>              else
> -                pixel = (r << 22) | (g << 12) | b;
> +                pixel = (r << 22) | (g << 12) | b & 0xFFFFFFFC;

Wouldn't this be more obvious as
| (b & 0xffc);

Or even better with more context (saves one operation for the R210 case, and no change to your patch for the other case
- though I have to admit it almost looks like the extra 2 bits for blue were intended originally):

@@ -58,11 +58,11 @@ static int encode_frame(AVCodecContext *avctx, AVPacket *pkt,
             uint32_t pixel;
             uint16_t r = *src++ >> 6;
             uint16_t g = *src++ >> 6;
-            uint16_t b = *src++ >> 4;
+            uint16_t b = *src++ >> 6;
             if (avctx->codec_id == AV_CODEC_ID_R210)
-                pixel = (r << 20) | (g << 10) | b >> 2;
+                pixel = (r << 20) | (g << 10) | b;
             else
-                pixel = (r << 22) | (g << 12) | b;
+                pixel = (r << 22) | (g << 12) | (b << 2);
             if (avctx->codec_id == AV_CODEC_ID_AVRP)
                 bytestream_put_le32(&dst, pixel);
             else