[FFmpeg-devel] [PATCH] get rid of nonsense palette pointer for RGB8, GRAY8 etc.

[Reimar Döffinger]

On Fri, Mar 20, 2009 at 09:12:49PM +0100, Michael Niedermayer wrote:
> On Fri, Mar 20, 2009 at 08:30:38PM +0100, Reimar D?ffinger wrote:
> > About fixing the swscale planarCopy, are you fine with this ugly hack (I
> > assume there is no need to "reset" the GRAY8 palette data in dst):
> 
> no, sws_scale() passes dst through and this would end up changing the
> users dst[] i think

Sorry, that was a thinking error. Even more ugly, but this is what I
meant:
Index: libswscale/swscale.c
===================================================================
--- libswscale/swscale.c        (revision 29018)
+++ libswscale/swscale.c        (working copy)
@@ -2107,10 +2107,15 @@
     return srcSliceH;
 }
 
-static int planarCopy(SwsContext *c, uint8_t* src[], int srcStride[], int srcSliceY,
-                      int srcSliceH, uint8_t* dst[], int dstStride[])
+static int planarCopy(SwsContext *c, uint8_t* srcp[], int srcStride[], int srcSliceY,
+                      int srcSliceH, uint8_t* dstp[], int dstStride[])
 {
     int plane;
+    uint8_t *src[4] = {srcp[0], srcp[1], srcp[2], srcp[3]};
+    uint8_t *dst[4] = {dstp[0], dstp[1], dstp[2], dstp[3]};
+    // ignore palette for GRAY8
+    if (!dst[2]) dst[1] = NULL;
+    if (!src[2]) src[1] = NULL;
     for (plane=0; plane<4; plane++)
     {
         int length= (plane==0 || plane==3) ? c->srcW  : -((-c->srcW)>>c->chrDstHSubSample);