[FFmpeg-cvslog] libavcodec/qsvenc: Do not pass RGB solorspace to VPL/MSDK

[Wenbin Chen]

ffmpeg | branch: master | Wenbin Chen <wenbin.chen at intel.com> | Mon Feb 13 16:49:26 2023 +0800| [e530d38bbf5f58e3e22f87dff635a50e267abb77] | committer: Haihao Xiang

libavcodec/qsvenc: Do not pass RGB solorspace to VPL/MSDK

When encode RGB frame, Intel driver convert RGB to YUV, so we cannot
set RGB colorspace to VPL/MSDK.

Signed-off-by: Wenbin Chen <wenbin.chen at intel.com>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=e530d38bbf5f58e3e22f87dff635a50e267abb77
---

 libavcodec/qsvenc.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/libavcodec/qsvenc.c b/libavcodec/qsvenc.c
index 2f0e94a914..d3f7532fc0 100644
--- a/libavcodec/qsvenc.c
+++ b/libavcodec/qsvenc.c
@@ -1185,7 +1185,12 @@ static int init_video_param(AVCodecContext *avctx, QSVEncContext *q)
         q->extvsi.ColourDescriptionPresent = 1;
         q->extvsi.ColourPrimaries = avctx->color_primaries;
         q->extvsi.TransferCharacteristics = avctx->color_trc;
-        q->extvsi.MatrixCoefficients = avctx->colorspace;
+        if (avctx->colorspace == AVCOL_SPC_RGB)
+            // RGB will be converted to YUV, so RGB colorspace is not supported
+            q->extvsi.MatrixCoefficients = AVCOL_SPC_UNSPECIFIED;
+        else
+            q->extvsi.MatrixCoefficients = avctx->colorspace;
+
     }
 
     if ((avctx->codec_id != AV_CODEC_ID_VP9) && (q->extvsi.VideoFullRange || q->extvsi.ColourDescriptionPresent)) {