[Libav-user] Decoding h264 (NV12) stream from NVENC

[Michael IV]

Well,Hendrik is right. The decoder knows to detect that the YUV is NV12. I
do get the decoded YUV correct now,but not for the first frame.For some
reason the first frame has completely green color. I do tell the NVENC to
set SPSPPS at the beginning, but still the first frame gets corrupted.
Also, how do I flip the frame vertically on decoding ?

On Jul 31, 2017 2:07 PM, "jing zhang" <zhangjing.lm at gmail.com> wrote:

You should use decoded AVFrame->format to check decoded yuv format.
Maybe H.264 bitstream encoded by NVENC is corrupted?

2017-07-31 16:27 GMT+08:00 Michael IV <explomaster at gmail.com>:

> So you basically say that the decoder 'knows' to detect interleaved UV in
> NV12 UV plane and decode it correctly to yuv420p without any hints from the
> application?
> Then why my YUV frame is corrupted? How can I detect where is the problem?
> I also found this SO answer regarding h264 streaming from NVENC.
> https://stackoverflow.com/questions/33185966/how-to-stream-
> h-264-video-over-udp-using-the-nvidia-nvenc-hardware-encoder
>
> And I currently don't set those params for encoder.Do I have to?
>
>
> On Mon, Jul 31, 2017 at 11:04 AM, Hendrik Leppkes <h.leppkes at gmail.com>
> wrote:
>
>> On Mon, Jul 31, 2017 at 10:00 AM, Michael IV <explomaster at gmail.com>
>> wrote:
>> > Hi! I am using NVENC encoder to create h264 stream.The YUV format is
>> NV12.
>> > Now, I need also to decode that stream back to YUV.I wrote a module for
>> > that,based on
>> > this example:
>> >
>> > https://gist.github.com/roxlu/9329339
>> >
>> > But, because my YUV format is not 420P, but NV12 I am trying to force
>> it on
>> > AVCodecContext when initializing it with this line:
>> >
>> >         avcodec_register_all();
>> >         mCodec = avcodec_find_decoder(AV_CODEC_ID_H264);
>> >         mCodecContext = avcodec_alloc_context3(mCodec);
>> >         mCodecContext->pix_fmt = AVPixelFormat::AV_PIX_FMT_NV12;
>> >
>> >
>> > But when I am receiving the decoded AVFrame, I see that the
>> AVFrame->format
>> > property is
>> > set to zero, which is enum for 420P. Also inspecting the planes and
>> other
>> > props of this struct I see that the decoder tried to decode it as 420P.
>> (3
>> > planes , 3 line sizes etc.) .And of course,when trying to play the
>> output of
>> > such a YUV frame , there is just a green screen.
>> > I am trying to understand where is my error.Is there some data that
>> decoder
>> > misses from the h264 stream and it doesn't allow it to guess the pixel
>> > format?Or maybe I need to set pixel format for the codec context in a
>> some
>> > different way?
>> >
>>
>> You can't control the output pixel format of the H264 decoder. A video
>> is not encoded as "NV12", its encoded as 4:2:0 YUV, NV12 is just one
>> representation of this format.
>> As such, our decoder will always decode 4:2:0 YUV content to yuv420p,
>> not nv12. If you require NV12 in your application, you can convert
>> from yuv420p to nv12 using swscale (a lossless conversion).
>>
>> - Hendrik
>> _______________________________________________
>> Libav-user mailing list
>> Libav-user at ffmpeg.org
>> http://ffmpeg.org/mailman/listinfo/libav-user
>>
>
>
> _______________________________________________
> Libav-user mailing list
> Libav-user at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/libav-user
>
>

_______________________________________________
Libav-user mailing list
Libav-user at ffmpeg.org
http://ffmpeg.org/mailman/listinfo/libav-user
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://ffmpeg.org/pipermail/libav-user/attachments/20170731/703506f1/attachment.html>