[FFmpeg-devel] [PATCH] lavc/vaapi_encode_h264: add option to insert AUD nal to AVC stream

Jun Zhao mypopydev at gmail.com
Wed May 3 06:06:20 EEST 2017



On 2017/5/3 4:23, Mark Thompson wrote:
> On 02/05/17 03:49, Jun Zhao wrote:
>> From e39078e9fe02c8d77c5e28436aef4d80a2d7b3a0 Mon Sep 17 00:00:00 2001
>> From: Jun Zhao <jun.zhao at intel.com>
>> Date: Tue, 2 May 2017 10:36:55 +0800
>> Subject: [PATCH] lavc/vaapi_encode_h264: add option to insert AUD nal to AVC
>>  stream.
>>
>> Add aud option to support insert AUD nal in AVC stream. Default is
>> disable.
>>
>> Signed-off-by: Jun Zhao <jun.zhao at intel.com>
>> Signed-off-by: Yi A Wang <yi.a.wang at intel.com>
>> ---
>>  libavcodec/vaapi_encode.c      | 15 ++++++++++++++
>>  libavcodec/vaapi_encode.h      |  4 ++++
>>  libavcodec/vaapi_encode_h264.c | 45 ++++++++++++++++++++++++++++++++++++++++++
>>  3 files changed, 64 insertions(+)
> 
> This puts the AUDs in the wrong place?
> 
> ./ffmpeg_g -y -vaapi_device /dev/dri/renderD128 -hwaccel vaapi -hwaccel_output_format vaapi -i in.mp4 -an -c:v h264_vaapi -aud 1 -b 5M -maxrate 5M out.h264
> 
> IDR frame:
> 
> 00000000  00 00 00 01 67 64 00 33  ac 2e c0 78 02 27 e5 c0  |....gd.3...x.'..|  <- SPS
> 00000010  44 00 00 03 00 04 00 00  03 01 e3 89 80 00 98 96  |D...............|
> 00000020  00 01 31 2c bd ce 01 e1  10 8a 70 00 00 00 01 68  |..1,......p....h|  <- PPS
> 00000030  ee 38 b0 00 00 00 01 06  00 07 80 83 d6 00 00 03  |.8..............|  <- SEI
> 00000040  00 40 01 04 00 00 03 00  02 05 79 59 94 8b 28 11  |. at ........yY..(.|
> 00000050  ec 45 af 96 75 19 d4 1f  ea a9 4d 4c 61 76 63 35  |.E..u.....MLavc5|
> 00000060  37 2e 39 33 2e 31 30 30  20 2f 20 56 41 41 50 49  |7.93.100 / VAAPI|
> 00000070  20 30 2e 34 30 2e 30 20  2f 20 49 6e 74 65 6c 20  | 0.40.0 / Intel |
> 00000080  69 39 36 35 20 64 72 69  76 65 72 20 66 6f 72 20  |i965 driver for |
> 00000090  49 6e 74 65 6c 28 52 29  20 43 68 65 72 72 79 56  |Intel(R) CherryV|
> 000000a0  69 65 77 20 2d 20 31 2e  38 2e 32 2e 70 72 65 31  |iew - 1.8.2.pre1|
> 000000b0  20 28 31 2e 38 2e 31 2d  35 35 2d 67 39 39 36 65  | (1.8.1-55-g996e|
> 000000c0  63 65 64 29 80 00 00 00  01 09 10 00 00 00 01 65  |ced)...........e|  <- AUD, IDR slice
> 000000d0  b8 04 0f fe c5 c0 00 ce  c7 8f f2 87 2b 23 ba dd  |............+#..|
> 000000e0  b5 a3 38 20 c6 f1 d3 00  00 03 00 00 03 00 00 03  |..8 ............|
> 000000f0  00 00 03 00 00 12 8f f3  09 91 c0 00 00 03 00 05  |................|
> 00000100  14 00 00 03 02 26 00 00  03 01 37 00 00 03 00 d4  |.....&....7.....|
> 00000110  00 00 03 00 c9 00 00 03  00 e2 00 00 03 00 fe 00  |................|
> 00000120  00 03 01 32 00 00 03 02  1a 00 00 04 04 00 00 06  |...2............|
> 00000130  28 00 00 0e c0 00 00 03  00 00 03 00 00 03 00 00  |(...............|
> 00000140  03 00 00 03 00 00 03 00  00 03 00 00 03 00 00 03  |................|
> 
> Normal frame:
> 
> 00000250  00 03 00 00 03 00 00 03  00 00 03 00 00 03 00 00  |................|
> 00000260  03 00 00 03 00 00 03 00  01 01 00 00 00 01 06 01  |................|  <- SEI
> 00000270  04 00 00 03 02 06 80 00  00 00 01 09 30 00 00 00  |............0...|  <- AUD
> 00000280  01 21 e0 26 3f 00 00 03  00 00 03 00 00 03 00 00  |.!.&?...........|  <- Non-IDR slice
> 00000290  03 00 00 03 00 00 03 00  00 03 00 00 03 00 00 03  |................|
> 
> (Braswell N3700, i965 driver from git as of an hour ago.)

I have verified in Skylake, i965 driver fix this issue in GEN9 with the commit fd52137
in master branch, but don't fix in Braswell GEN8, open a issue to i965 driver
https://github.com/01org/intel-vaapi-driver/issues/155

> 
>>
>> diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c
>> index 7e9c00f51d..77a10f98a7 100644
>> --- a/libavcodec/vaapi_encode.c
>> +++ b/libavcodec/vaapi_encode.c
>> @@ -236,6 +236,21 @@ static int vaapi_encode_issue(AVCodecContext *avctx,
>>              goto fail;
>>      }
>>  
>> +    if (ctx->va_packed_headers & VA_ENC_PACKED_HEADER_RAW_DATA &&
>> +        ctx->codec->write_aud_header) {
>> +        bit_len = 8 * sizeof(data);
>> +        err = ctx->codec->write_aud_header(avctx, pic, data, &bit_len);
>> +        if (err < 0) {
>> +            av_log(avctx, AV_LOG_ERROR, "Failed to write aud "
>> +                   "header %d: %d.\n", err);
>> +            goto fail;
>> +        }
>> +        err = vaapi_encode_make_packed_header(avctx, pic, VAEncPackedHeaderRawData,
>> +                                              data, bit_len);
>> +        if (err < 0)
>> +            goto fail;
>> +    }
>> +
>>      if (pic->type == PICTURE_TYPE_IDR) {
>>          if (ctx->va_packed_headers & VA_ENC_PACKED_HEADER_SEQUENCE &&
>>              ctx->codec->write_sequence_header) {
>> diff --git a/libavcodec/vaapi_encode.h b/libavcodec/vaapi_encode.h
>> index 0edf27e4cb..09a5d87f7d 100644
>> --- a/libavcodec/vaapi_encode.h
>> +++ b/libavcodec/vaapi_encode.h
>> @@ -267,6 +267,10 @@ typedef struct VAAPIEncodeType {
>>                                   VAAPIEncodePicture *pic,
>>                                   int index, int *type,
>>                                   char *data, size_t *data_len);
>> +     // Write an AU packed header, called by AVC encoder to insert AUD
>> +     int    (*write_aud_header)(AVCodecContext *avctx,
>> +                                VAAPIEncodePicture *pic,
>> +                                char *data, size_t *data_len);
>>  } VAAPIEncodeType;
> 
> You don't need a separate function - this is what write_extra_header is for.
> 
> 
>> diff --git a/libavcodec/vaapi_encode_h264.c b/libavcodec/vaapi_encode_h264.c
>> index 47d0c9496a..7fa9ca70e0 100644
>> --- a/libavcodec/vaapi_encode_h264.c
>> +++ b/libavcodec/vaapi_encode_h264.c
>> @@ -168,6 +168,7 @@ typedef struct VAAPIEncodeH264Options {
>>      int qp;
>>      int quality;
>>      int low_power;
>> +    int aud;
>>  } VAAPIEncodeH264Options;
>>  
>>  
>> @@ -750,6 +751,41 @@ static int vaapi_encode_h264_write_slice_header(AVCodecContext *avctx,
>>                                                          tmp, header_len);
>>  }
>>  
>> +static int vaapi_encode_h264_write_aud_header(AVCodecContext *avctx,
>> +                                              VAAPIEncodePicture *pic,
>> +                                              char *data, size_t *data_len)
>> +{
>> +    VAAPIEncodeContext *ctx = avctx->priv_data;
>> +    PutBitContext pbc;
>> +    char tmp[256];
>> +    size_t header_len;
>> +    int primary_pic_type;
>> +
>> +    init_put_bits(&pbc, tmp, sizeof(tmp));
>> +    vaapi_encode_h264_write_nal_header(&pbc, H264_NAL_AUD, 0);
>> +    switch (pic->type) {
>> +        case PICTURE_TYPE_IDR:
>> +        case PICTURE_TYPE_I:
>> +            primary_pic_type = 0;
>> +            break;
>> +        case PICTURE_TYPE_P:
>> +            primary_pic_type = 1;
>> +            break;
>> +        case PICTURE_TYPE_B:
>> +            primary_pic_type = 2;
>> +            break;
>> +        default:
>> +            av_assert0(0 && "unknown pic type");
>> +            break;
>> +    }
>> +    write_u(&pbc, 3, primary_pic_type, primary_pic_type);
>> +    vaapi_encode_h264_write_trailing_rbsp(&pbc);
>> +    header_len = put_bits_count(&pbc);
>> +    flush_put_bits(&pbc);
>> +    return ff_vaapi_encode_h26x_nal_unit_to_byte_stream(data, data_len,
>> +                                                        tmp, header_len);
>> +}
> 
> Mild preference for this being derived in init_picture_params rather than inside the writing function?

I can change it with this way, but I don't think this way have more benefit. BTW, we also need to 
support insert AUD in HEVC encoder.

> 
>> +
>>  static int vaapi_encode_h264_write_extra_header(AVCodecContext *avctx,
>>                                                  VAAPIEncodePicture *pic,
>>                                                  int index, int *type,
>> @@ -1180,6 +1216,8 @@ static const VAAPIEncodeType vaapi_encode_type_h264 = {
>>      .write_slice_header    = &vaapi_encode_h264_write_slice_header,
>>  
>>      .write_extra_header    = &vaapi_encode_h264_write_extra_header,
>> +
>> +    .write_aud_header      = &vaapi_encode_h264_write_aud_header,
>>  };
>>  
>>  static av_cold int vaapi_encode_h264_init(AVCodecContext *avctx)
>> @@ -1265,6 +1303,11 @@ static av_cold int vaapi_encode_h264_init(AVCodecContext *avctx)
>>          VA_ENC_PACKED_HEADER_SLICE    | // Slice headers.
>>          VA_ENC_PACKED_HEADER_MISC;      // SEI.
>>  
>> +    if (opt->aud == 1) {
>> +        ctx->va_packed_headers |=
>> +            VA_ENC_PACKED_HEADER_RAW_DATA;
>> +    }
>> +
>>      ctx->surface_width  = FFALIGN(avctx->width,  16);
>>      ctx->surface_height = FFALIGN(avctx->height, 16);
>>  
>> @@ -1282,6 +1325,8 @@ static const AVOption vaapi_encode_h264_options[] = {
>>      { "low_power", "Use low-power encoding mode (experimental: only supported "
>>        "on some platforms, does not support all features)",
>>        OFFSET(low_power), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 1, FLAGS },
>> +    { "aud", "Use access unit delimiters",
>> +      OFFSET(aud), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 1, FLAGS },
>>      { NULL },
>>  };
>>  
>> -- 
>> 2.11.0
>>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 


More information about the ffmpeg-devel mailing list