[FFmpeg-trac] #11013(avcodec:new): thread assertion if avcodec_get_hw_frames_parameters fails

[FFmpeg]

#11013: thread assertion if avcodec_get_hw_frames_parameters fails
-------------------------------------+-------------------------------------
             Reporter:  Francois     |                     Type:  defect
  Cartegnie                          |
               Status:  new          |                 Priority:  normal
            Component:  avcodec      |                  Version:
                                     |  unspecified
             Keywords:               |               Blocked By:
             Blocking:               |  Reproduced by developer:  0
Analyzed by developer:  0            |
-------------------------------------+-------------------------------------
 Summary of the bug:

 Using libavcodec's leads to assertion due to missing private_date cleanup
 on failed VAAPI VP9 acceleration.

 release/7.0 e7d2238ad751e4ce4ebc5118af750fd5cc0c5055


 {{{
 libva info: VA-API version 1.20.0
 libva info: Trying to open /usr/lib64/dri/iHD_drv_video.so
 libva info: Found init function __vaDriverInit_1_20
 libva info: va_openDriver() returns 0
 [vp9 @ 0x619000022180] No support for codec vp9 profile 0.
 [0000604000066ad0] vaapi generic error: avcodec_get_hw_frames_parameters
 failed: -38
 Assertion p_dst->hwaccel_threadsafe || (!dst->hwaccel &&
 !dst->internal->hwaccel_priv_data) failed at
 libavcodec/pthread_frame.c:348

 }}}

 Fix attempt

 {{{

 diff --git a/libavcodec/pthread_frame.c b/libavcodec/pthread_frame.c
 index fd356bd190..a13bb9e996 100644
 --- a/libavcodec/pthread_frame.c
 +++ b/libavcodec/pthread_frame.c
 @@ -344,6 +344,10 @@ FF_ENABLE_DEPRECATION_WARNINGS
                  return err;
          }

 +       // hwaccel could have failed init due to some params
 +       if(!dst->hwaccel && dst->internal->hwaccel_priv_data)
 +           ff_hwaccel_uninit(dst);
 +
          // reset dst hwaccel state if needed
          av_assert0(p_dst->hwaccel_threadsafe ||
                     (!dst->hwaccel && !dst->internal->hwaccel_priv_data));
 }}}

 How to reproduce:

 Sample reference
 {{{

 4d8e399e9ecf06363840f3a7880e0e2937d527438bcf87bb3c250e5f4ac71c7e Agent 327
 - Operation Barbershop.web

 }}}
-- 
Ticket URL: <https://trac.ffmpeg.org/ticket/11013>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker