[FFmpeg-devel] [PATCH] avcodec/amfenc: DX12 Reference-only feature support

[Araz]

>
> Benjamin Cheng Jan. 31, 2025, 1:22 a.m. UTC

This patch only affects d3d12va, why is the commit message amfenc?
>

We accept this review comment and update it in this patch:
avcodec/d3d12va_decode: enable reference-only decoder mode


> You could optimize these barriers since reference-only resources don't
> need to be transitioned to COMMON, and can remain in
> VIDEO_DECODE_{READ,WRITE}.
>
> I propose the following:
> - Transition all reference texture to VIDEO_DECODE_READ at creation time.
> - When preparing resources for input to DecodeFrame(), transition only
> the texture for reference output to VIDEO_DECODE_WRITE
> - After DecodeFrame(), transition the reference output texture to
> VIDEO_DECODE_READ. This is already implicitly handled by the barrier SWAP.
>
> All-in-all, for the cost of an initial transition at creation time, you
> decrease the number of barriers in each frame to just 2.


We have conducted some evaluation work
regarding the suggestions from the review:

1. The proposed method can successfully complete decoding,
and the playback quality of the decoded YUV raw data is normal.
However, during the execution, a large number of D3D12 errors
were encountered, and error sample is shown below.

D3D12 ERROR: ID3D12CommandQueue::ExecuteCommandLists:
Using DecodeFrame on Command List (0x0000018A94557FD0:
'Unnamed ID3D12VideoDecodeCommandList Object'):
Resource state(0x 10000: D3D12_RESOURCE_STATE_VIDEO_DECODE_READ) of
resource(0x0000018A95B1C740:'Unnamed ID3D12Resource Object')
(subresource : 1) must be in COMMON state when transitioning
to use in a different Command List type, because resource state on
previous Command List type : D3D12_COMMAND_LIST_TYPE_DIRECT,
is actually incompatible and different from that on the
next Command List type : D3D12_COMMAND_LIST_TYPE_VIDEO_DECODE.
[RESOURCE_MANIPULATION ERROR #990:
RESOURCE_BARRIER_MISMATCHING_COMMAND_LIST_TYPE]

Further analysis revealed that...
According to the suggestion, when decoding each frame,
only the current reference texture should be transitioned
to VIDEO_DECODE_WRITE, and after DecodeFrame(), the current reference
texture should be transitioned to VIDEO_DECODE_READ (this is already
implicitly handled by the barrier FFSWAP).
For all other reference textures, they should remain in the READ state.
However, the evaluation shows that there is a problem when keeping
the all other reference textures in the READ state.
It expects a transition from COMMON to READ every time,
even if the texture is already in the READ state.
If the other reference textures does not transition to COMMON,
they will throw the D3D12 error mentioned above.
In other words, transitioning to the COMMON state is mandatory.

2. Performance testing.
We evaluated the performance of the currently submitted patent
and the performance of the suggested method from the review.
After conducting few dozen tests, minor changes were identified
that can be attributed to error,
since only 10 out of 18 tests showed a speed improvement of 0.2%.
The remaining 8 showed that the current method is faster
than the proposed one.

Tong Wu Feb. 10, 2025, 3:18 p.m. UTC
> Need to handle when DecodeTier == D3D12_VIDEO_DECODE_TIER_2?


The submitted patch has been tested on GPUs
supporting D3D12_VIDEO_DECODE_TIER_2
(GPU such as AMD RTX 7900XTX), and it works perfectly.
Therefore, no additional work is needed for D3D12_VIDEO_DECODE_TIER_2.

Lynne Feb. 10, 2025, 4:34 p.m. UTC
> This information doesn't belong here, and this value does definitely not
> belong here, as the hwcontext_d3d12.c code doesn't even touch it.
> We try to not put random internal fields in public structs anymore.


We accept this review comment and update it in this patch:
avcodec/d3d12va_decode: enable reference-only decoder mode