I am trying to use source reader initialized with an H.264 MP4 file and Direct3D 11 device. Windows 10 system, classic desktop application.
The background of the process is given in Supporting Direct3D 11 Video Decoding in Media Foundation with the exception that I am not building a topology and I don't have a session. The case is reduced to just source reader. And even though the article describes the process, I am not implementing decoder here, I am trying to utilize this decoder instead assuming that stock H.264 decoder is working along the given lines.
The DXGI Device Manager is used to share the Direct3D 11 between components. The DXGI Device Manager exposes the
IMFDXGIDeviceManager
interface. The pipeline sets theIMFDXGIDeviceManager
pointer on the software decoder by sending theMFT_MESSAGE_SET_D3D_MANAGER
message.
I already have Direct3D 11 device, I created IMFDXGIDeviceManager
and initialized with this device. UsingMF_SOURCE_READER_D3D_MANAGER
the manager is passed to source reader withMFCreateSourceReaderFromURL
.
Things are going well up to getting a sample from the source reader. I have
NV12
media sample and I see that it's coming from internally used MFT, with attributes that I see (among other):
MF_SA_D3D11_AWARE
, vValue 1 (TypeVT_UI4
)CODECAPI_AVDecVideoAcceleration_H264
, vValue 1 (TypeVT_UI4
)
They are not confirming DXVA use, as I suppose - these only confirm that I requested hardware assisted decoding. Getting a media sample I actually don't have any confirmation as for whether decoding is hardware accelerated or not.
IMFSample
has one buffer, which is IMFMediaBuffer
implemented by MF'sCMF2DMediaBuffer
. It does have IMFGetService
but it does not expose any services, specifically it does not exposeMR_BUFFER_SERVICE
which presumably makes it unable to reach underlying surface/texture (if it at all exists).
As far as I can see I either failed to initialize Direct3D enabled H.264 decompression, or decoder is unable to initialize the DXVA decoding context and fell back to software decoding, or it is doing all right and I am just unable to access the underlying Direct3D 11 Texture2D.
- Is there any way to query H.264 decoder if it is actually using hardware Direct3D 11 decoder?
- Any way to troubleshoot MFT configuration within source reader to ensure it picked up Direct3D device?
- How am I supposed to resolve IMFSample to Direct3D resource, is it still buffer's
IMFGetService::GetService(MR_BUFFER_SERVICE, __uuidof(ID3D11Texture2D)
call?