SetInputType fails on Hardware MFT

I have code that activates the Intel M-JPEG Decoder MFT.  Calling SetInputType on the MFT return E_FAIL.  Is there another attribute that must be set on the MFT?  This code works on synchronous MFT from Microsoft.  For clarity, I've removed looking at the hr return, and dealing with errors.

MFTEnumEx(MFT_CATEGORY_VIDEO_DECODER, MFT_ENUM_FLAG_HARDWARE, &inputTypeGUID, &outputTypeGUID, &availableMFTs, &numMFTsAvailable);

availableMFTs[0]->ActivateObject(IID_PPV_ARGS(&m_mjpegMFT));

UnlockAsyncMFT( m_mjpegMFT );  // Comment this out for MFT_ENUM_FLAG_SYNCFMT

mfVideoFormat.dwSize = sizeof(mfVideoFormat);

mfVideoFormat.guidFormat = MFVideoFormat_MJPG;

mfVideoFormat.videoInfo.dwWidth = m_imageWidth;

mfVideoFormat.videoInfo.dwHeight = m_imageHeight;

MFCreateVideoMediaType(&mfVideoFormat, &pVideoMediaTypeInput);

// This next call fails,  hr = E_FAIL

hr = m_mjpegMFT->SetInputType(0, pVideoMediaTypeInput, 0);   // FAILS

======================

I've queried the MFT for it's attributes, and it is indeed a video decoder.  I've added this code after activating the MFT ...

IMFMediaType* pMediaType;