I wrote a simple test program which reads an h.264 video using a source reader as BGRA textures, copies them to another texture, and writes that texture (after wrapping it with MFCreateDXGISurfaceBuffer) to a sink writer.
If I wait between each frame (Sleep(10)) or use a separate texture for each frame, the output video is a good copy of the source. If I don't, it looks like a lot of frames get dropped, as if the source is updating the texture several times before the data is finally written.
I tried various buffer size, reading frames into a circular buffer of textures and writing them. On the GeForce 980 Ti I ran this on, a buffer of size 30 worked well, a buffer of size 12 worked terribly, and 25 worked terribly while 15 had one jump then was smooth.
Edit: I noticed that I was creating the sink writer without hardware acceleration (MF_READWRITE_ENABLE_HARDWARE_TRANSFORMS and MF_SINK_WRITER_D3D_MANAGER attributes were commented out). That doesn't change the question (especially since this condition is valid), but I will try to see how this behaves with hardware encoding (I assume better because it will take less time to encode).
The question is, how do I get this under control? Is there any way to know that a frame has been written and I can reuse the texture? The IMFSinkWriter::WriteSample docs say "By default, the sink writer limits the rate of incoming data by blocking the
calling thread inside the WriteSample method. This prevents the application from delivering samples too quickly." However, this doesn't seem to be the case.
(I'm running on Windows 10, but got a similar issue on 8.1, which is why I wrote the test program.)