Quantcast
Channel: Media Foundation Development for Windows Desktop forum
Viewing all articles
Browse latest Browse all 1079

Need technique for removing frames from an MP4 video

$
0
0

I am currently prototyping the ability to remove sections from a video and saving the results using Media Foundation in a C++ program. There is a fundamental flaw in my approach that I'm hoping somebody will have a suggestion for a work-around, or at least an alternative idea that could be used instead (but it must be MF-based, DirectShow is not an option).

The initial approach chosen for the encoding involved the use of the "Source Reader plus Sink Writer" algorithm as briefly outlined in this article: http://msdn.microsoft.com/en-us/library/windows/desktop/ff485861(v=vs.85).aspx

Since there is no transcoding involved, the ReadSample+WriteSample loop is very fast since I can just use the source native MediaType in the sink writer. My initial thought was to just drop the samples[frames] that are within the interval to be cut out of the video. This turns out to be a bad idea because it doesn't take into consideration the compressed format of the video/audio streams. That is, that there are I-frames (a.k.a., key frames) and P-frames (partial frames) and the P-frames are highly dependent on the existence of the preceding I-frame.

In this example, choosing to discard arbitrarily (user) chosen frames, would result in incorrectly rendered frames following the discarded section. It would correct itself upon reaching the next key frame.

[ I ][ P ][ P ][ P ]  [ P ][ P ][ I ][ P ][ P ]  [ P ][ P ][ P ][ I ][ P ][ P ]
                         ^                               ^
                          x    <-   discard   ->   x

My hope was to try to either (a) force the next sample/frame after the cut section to be a key frame, or (b) manually insert a sample as a key frame into the output sequence that represents what the frame would have looked like at the end of the cut.

Regarding (a), this article suggests a way to do that with the MFSampleExtension_CleanPoint attribute, but setting that to TRUE with a sample that is a P-frame does not work (not surprising, I suppose).

And (b) could work, but the trick is to get the desired image and then encode it appropriately for the sink writer. Can't seem to get that to work (including leveraging an H.264 encoder).


So, the ReadSample+WriteSample loop doesn't seem promising. If so, what other approaches, using Media Foundation, would work for removing portions of a MP4 video?

Media Foundation is kind enough to provide these two nifty attributes for trimming: MF_TOPONODE_MEDIASTART and MF_TOPONODE_MEDIASTOP. But there are no equivalents for cutting sections out of the middle.

-----

This code snippet shows the relevant details of the process (omitting most comments, all error checking, and the sink writer creation and stream assignments wrapped in the custom CMFMP4SinkWriter class):

bool Transcode(
   const std::wstring& infile,
   const std::wstring& outfile,
   const std::vector<Interval>& segments )
{
  // 1. Create the IMFSourceReader from the input file

  ATL::CComPtr<IMFSourceReader> mediaSourceReader;
  MFCreateSourceReaderFromURL( infile.c_str(), NULL, &mediaSourceReader )

  // 2. Create the IMFSinkWriter from the output file

  CMFMP4SinkWriter sinkWriter( nullptr );
  sinkWriter.InitializeSinkWriter( outfile.c_str(), false, enableHwTransforms );
  int streamCount = 0;
  sinkWriter.AddMediaToSinkWriter( mediaSourceReader, streamCount );

  // 3. Loop through all the samples

  ProcessSamples( mediaSourceReader, sinkWriter, streamCount, segments );
}


void ProcessSamples( ... )
{
   // Read all samples from the source reader and pass them to the sink writer.

   sinkWriter.StartWriting();

   // Loop as long as there are unfinished (unread) streams (audio and video)
   //
   int nFinishedStreams = 0;

   while ( nFinishedStreams < streamCount )
   {
      ATL::CComPtr<IMFSample> sample;

      mediaSourceReader->ReadSample(
         (DWORD)MF_SOURCE_READER_ANY_STREAM,
         0, &streamIndex, &flags, &timestamp, &sample );

      if ( sample != NULL )
      {
         if ( IsUnCutSample( segments, timestamp, sample ) )
         {
            // Push the sample to the sink writer
            sinkWriter.WriteSample( streamIndex, sample );
         }
      }
      else if ( flags & MF_SOURCE_READERF_STREAMTICK )
      {
         sinkWriter.SendStreamTick( streamIndex, timestamp );
      }

      if ( flags & MF_SOURCE_READERF_ENDOFSTREAM )
      {
         sinkWriter.NotifyEndOfSegment( streamIndex );

         nFinishedStreams++;
      }
   }

   sinkWriter.FinalizeWrite();
}


Viewing all articles
Browse latest Browse all 1079

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>