Building a better screenshotter

High definition snow

My previous forays into crafting an automatic screenshot taker were, at the time, very successful. The system managed to pump out usable images in a fraction of the time it would have taken me to seek and do them manually; I even extended the script to handle multiple file-inputs which made 'capping an entire series a breeze. Lamentably, this was a honeymoon period before cracks started to show, followed by gaping chasms.

The only workaround the first screenshotter used was a glitch for Windows Media files which meant the first frame sought was always blank, it swerved around this limitation by taking two shots and discarding the first. This symptom, however, was indicative of what would become a persistent problem.

Background

The first significant problem I encountered with the setup was with the series Claymore, a great many of the resulting images seemed to have a lot of "bleed through", as if one frame were being intermingled with another, this was above an beyond the standard cross-fade transition screenshots that were common. At the time I assumed it was because the files I used were modern H264 MKV files rather than the standard XViD ones I had been using before, or that the encoding was particularly shoddy. After downloading an updated version of mplayer for Windows the problem seemed to disappear; I ended up regenerating a lot of the images for episodes which were most severe offenders.

The thing to understand is that seeking in a video files is very difficult

After a spate of swift updates, I didn't blog anime any more so the screenshotter shortcut on my desktop lay dormant until I decided to unleash some madness on Strawberry Panic. While the setup worked, it was producing an unusual amount of exact duplicate images, despite being over five seconds apart. I realised there was a fundamental underlying cause for this that an mplayer update wouldn't fix. True high-definition versions (not upscales) of certain releases were now readily available, namely the seminal Ghost in the Shell: Standalone Complex (and associated movie Solid State Society) and a selection of Makoto Shinkai works including 5 centimeters per second, which I wanted to pluck some quality captures from (for desktop wallpaper or other purposes). These files did not agree with the screenshotter at all and stoically produced correct resolution but entirely black captures which was less than useful.

High definition solitude

The cause

While not entirely sure of the cause, my hunch lay with the way newer files operate. Modern video compression usually comes in three frame types, I-Frame (usually termed keyframes), P-Frame which are predictive frames (sometimes thought of as "only the bits that change") and B-Frames which which are also predictive frames but don't just rely on the previous decoded frames but the forward frames as well. H264, the most modern MPEG standard also gives three other frame types: SI, SP and multi-frame motion frames, all of which are outside the scope as to why they're different. The thing to understand is that seeking in a video files is very difficult as not only does the decoder need to potentially decode all the way back to an I-frame, but also forward to get all the data needed which can be time consuming and intensive.

The older screenshotter worked by repeatedly spawning mplayer with command line arguments stating it should jump to a specific point, play only 2 frames which it outputs as images and then exit. The scripting part (handled by PHP) was doing all the calculations as to where to seek to rather than expecting mplayer to do this (the reason for this will be explained below). The problem of duplicate screenshots was likely being caused by forcing mplayer to seek to an arbitrary point where it would display the first frames it could reliably show, and in some highly compressed files, this could be the same across one or more screengrab attempts. The blank frame problem I postulated was probably a computational bottleneck, my computer can't play most 1080p files in real time and can baulk at some 720p ones, my theory is that mplayer was doing the best job it could without desynching and displayed a blank frame so it could "catch up" as the file progressed, which it couldn't do due to only forcing two frames.

In short, because mplayer was being continually restarted, the screenshots lacked the "context" of the rest of the file and thus were substandard or just plain missing.

High definition consumerism

Version 2

The original screenshotter layered PHP on top of mplayer because of a number of perceived deficits in mplayer's operation. mplayer has always had a "framestep" option (which in the most recent versions has been bundled into the video filter architecture) which seemed like exactly what was needed, unfortunately it only forced the rendering of those particular frames rather than doing what I had hoped and only playing those frames; using this option means taking screenshots would take as long as the file itself.

My next thought was to use something a bit more specialised for scripting, AviSynth sprung immediately to mind. As well as a cornucopia of other options, it had a function which seemed to be what I was looking for: SelectEvery. My hope that it would provide a simple solution to this problem was dashed as it did exactly the same as the mplayer framestep option, rendering only the selected frames rather than compressing.

Version 2 is more of a compromise than a solution. Essentially mplayer needed to play through the file in its entirety to be able to extract good screenshots but needed to be quick enough to be usable. The result is using previously mentioned options of mplayer (frameskip, JPEG output) but forcing the frame rate to be fast so that mplayer can tear through the file as fast as it can. Thankfully by default mplayer doesn't do any frame skipping and it requires you to stipulate it skip frames (aggressively or otherwise). The final command line I used for my shortcut was:

mplayer.exe -quiet -nosound -vo jpeg:progressive:quality=85:outdir=screenshots -fps 1000 -vf framestep:i120

I would have used -really-quiet except my version of mplayer didn't seem to like that option. Dropping this into a shortcut on my desktop, I'm able to drag a file onto the shortcut and for everything to just whir away in the background. The lower case "i" in front of the framestep intervals tells mplayer to spit out an "I!" everytime it captures a frame which you can disable if you want the minimum of fuss. The framestep value is in frames which means the time between files is dependant on the frame of the video you're capturing, however I usually do screenshots in depth and then prune them afterwards, you can adjust this to suit your style.

High definition grey day

Conclusions and ways forward

In terms of results, the images dotted about this article say all that they need to really. The number of duplicates is none existent and there is no apparent visual distortion or corruption or errant blanks even with high definition files.

As far as benchmarks are concerned, I haven't yet done any tests to see whether this way is faster or more complete than my previous version. Setting the frame rate so high may be detrimental rather than a beneficial as mplayer may be pushing harder to get through the file than it should be, FPS could be optimal at a much lower value (perhaps a multiple of the actual framerate).

This method also requires a lot more maintenance on the part of the user as the screenshots are unceremoniously dumped into a single directory and are subject to being overwritten if doing a sequence of files. There may yet be a place for scripting in this version, although I'm loathe to do so as the purity of keeping the method entirely within a shortcut is not to be underestimated.

Respond to “Building a better screenshotter”

Community rules:

  1. Keep it civil: no personal attacks, slurs, harassment, hate speech, or threats
  2. No spam: includes marketing, pyramid schemes, scams etc.
  3. Notify of any spoilers: even if it's for something the post isn't about
  4. Your response may be edited or removed: if your response was in good faith, you may be contacted via email explaining why

Your address will never be shared

The following HTML tags are allowed: <b> <strong> <i> <em> <a href>