Action video is often lacking clear persistent references to stabilize video with – they either are not present in the frame for very long, or often compressed out of frame as the codec is trying to spend most of its bandwidth on moving objects in the shot.
Deshaker, a plugin for Windows program VirtualDub, with its capability to find and track multiple visual references (literally hundreds of them) automatically from frame to frame is the king of action video stabilization. (It works on Linux, too, see below).
Deshaker works in 2 passes – analysis and stabilization. During pass 1, a log file is created that contains correction numbers necessary for each frame.
Deshaker is a video stabilizer for. Create your account / Login. You must be logged in to submit changes. Jun 22, 2014 Neither can add Deshaker as a 'new' plugin which might have allowed you to use all the settings as it won't be recognised. However you can use the stabilisation program (Deshaker) with your own parameters if you use it via the Virtual Dub program.
This file could be used in Blender as a source for transform track. Here is how:
Creating Deshaker Log
- Export video from Blender, audio is not necessary. It’s OK to use 50% render resolution (IMPORTANT: make sure to use the same resolution during Deshaker log import).
- Run VirtualDub.exe with Deshaker plugin installed in VirtualDub’s “plugins” folder
- Enable Deshaker with Video > Filters > Add > Deshaker
- Set or note location for log file, click OK – OK
- Open Blender video made in step 1
- I like to right-click and reduce input (left) video window to 25% at this point
- Make sure Video > Full Processing Mode is set
- Run analysis by invoking File > Save as AVI and setting some dummy file name. This file is discardable.
- Use Deshaker.log file for next step:
Using Deshaker Log in Blender
- Install one of 2 plugins from https://github.com/sergem155/blender-deshaker-log-import . There are 2 versions, using different stabilization algorithms (more below, but “damper” version is working well, “sinc” version is experimental).
- Import the plugin into Blender: File > User Preferences > Add-ons > Install from File
- Right-click the video (movie) track to stabilize. (It might make sense to make a metastrip out of the composition first.)
- IMPORTANT: make sure to use the same render resolution % used to render file for Deshaker analysis, as the log numbers are divided by that %!
- Import Deshaker log: File > Import > Import Deshaker log
- To speed things up a little, in Graph Editor, set interpolation type: Key > Interpolation Type > Linear
- Now you can scrub through video and see the Deshaker tracking data applied
Example output
Simple damper algorithm – the distance from center is reduced by 10% in each frame:
Music is Skeewiff – Blue Crystal (feat Rayna)
The black bars are kind of large, I’ve done a couple of attempts to reduce them, using a digital filter to remove low frequency motion from stabilization movement and remove “DC bias” building up. I still like simple damper better, but maybe I wasn’t playing with settings long enough, or windowed sinc’s ripples were acting up, and another algo would have been better.
Windowed sinc filter (17 points) – remove all motion < 1Hz, but with less steep roll-off:
Raw video without stabilization, for comparison:
Other Cool Features of Deshaker
I implemented just motion. Deshaker can also do adaptive zoom, rolling shutter rubber panning effect correction and can paint black bars with contents of previous frames, to name a few. If you’d like to code that, be my guest, the code is open.
What About Manual Correction?
It would have been awesome to put a second transform track and correct the movement a bit, but sadly, Blender’s transforms are fully applied and cropped between strips, so you get a crop upon crop:
I also tried to put the sequence as a source strip in another scene’s sequence, with a smaller viewport. While this is possible, the strip is scaled to the second composition’s size, even with image offset option turned on, so it is, unfortunately, useless too:
Running VirtualDub on Linux
It works very nice:
- Install Wine Windows Emulator
- Install MP4 Windows codec https://sourceforge.net/projects/x264vfw/files/latest/download
- Enable vcrun2005 in winetricks (Wine utility)
- Download and unpack VirtualDub
- Download Deshaker and put Deshaker.vdf into virtualdub’s plugin folder
- Run virtualdub.exe
Update
There was a bug (sorry) in the code that was shifting all transforms 2 frames back, that is now fixed in github. (And once again, make sure to import the log with the same render resolution % you used to export it).
(Please note that, except for Deshaker, these are quite old and have not been updated for many years.)
This page contains the latest versions of some filters I made for VirtualDub. Put them into the VirtualDub plugins folder and restart VirtualDub to make them available from VirtualDub.
Deinterlace - smooth (go here for Xesdeeni's great AVISynth port of this deinterlacer)
Deinterlace - PAL movie
(Please note that this filter, including its description, is quite old. I believe VirtualDub's newer FieldDelay filter does the same thing as this filter.)
In PAL, two kinds of video exist:
50fps video with half vertical resolution (i.e. interlaced video) and
25fps video with full vertical resolution (i.e. progressive video)
Both of these are usually captured as 25fps full resolution video which means that for 50fps video, each frame will consist of two subframes, or 'fields'. When these fields are merged into the full frame, interlace lines appear where the fields differ (usually due to motion). For this kind of video you should NOT use this deinterlacer. You need to use a 'real' deinterlacer such as the area based deinterlacer below. You can also split the frames into their subframes and double the frame rate in which case you won't need to do any deinterlacing at all. But I recommend that you use my field shift filter below if you do this.
If, on the other hand, you have captured video with real 25fps (such as most PAL movies), there should be no interlace lines, since the fields belong to the same frame. However, sometimes the capture card gets it wrong and merges fields that should belong to different frames. This can easily be corrected by delaying all frames by one field which is exactly what the PAL movie deinterlacer does. This reconstruction is completely non-destructive. You should, however, always make sure your codec compresses the fields individually if you have this problem. Because interlace lines don't compress very well. MJPEG codecs usually have this option.
This filter needs to be first in the filter chain and before any clipping.
Settings
Swap fields: Sometimes the fields of a frame are in wrong order. Check this to fix that.
Version history:
Version 1.1: Added script support.
Note: To avoid confusion I think I should tell you that since I made this filter Donald Graft has added PAL deinterlacing to his smart deinterlacer too. It's called 'phase shift by one field' and does everything my PAL deinterlacer does and a little more, so you might as well use that one instead since it's included with VirtualDub. But I'll keep this filter here anyway.
Download PAL movie deinterlacer
(Source code is available here)
Deinterlace - area based
(Please note that this filter, including its description, is quite old. I suppose there are better deinterlacers out there today.)
Deshaker Image Stabilization
This filter deinterlaces the frames only where it's needed by simply looking for interlace lines in each frame individually. When it detects interlace lines, it removes them using either blending or interpolation depending on your settings.
This is a very general deinterlacer since it doesn't care about how the interlace lines appeared. It could be due to motion or due to compression artifacts from 2-field compressed MPEG video (e.g. from digital TV). It's also perfect for when only parts of the frame area (or only some of the frames) suffer from interlace lines, such as a PAL movie with subtitles that changes between the two fields in a frame.
Note: Personally I hardly use this filter anymore. The smooth deinterlacer below is so much better, but only if you can deal with doubled framerate.
Settings
Show deinterlaced areas only: Areas that aren't deinterlaced are greyed (with a slight transparency).
Blend: Simply blends the interlace lines.
Interpolate: Discards every other interlace line and replaces them with the average of the rows above and below.
Threshold: Controls how much to deinterlace. Lower values deinterlaces more.
Edge detect: It's difficult to distinguish between interlace lines and real edges (which should not be deinterlaced). This value controls this decision. Higher value leaves more edges intact.
Log to file: The frames that have an interlaced area exceeding the given number of percent can be logged to a file which will be placed in the current directory (probably the same directory as the output video when encoding). It's called DeinterlaceAreaBased.log and contains frame numbers and the interlaced area (in %) for each frame. Can be used to detect frames that IVTC couldn't deinterlace properly.
Version history:
Version 1.4: Minor fix. Version 1.3 couldn't read the same processing settings and avisynth scripts as version 1.2. This version is backwards compatible. Thanks to Eric Borremans for bug report and testing.
Version 1.3: Added logging (suggested by Johan Rior).
Version 1.2: Added script support.
Version 1.1: Improved interlace detection algorithm. It doesn't do as much erroneous deinterlacing in detailed areas.
Download the area based deinterlacer
(Source code is available here)
Deshaker - Video Stabilizer
Field shift
(Please note that this filter, including its description, is quite old. Also, it's a pretty useless filter...)
If you use the VirtualDub routines for MJPEG decoding and split each frame into two subframes and double the frame rate, you may notice that the output jumps up and down slightly. To make this disappear, every other frame should be shifted half a line. The easiest way to do this is to double the lines and shift every other frame down one line, which is exactly what this little filter does. You will probably want to reduce the size after this operation.
NOTE: You'll probably want to use the new smooth deinterlacer below instead of this one, since it maintains the frame's full resolution in static video.
Settings
Shift even frames: Choose between shifting even or odd frames.
Download the field shifter
(Source code is available here)
Deinterlace - smooth
(Please note that this filter, including its description, is quite old. I suppose there are better deinterlacers out there today.)
This filter does almost the same as the area based deinterlacer above, except that this one delivers video in twice the frame rate, which results in smoother video. No temporal information is lost with this filter. In non-interlaced areas, two fields are combined to give a full resolution frame. In interlaced areas, the current field is interpolated up to full frame size (or it can blend with the previous field, which isn't a recommended option).
Like for the field shift filter above, the input video to this filter must consist of individual fields at 50 or 60fps. If the source is in MJPEG format, VirtualDub can split the frames for you. Otherwise, use AVISynth to split them with the following script:
AVISource('clip.avi')
ComplementParity
SeparateFields
...and open this script from VirtualDub. Remove the 'ComplementParity' line if the fields arrive in wrong order.
Settings
Deshaker Tutorial
Colorcode: Areas are colored differently to help you find suitable parameter values.
Red - Deinterlaced areas
Blue - Non-static areas that would be deinterlaced if interlace patterns were found
Green - Static areas that do contain interlace patterns but are still left untouched
Grey - Static areas without interlace patterns
Blend: Blends this and previous field in interlaced areas. (You should probably avoid using this as it just blurs the video.)
Interpolate: Interpolate the field to full frame size in interlaced areas.
Alternate field order: Put even or odd fields on first line of output video. The correct setting depends on your video clip. If static video jumps up and down slightly when you step through it, change this setting.
Interlace threshold: Controls the detection of interlace patterns. Lower values deinterlaces more.
Edge detect: It's difficult to distinguish between interlace lines and real edges (which should not be deinterlaced). This value controls this decision. Higher value leaves more edges intact.
Static threshold: The filter tries to detect static areas to avoid deinterlacing fine details which could result in flickering. This value controls how much a pixel can variate and still be called static. Use as low value as possible to avoid leaving interlace patterns. Values above 50 (or so) are not recommended. Good quality video can use lower values. If you don't have any text or logos that may flicker I suggest using very low values. 0 makes it work like version 1.0.
Static averaging: Controls how long history to look at when determining if areas are static or not. Low values (= short history) find static fast (but maybe incorrectly, leaving interlace patterns). High values means static details may flicker for a longer time before 'converging'. Also, it can be slower to react when areas go from static to non-static. Valid range is 0-100. A good rule is to set <static averaging> = 2 * <static threshold>, or higher (but don't get too close to 100).
Log to file: The frames that have an interlaced area exceeding the given number of percent can be logged to a file which will be placed in the current directory. It's called DeinterlaceSmooth.log and contains frame numbers and the interlaced area (in %) for each frame.
Version history:
Version 1.1: Often, fine detailed static areas such as text would get deinterlaced in version 1.0 resulting in flickering. It now tries to detect static areas by looking at several old frames/fields and leave those areas alone. Thanks to Janik for ideas and testing. (He has a great deinterlacing page at www.100fps.com btw).
Deshaker Software
Download the smooth deinterlacer
(Source code is available here)
General convolution 3D
This filter is a general 5x5 convolution filter with an extra dimension - time. It applies one matrix to the current frame, another matrix to the previous and yet another one to the frame prior to that. This filter was suggested by Neil Lee.
Settings
Use output instead of input for previous frame: Determines which previous frame to use. If checked, it uses the previous frame as it looked AFTER this filter processed it. Otherwise, it uses the unfiltered previous frame.
Version history:
Version 1.1: Went from matrix size 3x3 to 5x5 and from two to three matrixes (suggested by Roger Kamben). It's also faster when you don't use all the matrixes or matrixes smaller than 5x5.
Download General convolution 3D
(Source code is available here)
Deshaker
Deshaker is a video stabilizer. It has its own page here.