Scintilla's Guide to AVISynth Postprocessing Filters: Spatial Smoothers

BlindPP - DCTFilter - Deen - eDeen - MSmooth - RemoveGrain
SmoothD - SpaceDust - TBilateral - UnDot - UnFilter - VagueDenoiser

( Home - Temporal Smoothers - Spatio-Temporal Smoothers - Sharpeners - Dealing with Dot Crawl - Dealing with Rainbows )


Spatial smoothers generally seek to increase compressibility (by "compressibility", we mean a measure of how small we can make a video's filesize without losing quality) and/or visual quality by smoothing pixels based on other pixels in the same frame, hopefully removing noise while keeping the picture intact. The effects of spatial cleaners are usually much more visually obvious than those of temporal cleaners, and as such this page will probably have more screenshots than any of the others. All images are high-quality .PNG files (unfortunately I don't yet know how to ensure that PNGOUT will save a lossless .PNG); I apologize in advance to those of you on dialup.

Filters listed as deblockers aim specifically to reduce blocky noise such as can sometimes result from insufficient bitrates in MPEG encodes. RemoveGrain was ostensibly developed for removing film grain but can also be used for more general smoothing. BlindPP is a deblocker and deringer all in one. DCTFilter is a special case.

All screenshots on this page can be hovered over to show the differences between the original frame and the filtered version.

Note: Opera is exhibiting this weird bug where if I open a link in the same tab and then go back to this page, the onMouseOver functionality no longer works. This doesn't seem to happen in Firefox and IE, and it also isn't happening on other pages I know that use the exact same code. If you experience this problem, simply reload the page.

The deblocker/deringer contained in the MPEG2Dec3/DGDecode package.

Type: Plugin
Found in file: MPEG2Dec3DG.dll, DGDecode.dll
Author: MarcFD? (The DGDecode plugin is currently maintained by Neuron2)
Colorspaces: YUY2 or YV12
Interlacing: Interlaced or progressive
Speed: Fast! (200+ fps)

Defaults: BlindPP(quant=2, cpu=6, ipp=false, moderate_h=20, moderate_v=40)
Parameters: Usage notes: Here's a screenshot from the Excel Saga opener, taken from the VOB with no postprocessing. As you can see, the background is a complete mess of blocks, almost giving a certain gridding effect -- exactly the kind of problem BlindPP is meant to deal with.
Hover your mouse over the image to bring up the version filtered with BlindPP(quant=12,cpu=4,moderate_h=15,moderate_v=25):

You can see how many of the blocks have been smoothed out. Even the filesize of the .PNG is noticeably smaller.
But these settings are pretty darn strong; if you dare use settings this strong, you had better check the less blocky scenes of your video to make sure that they aren't losing too much detail as a result. You may be able to limit the damage by raising the values of moderate_h and moderate_y.

Something to watch out for: Here you can find a screenshot taken with the totally ridiculous settings BlindPP(quant=32,cpu=4,moderate_h=15,moderate_v=20). Notice the blocks on the line above Excel's eyelashes (there are other blocks, but those are the most obvious) -- that used to be a perfectly fine line. If you see artifacts like this, then you're using BlindPP too aggressively. Back to top

An experimental filter that operates on DCT coefficients.

Type: Plugin
Found in file: DctFilter.dll
Author: Trbarry
Colorspaces: YUY2 or YV12
Interlacing: Progressive only (if your source is interlaced, you can try SeparateFields() before and Weave() after, but I'm telling you, it really doesn't work as well)
Speed: FAST! (300+ fps)

Defaults: DCTFilter(1,1,1,1,1,1,1,1)
Recommended: DCTFilter(1,1,1,1,1,0.75,0.25,0)
Parameters: Each of the 8 parameters is a floating point value that corresponds to a scaling factor for one of the 8 rows and columns of the 8x8 DCT blocks, starting with the top row and leftmost column (the first parameter should always be 1; otherwise your video will seriously look like crap).

Usage notes: Because of the way DCTFilter works, you should always put it after all cropping and resizing if you're trying to use it to increase your compressibility.

For an explanation of the DCT algorithm as used in image compression (if the following notes confuse you), go here.

DCTFilter is a pretty cool idea for a filter; it works directly on the DCT coefficients of an image. The reason this is useful is because distribution codecs such as MPEG-1/-2, DivX, XviD, etc. all use the DCT algorithm with the same 8x8 blocks that DCTFilter works on. DCTFilter can zero out, reduce, or amplify certain DCT coefficients with interesting effects; it's not strictly a smoother. But from a compressibility standpoint, it helps us the most to reduce the highest-frequency coefficients (the rightmost columns and bottom rows of the DCT frequency matrix). That's why the "Recommended" parameters are set the way they are.

However, as godly as DCTFilter can be for compressibility when used in this way (and the gains ARE awesome, make no mistake about that), it can cause problems with edges. Because anime is so full of sharp edges, which rely on high DCT frequencies, this can be a problem -- you may notice that DCTFilter partially undoes your previous sharpening efforts, or makes edges that weren't that well-defined to begin with even less sharp. And personally, I really like sharp, well-defined images. This is why I do not recommend DCTFilter for anime unless you're desperate to give your compressibility a big boost. And if you do use it, make sure that it's not blurring details or unsharpening edges too much.

Here's a screenshot from Azumanga Daioh episode 13, together with the same frame after going through DCTFilter(1,1,1,1,1,0.75,0.25,0) (hover your mouse over the image to see the filtered version):

The difference is very slight, but it is there. For example, pay attention to the topmost desk -- you can see that the horizontal lines lose some of their sharpness. (Okay, so I didn't pick the best reference frame, but you get the idea. The difference would probably be more noticeable if I had done some sharpening first; the edges in this scene aren't the sharpest things in the world.)

I've provided some more screenshots in order to give you an idea of the kinds of things DCTFilter can do besides reducing the higher frequencies to improve compressibility:
DCTFilter(1,1,0,0,0,0,0,0) - almost everything is zeroed out, only allowing the simplest possibilities for each 8x8 block. The file is almost half the size of the original screenshot, but who would want to watch this?
DCTFilter(1,1,1,1,1,1,1,8) - I've amplified the highest frequencies so high that they're impossible to miss. Also, the filesize is ridiculous. Back to top

6 smoothers in one; I'll only be discussing the "a2d" and "w2d" modes here.

Type: Plugin
Found in file: Deen.dll
Author: MarcFD
Homepage: (but see here for the older version that I recommend, as it includes the "W" gradient-preserving modes I'll mention below. MarcFD's page and the AVISynth Filter Collection both contain only the most recent version, which does not have these modes.)
Colorspaces: YV12 only
Interlacing: Progressive only
Speed: Average to slow (36 fps with mode="a2d" and rad=4, but only 10 fps with mode="w2d" and rad=7)

Defaults: Haven't got a clue.
Recommended: Deen(mode="a2d", rad=4, thrY=7, thrUV=9, min=0.5)
Parameters: Usage notes: Many of you already know Deen from EADFAG's page on AVISynth postprocessing, where it was the only spatial smoother gone into in any detail. While it is a good smoother, it's easy to go overboard with it, plus I have the aforementioned memory issues with it (don't expect that to be fixed anytime soon; this filter hasn't been updated since 2003).
If you can keep it from oversmoothing the image, then Deen is worth using; it's capable of posting some nice compressibility gains, and the overall effect is a pleasing one in moderation.

Our next screenshot comes to us from Love Hina, together with the same frame after going through Deen("a2d",7,11,13,min=0.5) (yup, hover over image to see filtered version):

Look at almost any line on the original, and you'll see mosquito noise; then look at how Deen cleans it up. Much better, isn't it? Of course, I did select a frame with mostly areas of flat, solid colors with not much detail; other scenes would probably require weaker settings.

Something to watch out for: I shouldn't even have to give any examples of this, but here's one anyway. Taken with the settings of Deen("a2d",7,14,20) on a source that didn't need it: This is Deen killing trees. Those trees used to be happy trees. Please, think of the trees... not to mention the rampant color fading. If you see things like this happening in your video, then you're using Deen too aggressively. Back to top

Self-proclaimed "spatial monster". Translation: very powerful, use with care.

Type: Plugin
Found in file: eDeen.dll
Author: MarcFD
Colorspaces: YV12 only
Interlacing: Progressive only
Speed: Average to slow (around 20 fps with r=7)

Defaults: eDeen(r=7, ty=14, tuv=21, ly=2, luv=3, hqc=true)
Parameters: Usage notes: The screenshot for this filter is taken from that wonderful shining paragon of visual quality known as A.D. Vision's original R1 release of Neon Genesis Evangelion (yes, that was sarcasm). This is a perfect reference frame for eDeen, because it contains large areas of flat colors with very few details. The target this time is not so much the mosquito noise on the edges as the little variances within the large flat colored areas themselves. The following comparison is brought to you by the settings eDeen(7,7,14,1,2):

Look at any area of color on the original, and you'll find little imperfections everywhere. Then watch as eDeen smooths them out. Just that one filter shaves 29% off the filesize of the image!

In conclusion, eDeen is a great smoother if you don't have much detail to worry about; just make sure you're not washing out the image. Back to top

Masked smoother, designed specifically for anime.

Type: Plugin
Found in file: MSmooth.dll
Author: Neuron2 (Donald Graft)
Homepage: (but see this forum thread for the latest RC version, which fixes the chroma bug; this is the version I've included in the ZIP)
Colorspaces: RGB32 or YV12
Interlacing: Progressive or field-based (at least, I don't know why it wouldn't work on fields)
Speed: Slow (14 fps with strength=3 and chroma=false)

Defaults: MSmooth(mask=false, threshold=15, strength=3, extend=false, blurmore=false, chroma=false)
Parameters: Usage notes: MSmooth was designed specifically for anime, and the idea behind it is a pretty cool one: Since smoothing should never occur across edges and details, a mask is first created that separates out the edges and details; then the REST of the image is smoothed.

Using MSmooth is normally a two-step process: first, with mask=true, you adjust the threshold until the mask preserves a proper amount of detail and edges, then you set mask=false and adjust the strength until the smoothing is at the right level (also tweak threshold if it turns out you're preserving more or less detail than you thought). To illustrate the process, we once again go to Neon Genesis Evangelion (non-Platinum), first with a screenshot that illustrates the mask, taken with settings of MSmooth(threshold=7, strength=3, chroma=true, mask=true):

Notice that the edges are bright while everything else is dark. Looks pretty good, but we'll see that some of the edges aren't preserved as well as this mask might lead us to believe they'd be. As it turns out, these settings preserve almost all of the lines of Kaoru's figure, but not the outlines of the clouds in the background. What follows is the standard comparison shot (the filtered version is what we see after setting mask=false; hover over to compare):

You can see that the clouds' outlines didn't hold up as well as Kaoru's figure. Also, certain edges in Kaoru's hair have also been blurred (check out the top right corner of the frame); thus, a lower value of threshold might have been wise in this case. Still, we got quite a healthy compressibility increase out of the deal.

Something to watch out for: This screenshot, filtered with MSmooth(strength=5,mask=false,threshold=25,chroma=true), illustrates the dangers of not setting threshold low enough. Ugh! Blurry and smeary all over. Not at all attractive. Back to top

Originally meant for degraining purposes but can be used for general denoising.

Type: Plugin
Found in file: RemoveGrain.dll
Author: Kassandro
Colorspaces: YUY2 (with planar=true) or YV12; claims to support RGB32 and RGB24, but I can't get RGB32 to work, and RGB24 gives weird artifacts on most modes. If anyone figures it out, please let me know. Until then, if the mode you want doesn't work with your current colorspace, then convert to YV12. All modes work in YV12.
Interlacing: Interlaced only (modes 13-16) or progressive only (all other modes)
Speed: Insanely FAST!! (800+ fps with mode=17)

Defaults: RemoveGrain(mode=2, modeU=2, modeV=2)
Parameters: Usage notes: RemoveGrain is meant for removing noise of the very small variety, on the order of a pixel or so. The changes it causes are not always the most visible (more subtle than those of, say, Deen or MSmooth), but they can have a large effect on compressibility. And of course, you can't argue with the speed -- this thing is just blazingly fast.
This time, I've chosen a reference frame from Angel Sanctuary. We're going to attempt to reduce the noise on the floor tiles without completely wiping out the texture. As usual, hover your mouse over the original image to see the frame as filtered with RemoveGrain(mode=17).RemoveGrain(mode=17) (yes, I ran the filter twice):

The noise on the floor is reduced quite a bit, and as a result, we've managed to shave 40% off the size of the image file. However, this came at the price of some softening of the lines in the image (especially the details on the chair have been blurred); it wouldn't be a bad idea to combine this with a sharpener of some kind, or to use a safer mode.

As long as you don't lose too much detail, RemoveGrain can give you some very handsome compressibility gains. But if you want to go one step further and use the power of RemoveGrain as part of a very nice spatio-temporal filter, then you'll want to check out LRemoveDust. Back to top

Meant as a deblocker but good for general denoising as well.

Type: Plugin
Found in file: SmoothD.dll
Author: 708145 (Tobias)
Colorspaces: YV12 only
Interlacing: Progressive only
Speed: Slow (15 fps with num_shift=4)

Defaults: SmoothD(quant=3, num_shift=4, adaptive_shift=0, zero_weight=1)
Recommended: SmoothD(2,6,0,1)

Usage notes: SmoothD is a fairly young filter; it's not even up to a version 0.1 yet. However, it's a very good denoiser that's capable of posting some impressive compressibility gains. It works by re-applying the JPEG compression process (I mean DCT-quantize-inverse DCT) on shifted copies of each block (and no, I have no idea how that helps). Though ostensibly developed for deblocking, it can also be effective against mosquito noise.
Take a look at this screenshot from FLCL, first unfiltered and then run through SmoothD(2,6,0,3):

The standard warnings against oversmoothing apply, of course. If you don't have much detail to worry about, then SmoothD might be a very good idea for compressibility purposes. (Especially if you have scenes that need deblocking, of course.) Back to top

The spatial filter from the Dust family. Subtle but effective.

Type: Plugin
Found in file: DustV5.dll
Author: Steady
Homepage: none
Colorspaces: RGB24, RGB32, or YUY2 (note: NO YV12 SUPPORT)
Interlacing: Progressive only (I believe)
Speed: Fast! (180+ fps)

Defaults: SpaceDust()
Parameters: There's only one, and it's not worth mentioning.

Usage notes:

SpaceDust is the spatial-only filter from the Dust family of filters, coded by Steady for AVISynth 2.0.x (a version for 2.5.x was never released). I have no idea how it works, so don't ask (and the source code was never released, so I couldn't figure it out that way either). All I know is that it's reasonably fast and can reduce noise and improve compressibility fairly nicely. If I remember correctly, the visual effects aren't as blatant as those of (say) Deen either.
The biggest complaint about the Dust filters is that they tend to cause blocking, but this shouldn't be much of an issue with SpaceDust. As far as I can tell, the only noticeable harm SpaceDust does is in slight blurring of details. However, if either of these becomes a problem for you, then about all you can do about it is drop it and use a different filter (unfortunately).

Once again, it's back to Neon Genesis Evangelion (not Platinum) for this comparison shot (hover over image to see filtered version):

You may notice... um... only the tiniest difference between the images. Heck, even I can't find anything that really sticks out as a noticeable difference. But the .PNG of the filtered version is 25% smaller than the original, and one can't argue with that. Again, don't ask me how it does it, because I have no idea. Let's just say "it's awesome" and leave it at that. :)
Because the effect is so subtle, SpaceDust can probably be combined with other smoothers to good effect.

In conclusion, SpaceDust is a fine choice for many sources; the effects are subtle, but the compressibility gains can be very nice. Plus, it's fast too. However, if you want to take full advantage of the Dust filters and do some stronger filtering, you might want to look into PixieDust and/or GoldDust; SpaceDust was meant mainly for those looking for maximum speed out of the Dust filters. Back to top

A filter that uses bilateral smoothing. Slow but effective.

Type: Plugin
Found in file: TBilateral.dll
Author: Tritical
Colorspaces: YUY2 or YV12
Interlacing: Progressive or field-based
Speed: Slow to very slow (8 fps at default settings; increasing diameter slows even more)

I'm not actually going to write up this filter yet, because it's quite complicated and very slow (though it's often very effective); I'll cover the other filters first, and hopefully I'll get to writing this one up eventually. Back to top

Simple minmax clipping for cheap, risk-free compressibility gains.

Type: Plugin
Found in file: UnDot.dll
Author: Trbarry
Colorspaces: YUY2 or YV12
Interlacing: Progressive or interlaced (I don't know why interlaced wouldn't work, at least)
Speed: Insanely FAST!! (800+ fps)

Defaults: UnDot()
Parameters: None!

Usage notes:

UnDot's theory of operation is very simple: it clips each pixel to stay within the mininum and maximum values of its eight spatial neighbors (pure minmax clipping, therefore identical to RemoveGrain(mode=1)). This doesn't produce a very visible effect at all, but it helps compressibility. Because UnDot can be used without losing any details* and with hardly any speed penalty, it's a good filter to use before other smoothers.
* Exception: Do NOT use UnDot if your video contains any dither dissolves. Since these rely on single isolated pixels being from a different clip than those around them, UnDot will destroy the effect by whipping those deviant pixels into line with their surroundings.

Because the visual effect of UnDot is barely noticeable, it's pointless to provide a comparison screenshot for it. So I won't. Back to top

The smoothing side of UnFilter is meant to counteract overdone edge enhancement on DVDs.

Type: Plugin
Found in file: UnFilter.dll
Author: Trbarry
Colorspaces: YUY2 or YV12
Interlacing: Progressive or field-based (I believe)
Speed: Insanely FAST!! (800+ fps with HSharp=-10, VSharp=-10)

Defaults: NONE (will give an "Invalid arguments" error if you try calling it without any parameters!)
Recommended (for porn smoothing): UnFilter(HSharp=-5, VSharp=-5)

Usage notes:

UnFilter is a dual-purpose filter: it can be used for either sharpening or smoothing. Both sides were meant to counteract detrimental practices in DVD authoring (its sharpening was meant to reverse oversmoothing; its smoothing was meant to reverse overdone edge enhancement).
When used as a smoother, UnFilter can provide nice compressibility gains, especially considering the blazing speed. However, it works mainly against sharp edges, and anime is full of those. So just make sure you're not smoothing so much that you start losing details.
The recommended values should be safe for most sources; for even better compressibility, lower the values of the arguments (that means closer to -100, not closer to 0, just so we're clear). But if you find that even the recommended values are hurting your edges too much, then use a different filter. Back to top
A wavelet-based denoiser, subtle but effective. A personal favorite.

Type: Plugin
Found in file: VagueDenoiser.dll
Author: Lefungus, Kurosu, and Fizick
Colorspaces: RGB24, RGB32, YUY2, or YV12
Interlacing: Progressive or interlaced
Speed: Average to slow (30 fps in YV12, 22 fps in YUY2; most settings made very little difference)

Defaults: VagueDenoiser(threshold=0, method=3, nsteps=4, chromaT=-1, interlaced=false)
Recommended: VagueDenoiser(threshold=2, method=3, nsteps=6, chromaT=2.0)

Usage notes:

VagueDenoiser translates frames into the wavelet domain, filters the coefficients, and then does an inverse wavelet transform. I admit I don't know just what that means, except it's kind of like the DCT-quant-iDCT process of JPEG compression, except it doesn't use DCT. I know, not very helpful. But somehow, it denoises and boosts compressibility without washing out the frame. Absolute Destiny describes it as "turning blocky noise into... noise which is more vague looking", and I'd say that's about right. The effect really is nice though, most of the time.
Let me make no bones about it: I really like this filter. I use it in most of my encodes, and in all of my filter tests, it always seems to help compressibility noticeably without hurting details or blurring too much (as filters like Deen are more prone to do). The only other spatial smoother I can think of that aids compressibility AND preserves detail so well is SpaceDust.

Here, have another Neon Genesis Evangelion (not Platinum) reference frame (hover over image to see version filtered with VagueDenoiser(threshold=6,method=3,nsteps=6,chromaT=2.0)):

Look at any area of what's supposed to be flat color, and you'll see the noise being smoothed away -- not removed completely, but reduced and made much less obvious. Isn't it much nicer that way? But of course, I only used settings that strong because the source was so bad; such a high threshold would probably be overkill on most sources. Besides, lower thresholds preserve even more detail. :)

One more note: If it's your goal in life to hunt down and destroy mosquito noise wherever it may be found, then you might be better off using a different denoiser. VagueDenoiser doesn't seem to do as well on mosquito noise as it does on blocky noise and fluctuations in large flat areas. Back to top

Home - Temporal Smoothers - Spatio-Temporal Smoothers - Sharpeners - Dealing with Dot Crawl - Dealing with Rainbows

Last updated on 10/2/08
Questions? Comments? Concerns? Contact Scintilla
Fight Spam! Click Here!