Scintilla's Guide to AVISynth Postprocessing Filters: Sharpeners

aWarpSharp - DeHalo_Alpha - FastLineDarken - LimitedSharpenFaster - mfToonLite -
MSharpen - PSharpen - SSXSharpen - Toon - UnFilter - vmToon - WarpSharp

( Home - Spatial Smoothers - Temporal Smoothers - Spatio-Temporal Smoothers - Dealing with Dot Crawl - Dealing with Rainbows )

Intro

Sharpening is the antithesis of spatial smoothing: it tries to accent, to enhance, to bring out the differences between pixels instead of trying to reduce them. And this usually hurts compressibility (but not always; for example, the warpsharpeners may actually help it).

So why would you want to do it? Easy answer: because The Sharper Image sharper images are nice to look at; well-defined edges are more pleasing to the eye than wimpy ones. And it just so turns out that we've got a few AVISynth filters that are meant just for enhancement of edges such as those found in anime, as well as those that aren't but still work well for our purposes. So here are some of them.

It should be noted that DeHalo_Alpha is NOT actually a sharpener; it's meant to reduce haloing, an artifact some sharpeners can leave behind if you overdo it (though you may also have halos in your source video before any processing). I didn't feel quite right putting it on the spatial smoothers page. Also, you may notice that the well-known mfToon is not on the list. That's because, in its more recent incarnations, its name was changed first to vmToon and then to just Toon.
One last note: these sharpeners should always be applied AFTER you've smoothed your image with the filters on the previous pages. If you sharpen before smoothing, the sharpening can accentuate noise, making it harder for the smoothers to remove.


aWarpSharp
A different warpsharper by MarcFD; meant for anime

Type: Plugin
Found in file: aWarpSharp.dll
Author: MarcFD
Homepage: http://ziquash.chez-alice.fr/
Colorspaces: YV12 only
Interlacing: Progressive only
Speed: Fast

Defaults: aWarpSharp(depth=16.0, blurlevel=2)
Parameters: Usage notes: aWarpSharp is a warpsharpening filter from MarcFD that claims to use a different algorithm from other warpsharpeners, and it appears to be optimized for anime sources. For those of you who have read this far and are wondering just what "warpsharpening" is, it is a sharpening method that differs from normal convolution-based sharpening in that it actually warps the image, expanding large flat areas and thinning lines. Warpsharpening is therefore very useful for older anime sources that have fat edges, as demonstrated by the following screenshot from Episode 24 of A.D. Vision's original R1 release of Neon Genesis Evangelion, treated with AWarpSharp(depth=22.0, blurlevel=2) (hover to see filtered version):

The difference is clear ClearEyes -- AND, would you believe that the warpsharpened image is actually MORE compressible than the original (shaved 32 KB off the size of the PNG)? Bet you didn't expect to read THAT anywhere on the sharpeners page.
As if that wasn't enough, this and other warpsharpeners can also help with dot crawl (though it's still better to use a dedicated dot crawl filter, such as DeDot, if possible; we'll get to those on a later page).

Something to watch out for: Unfortunately, it's easy to go overboard with warpsharpeners. Here is an example of what can happen when you do (settings: AWarpSharp(depth=50.0, blurlevel=2)). The lines are getting so thin as to almost disappear at points, and the warping is beginning to make some shapes break up; Kaoru's eyes in particular look odd. So for the love of Pen-Pen, watch your settings and please don't do this. Back to top


DeHalo_Alpha
NOT a sharpener, but useful for reducing halo artifacts that can occur when sharpening.

Type: Script function
Requires: MT_MaskTools (aka MaskTools v2.0), Repair (any one of the four DLLs)
Author: Didée (using plugins by Kurosu, Manao, and Kassandro); migrated to MaskTools 2.0 by Foxyshadis
Homepage: doom9 forum
Colorspaces: YV12 only
Interlacing: Progressive only
Speed: Slow (but improves to average if ss=1)

Defaults: DeHalo_alpha(rx=2.0, ry=2.0, darkstr=1.0, brightstr=1.0, lowsens=50, highsens=50, ss=1.5)
Parameters:

Usage notes:

Haloing is what often happens when you oversharpen an image. If you see a large flat area of color that suddenly gets brighter when it gets right up against a dark edge, that's haloing. It's caused by what's known as "overshooting" in convolution-based sharpeners: the sharpener does its work by amplifying the difference between adjacent pixels, but if the difference makes (say) brightened pixels significantly brighter than their OTHER neighbors that were originally SUPPOSED to be the same color, then that's overshooting.

This is one of those filters whose effects can be much more easily seen than described in words, so here's a comparison frame, taken from episode 24 of Neon Genesis Evangelion with settings of DeHalo_alpha(brightstr=2, ss=2.0) (hover your mouse over to see the filtered version); I realize that this frame looks VERY familiar, but (happily) halos that naturally occur on DVDs are hard to come by with recent releases, so I figured I should use the first good example I found.

Look at the edges on Gendo's eyebrows, glasses, gloves, etc. and how the halos disappear (mostly) in the filtered version. Also, we've managed to trim down the lossless filesize somewhat. Like I said, though, you won't often find halos this bad on recent animé releases, so you'll probably find this filter most useful for placing after your OWN sharpening jobs... that is, if your sharpeners didn't correct for overshoot already.
Of course, one thing you may have noticed is that the dehaloing makes the image look blander and more boring. You may decide you prefer the slightly haloed look. Which is fine too, but I should reiterate that dehaloing also improves compressibility. Your choice. Back to top


FastLineDarken
It darkens lines. Fast (relatively). That makes it useful for animé.

Type: Script function
Requires: MaskTools (1.5.1+)
Author: Vectrangle (using plugins by Kurosu and Manao)
Homepage: doom9 forum
Colorspaces: YV12 only
Interlacing: Progressive only
Speed: Average when thinning=0; slow otherwise

Defaults: FastLineDarken(strength=48, luma_cap=191, threshold=4, thinning=24)
Parameters: Usage notes: Thankfully, I don't have to say much here, because this function does exactly what it says. Animé often has a lot of dark edges, and it often improves the look to darken these edges further, to bring them out a little more. This is what FastLineDarken was made for: it's a line darkening specialist, only paying attention to dark edges, not like other sharpeners that sharpen everything. (So if your source has details like leaves in trees, FastLineDarken won't sharpen those.)

For this filter, we'll use a sample frame from episode 13 of Azumanga Daioh. This source doesn't require much smoothing, but the edges suffer from SERIOUS anorexia (I don't think any other source I've ever worked with has had thinner lines); and this, combined with a lack of definition on some of them, causes them to threaten to disappear sometimes. You'll see that we can use FastLineDarken to bring such lines out quite dramatically; settings for this one were FastLineDarken(strength=150,luma_cap=225,thinning=0) (hover over to view filtered version):

You can bet that THOSE lines won't get smeared by the encoder. Only problem is that FastLineDarken doesn't have any supersampling built in, so it can cause aliasing sometimes; this can be mitigated by adding some into your filter chain (resize up, darken, then resize back down). Back to top


LimitedSharpenFaster
Sharpening with correction for overshoot (the "Limited" part). Much faster than the original LimitedSharpen.

Type: Script function
Requires: MT_MaskTools (aka MaskTools v2.0), RemoveGrain 1.0 beta, WarpSharp package
Author: Didée (using plugins by Kurosu, Manao, Kassandro, and ???)
Homepage: http://www.avisynth.org/mediawiki/LimitedSharpen
Colorspaces: RGB32, RGB24, YUY2, or YV12
Interlacing: Progressive only
Speed: Somewhat slow

(Note: This function has too many parameters to list here; I only give the most useful ones. If you're curious, check out the wiki page for a full listing.)
Defaults: LimitedSharpenFaster(ss_x=1.5, ss_y=1.5, dest_x=[clip width], dest_y=[clip height], strength=100, overshoot=1, undershoot=1, soft=0, edgemode=0)
Parameters:

Usage notes: LimitedSharpenFaster actually has a few sharpening modes, but I haven't bothered to go into them here, as the default ("range sharpening") works fine most of the time. The idea that really sets this filter apart from other sharpeners is the limiting of sharpened pixels relative to their spatial neighbours, specifically designed to combat overshoot and therefore avoid introducing halos. It was one of the first AVISynth sharpeners to do this (if not THE first), and it still remains mostly unique in this regard.

The following comparison shot comes from episode 2 of Hellsing (TV); the source looks good, but it's rather blurry. A simple LimitedSharpenFaster helps quite a bit to bring it into proper focus; however, for the purposes of demonstration, I decided once wasn't enough and elected to hit it TWICE with the filter, thus LimitedSharpenFaster(strength=200).LimitedSharpenFaster() (hover mouse over to see filtered version):

Gorgeous, ne? Of course, as with all such sharpeners, the price you pay is in compressibility -- but that's why we denoise and smooth the footage first. Back to top


mfToonLite
Based on mfToon, but much faster because it doesn't warpsharp or supersample.

Type: Script function
Requires: MT_MaskTools (aka MaskTools v2.0), WarpSharp
Author: mf and Vectrangle (using plugins by Kurosu, Manao, and ???)
Homepage: http://mf.creations.nl/avs/ (Ignore what the page says about v0.44 being the most recent. If you go into "function includes", you can clearly see the list go up to v0.52. Why mf doesn't have v0.54 (the one we're using) in there, I don't know.)
Colorspaces: YV12 only
Interlacing: Progressive only
Speed: Slow

Defaults: mfToonLite(strength=255, dstren=255, drange=64, dboost=1.0, dlimit=30, mask="fastest")
Parameters:

Usage notes: mfToonLite was basically born out of a desire to make a realtime filter (for use at playback, e.g. in ffdshow's AVISynth filters section) out of the old mfToon, which was known as a great line darkener and sharpener for animé but which was also notoriously slow. (mfToon has since developed into vmToon, which is still horrendously slow; we will discuss this filter later.) So mf removed the warpsharpening and supersampling capabilities from mfToon, and mfToonLite was the result. Sure enough, it is significantly faster than its progenitor.

The following sample frame comes from Haré+Guu, with mfToonLite used at default settings (hover mouse over to view filtered version). It's easiest to see the differences in the hair, but all the other edges are enhanced as well.

Back to top


MSharpen
Masked sharpener (to only sharpen edges), designed specifically for anime.

Type: Plugin
Found in file: MSharpen.dll
Author: Neuron2 (Donald Graft)
Homepage: http://www.neuron2.net/msharpen/msharpen.html
Colorspaces: RGB32, YUY2, or YV12
Interlacing: Progressive only
Speed: Fast

Defaults: MSharpen(threshold=10, strength=100,mask=false, highq=true)
Parameters: Usage notes: Remember MSmooth (from the spatial smoothers page)? The filter that created a detail mask and then only smoothed the parts of the image that were not detected as detail? Well, MSharpen follows pretty much the exact same design philosophy, except kind of opposite: this time, we're taking the detail only and sharpening it. As you might expect, such an approach usually works very well for animé.
Using MSharpen is normally a two-step process: first, with mask=true, you adjust the threshold until the mask covers all the edges and details you want to sharpen, then you set mask=false and adjust the strength until the sharpening is at the right level (also tweak threshold if it turns out you're sharpening more or less detail than you thought). I already illustrated the first step on the smoothers page; it's here if you need to review it.

Our sample frame for this one comes from episode 3 of Princess Tutu; this is a series that is generally quite clean, but can be kind of soft in some scenes. You may think this frame looks pretty good as it is, but watch what happens when you roll your mouse over (settings were MSharpen(threshold=10,strength=140)):

Once you've seen the sharpened version once, you notice the difference, don't you? The original suddenly looks blurred and out of focus by comparison. The only problem is slight aliasing on some of the edges, but this can be mitigated by adding in supersampling, as described above. Back to top


PSharpen
Two-point sharpening that avoids overshoot.

Type: Script function
Requires: MaskTools (1.5.1+)
Author: ilpippo80
Homepage: doom9 forum
Colorspaces: YV12 only
Interlacing: Progressive only
Speed: Average

Defaults: PSharpen(strength=25, threshold=75, ss_x=1.0, ss_y=1.0, dest_x=[clip width], dest_y=[clip height])
Parameters:
Before I begin this part, I have to explain that this sharpener compares each pixel's (luma) value to the maximum and minimum values of its eight spatial neighbours. If you see me refer to min and max in the following discussion, keep in mind that it's these local minima and maxima that I'm talking about. Usage notes: PSharpen is one of those sharpeners I keep coming back to for animé; it consistently gives pleasing results, it doesn't overshoot, and it is faster than LimitedSharpenFaster. However, I owe everyone an explanation of this filter's design philosophy, because I fear I wasn't able to explain it very well in the Parameters section. It's not that easy to put into words; I find it easier to follow this image to see what's going on.

As I've mentioned, PSharpen starts work on each pixel by looking at each of its 8 spatial neighbour pixels and taking the minimum and maximum from those eight pixel values (luma only); this is why they call it a "two-point" sharpener. Depending on the values of strength and threshold, it then divides the spectrum between the local minimum and maximum into a range that gets expanded (around the midpoint) and two ranges that get compressed (around the min/max; see picture linked above).
Following this analogy (which should pose no trouble to those of you who've worked with dynamic range processors in audio -- it's basically a compander), strength determines the severity of the expansion and threshold controls the degree of the compression.
The two things you can be sure of with PSharpen: (1) each pixel will always end up farther away from the midpoint between its local min/max than it was before, and (2) each pixel will always remain within the range defined by its local min and max.

PSharpen operates on every pixel, but the effect in the following frame is most noticeable around the eyes; the rest is subtler. The frame is from Princess Tutu, and the arguments used were PSharpen(strength=90,threshold=90,ss_x=1.5,ss_y=1.5) (hover mouse over to view filtered version):

As with LimitedSharpenFaster, PSharpen calls can sometimes be stacked to good effect if you want a more dramatic sharpening than even the strongest settings can provide. Back to top


SSXSharpen
Simple sharpening with supersampling. (Has nothing to do with snowboarding.)

Type: Script function
Requires: WarpSharp
Author: mf (using plugin by ???)
Homepage: http://mf.creations.nl/avs/ (It's under "function includes", as part of the SharpTools script.)
Colorspaces: RGB32, YUY2, or YV12
Interlacing: Progressive only
Speed: Glacially SLOW at default settings; gets up to more normal levels of slow at ssw=2, ssh=2

Defaults: SSXSharpen(ssw=4, ssh=4, xstren=255, xthresh=255)
Parameters: Usage notes: SSXSharpen is based on XSharpen, a VirtualDub filter by Donald Graft (Neuron2), a clone of which is included in the WarpSharp package plugin. The idea is similar to PSharpen but much simpler: Each pixel is compared to the minimum and maximum (luma only) of its 8 spatial neighbour pixels. If the difference between the pixel and whichever of its local min/max is closer to it is less than xthresh, then the pixel's original value is blended with that of the min/max (whichever one is closer), with the blending amount controlled by xstren.
So what does SSXSharpen add to this? Simple: it wraps XSharpen in supersampling. That's it. Just resize up, XSharpen, and resize down. And why is this supersampling needed? Because using XSharpen on animé sources without it can produce some really ugly results.

The following frame comes from FUNimation's release of Fruits Basket, filtered with SSXSharpen(ssw=2,ssh=2) (hover mouse over to view filtered version). This time I picked a frame with a good bit of detail in the scenery, rather than yet another frame featuring dark outlines of characters, to show how this and similar sharpeners can improve your backgrounds.

Unfortunately, it really doesn't make sense to stack this filter -- it's so slow as to be impracticable. Back to top


Toon
Faster and simpler than mf/vmToon, but you only get one parameter. Comes in two flavors.

Type: Plugin
Found in file: Toon-v1.0.dll and Toon-v1.0-lite.dll
Author: mf
Homepage: http://mf.creations.nl/avs/; announced on doom9 forum
Colorspaces: YV12 only
Interlacing: Progressive only
Speed: Somewhat fast (Toon) to very FAST! (ToonLite)

Defaults: Toon(strength=0.6)
or ToonLite(strength=0.6)

Parameters: Usage notes: Toon 1.0 is the latest release in mf's series of __Toon filters; I hesitate to say "the latest incarnation of mfToon/vmToon" because the algorithm was almost completely rewritten as of version 0.8. mf's goal with the rewrite was to simplify and speed up the filter, and it worked. Whereas vmToon has 13 parameters, Toon/ToonLite has only one -- how much simpler can you get?

So what does vmToon have that Toon does without? Well, as far as I can tell: Toon does not do any supersampling, it uses something besides XSharpen to do its sharpening, and it doesn't let you control the amount of line thinning (that is, short of disabling it completely by using ToonLite instead) or the thresholds for line detection or noise sharpening. Still, even though it's not as tweakable as its predecessor, Toon still produces a very usable result, and does it in a fraction of the time vmToon would have taken.

Again, I've selected a sample frame from Fruits Basket; this time, I had to make sure it DID have some outlines in it, as that's what this filter is supposed to target. All the outlines get darkened, but the effect is perhaps most obvious on Tohru's shirt. And to make it even better, the size of the lossless image has only increased by 4 KB! The setting was Toon(strength=0.75) (hover mouse over to view filtered version):

Stacking this filter is not recommended, as you can get some pretty ugly aliasing pretty fast, unless you stick to low settings of strength. Back to top


UnFilter
The sharpening side of UnFilter is meant to counteract overdone softening on DVDs.

Type: Plugin
Found in file: UnFilter.dll
Author: Trbarry
Homepage: http://www6.impacthosting.com/trbarry/downloads.htm
Colorspaces: YUY2 or YV12
Interlacing: Progressive or field-based (I believe)
Speed: Insanely FAST!!

Defaults: NONE (will give an "Invalid arguments" error if you try calling it without any parameters!)
Recommended (for porn sharpening): UnFilter(HSharp=5, VSharp=5)
Parameters:

Usage notes: UnFilter is a dual-purpose filter: it can be used for either sharpening or smoothing. Both sides were meant to counteract detrimental practices in DVD authoring (its sharpening was meant to reverse oversmoothing; its smoothing was meant to reverse overdone edge enhancement).
When used as a sharpener, well... let's just say: why didn't I try this filter a lot sooner? I don't know what this filter's design philosophy is, but it provides very natural-looking sharpening for animé; in the tests I'm running right now, it's looking generally gorgeous. The result looks kind of like that of LimitedSharpenFaster or PSharpen, but UnFilter runs way faster than either of those scripts. And it doesn't even cause lots of aliasing.

For example, look at this screencap from Hellsing (same episode I used for the LimitedSharpenFaster sample frame), filtered with just one pass of UnFilter(100,100) (hover mouse over to see filtered version). This would normally be considered an extreme setting, but for a source as blurry as Hellsing, it works nicely:

Come on, you gotta admit those edges are pretty hot. Back to top


vmToon
The successor to mfToon. Darkens lines, thins lines, and does supersampled sharpening all in one, but infamously slow.

Type: Script function
Requires: MT_MaskTools (aka MaskTools v2.0), WarpSharp
Author: mf and Vectrangle (using plugins by Kurosu, Manao, and ???)
Homepage: http://mf.creations.nl/avs/; unofficial port to MT_MaskTools at http://manao4.free.fr/ (this is the version I include)
Colorspaces: YV12 only
Interlacing: Progressive only
Speed: Glacially SLOW at default settings (we're talking less than half the speed of PixieDust here!). Turning off sharpening gives a considerable speed boost; turning off thinning on top of that gets it up to average speed (though turning off thinning BY ITSELF won't help the speed). If you want to keep the sharpening, you can turn down the supersampling factors to improve the speed.

Defaults: VMToon(strength=48, luma_cap=191, threshold=4, thinning=24, sharpen=true, mask=true, ssw=4, ssh=4, xstren=255, xthresh=255)
Parameters: Usage notes: In the second edition of the venerable AD&E's Guide to All Things Audio and Video, AbsoluteDestiny described the mfToon filter as "the god of anime sharpening" and "nothing short of insanity". What it was was an all-in-one: line darkening, line thinning, and two-point sharpening. What it ALSO was was horrendously slow.
More recent developments to the filter have seen Vectrangle (the author of FastLineDarken, already covered farther up the page) on board as well as mf. So, the name was changed to vmToon to reflect the new co-author. (If you noticed that the descriptions for some of those parameters looked familiar, that's why.) Most of mfToon's original line darkening code has been replaced by Vectrangle's contributions. It's still slow as hell, but it gets the job done.

NOTE: I've been informed that the Toon filter, discussed above, is actually the latest evolution of the mfToon/vmToon series. However, I'm keeping vmToon here for those who prefer to tweak and experiment. If you want something simpler (but less tweakable) and much faster, use Toon instead.

For the following screencap, we revisit again A.D. Vision's original (not Platinum, that is) R1 release of Neon Genesis Evangelion, which has long been generally agreed to be an awful source to work with. As you will see, in addition to all the noise we attempted to clean up on the spatial smoothers pages, the rainbows we'll tackle on the rainbow removal page, and the horrific frame blending which is beyond the scope of this guide, the show could also benefit from some sharpening and edge enhancement.
The frame was filtered with vmToon(strength=100, luma_cap=150, threshold=2, thinning=100, ssw=3,ssh=3) (hover your mouse over to see the filtered version):

Would you believe that the sharpened lossless image is only 4% bigger, filesize-wise, than the original? The results speak for themselves; notice how even the background details, such as where the railing meets the ground, have been improved. The only bad thing about the sharpened image is that the line running down the left side of the frame, apparently a bad haloing artifact from the original crappy transfer, has been darkened; we didn't want that. This can be fixed by carefully tweaking luma_cap, or else by just cropping the stupid thing out like you probably would have done anyway. :P Back to top


WarpSharp
A clone of VirtualDub's warpsharp filter.

Type: Plugin
Found in file: WarpSharp.dll
Author: ???, after Avery Lee and Donald Graft
Homepage: http://www.geocities.co.jp/SiliconValley-PaloAlto/2382/ (or, in English)
Colorspaces: YUY2 or YV12
Interlacing: Progressive only
Speed: Average, but slows as blur increases

Defaults: WarpSharp(depth=128, blur=3, bump=128)
Parameters: Usage notes:

This filter is a clone of the original WarpSharp filter for VirtualDub, written by Avery Lee and modified to be able to be imported into AVISynth by Donald Graft (neuron2). Some Japanese people, names unknown (though you can find their website listed above), turned it into a dedicated AVISynth filter, as part of a plugin package. It appears they added a parameter or two while they were at it.
For those of you who came in late and didn't read this under the writeup for aWarpSharp, warpsharpening is a sharpening method that differs from normal convolution-based sharpening in that it actually warps the image, expanding large flat areas and thinning lines.

For this filter, I finally decided I was sick of using the original (non-Platinum) R1 release of Neon Genesis Evangelion... and so I switched to the remastered (Platinum) version instead. :P Though it's pure heaven compared to the original source, its edges can still be a little fat for my tastes. So here's a screenshot from episode 5, filtered with WarpSharp() (default settings; hover mouse over to view filtered version):

And once again, just as with aWarpSharp, I'm happy to report that the sharpened lossless image is actually smaller than the original -- not by much, but still a far sight better than most convolution-based sharpeners. The edges are nice and slim (but not disappearing), Shinji's about to get bitchslapped, and all is right with the world. :)

Something to watch out for: Same warning as applied to aWarpSharp applies to this one: it's easy to go overboard with warpsharpeners. Here is an example of what can happen when you do. The lines are getting so thin as to almost disappear at points, and the warping is beginning to make some shapes break up; Kaoru's eyes in particular look odd. So for the love of Pen-Pen, watch your settings and please don't do this. Back to top


Home - Spatial Smoothers - Temporal Smoothers - Spatio-Temporal Smoothers - Dealing with Dot Crawl - Dealing with Rainbows

Last updated on 9/12/09
Questions? Comments? Concerns? Contact Scintilla
Fight Spam! Click Here!