Bilateral filter

Definition

The bilateral filter is defined as

: I^\text{filtered}(x) = \frac{1}{W_p} \sum_{x_i \in \Omega} I(x_i)f_r(|I(x_i) - I(x)|)g_s(|x_i - x|),

and normalization term, {W_p}, is defined as

: W_p = \sum_{x_i \in \Omega}{f_r(|I(x_i) - I(x)|)g_s(|x_i - x|)}

where

: I^\text{filtered} is the filtered image; : I is the original input image to be filtered; : x are the coordinates of the current pixel to be filtered; : \Omega is the window centered in x, so x_i \in \Omega is another pixel; : f_r is the range kernel for smoothing differences in intensities (this function can be a Gaussian function); : g_s is the spatial (or domain) kernel for smoothing differences in coordinates (this function can be a Gaussian function).

The weight W_p is assigned using the spatial closeness (using the spatial kernel g_s) and the intensity difference (using the range kernel f_r). Consider a pixel located at (i, j) that needs to be denoised in image using its neighbouring pixels and one of its neighbouring pixels is located at (k, l). Then, assuming the range and spatial kernels to be Gaussian kernels, the weight assigned for pixel (k, l) to denoise the pixel (i, j) is given by : w(i, j, k, l) = \exp\left(-\frac{(i - k)^2 + (j - l)^2}{2 \sigma_d^2} - \frac{|I(i, j) - I(k, l)|^2}{2 \sigma_r^2}\right),

where σd and σr are smoothing parameters, and I(i, j) and I(k, l) are the intensity of pixels (i, j) and (k, l) respectively.

After calculating the weights, normalize them: : I_D(i, j) = \frac{\sum_{k, l} I(k, l) w(i, j, k, l)}{\sum_{k, l} w(i, j, k, l)},

where I_D is the denoised intensity of pixel (i, j).

Parameters

• As the range parameter σr increases, the bilateral filter becomes gradually equivalent to Gaussian blur; i.e. σd becomes dominant, because the gaussian kernel is then mostly flat.
• As the spatial parameter σd increases, the larger features get smoothened.

Limitations

The bilateral filter in its direct form can introduce several types of image artifacts:

• Staircase effect – intensity plateaus that lead to images appearing like cartoons
• Gradient reversal – introduction of false edges in the image.

There exist several extensions to the filter that deal with these artifacts, like the scaled bilateral filter that uses downscaled image for computing the weights. Alternative filters, like the guided filter, have also been proposed as an efficient alternative without these limitations.

Implementations

Adobe Photoshop implements a bilateral filter in its surface blur tool.

GIMP implements a bilateral filter in its Filters → Blur tools; and it is called Selective Gaussian Blur. The free G'MIC plugin Repair → Smooth [bilateral] for GIMP adds more control. A simple trick to efficiently implement a bilateral filter is to exploit Poisson-disk subsampling.

OpenCV implements the function: bilateralFilter( source, destination, \Omega, f_r, g_s ).

Related models

The bilateral filter has been shown to be an application of the short time kernel of the Beltrami flow that was introduced as an edge preserving selective smoothing mechanism before the bilateral filter.

Other edge-preserving smoothing filters include: anisotropic diffusion,Durand, Frédo and Dorsey, Julie. "Fast Bilateral Filtering for the Display of High-Dynamic-Range Images" In ACM Transactions on Graphics, Volume 21, Issue 3, pp 257–266. https://people.csail.mit.edu/fredo/PUBLI/Siggraph2002/DurandBilateral.pdf weighted least squares, edge-avoiding wavelets, geodesic editing, guided filtering, and domain transforms.

References

References

1. (2011). "A Low-Memory, Straightforward and Fast Bilateral Filter Through Subsampling in Spatial Domain". Computer Graphics Forum.
2. (1998). "Bilateral filtering for gray and color images".
3. (2007). "Limitations? - A Gentle Introductionto Bilateral Filteringand its Applications".
4. "Guided Image Filtering".
5. (December 2011). "2011 Third National Conference on Computer Vision, Pattern Recognition, Image Processing and Graphics".
6. "Guided Image Filtering".
7. "G'MIC - GREYC's Magic for Image Computing: An Open and Full-Featured Framework for Image Processing".
8. R. Kimmel, R. Malladi, and N. Sochen. Images as embedding maps and minimal surfaces: Movies, color, and volumetric medical images. IEEE CVPR'97, pp. 350-355, Puerto Rico, June 17–19, 1997. https://www.cs.technion.ac.il/~ron/PAPERS/cvpr97.pdf
9. R. Kimmel, R. Malladi, and N. Sochen. Images as Embedded Maps and Minimal Surfaces: Movies, Color, Texture, and Volumetric Medical Images. International Journal of Computer Vision, 39(2):111-129, Sept. 2000. https://www.cs.technion.ac.il/~ron/PAPERS/KimMalSoc_IJCV2000.pdf
10. N. Sochen, R. Kimmel, and A.M. Bruckstein. Diffusions and confusions in signal and image processing, Journal of Mathematical Imaging and Vision, 14(3):195-209, 2001.https://www.cs.technion.ac.il/~ron/PAPERS/SocKimBru_JMIV2001.pdf
11. Farbman, Zeev, Raanan Fattal, Dani Lischinski, and Richard Szeliski. "Edge-preserving decompositions for multi-scale tone and detail manipulation." In ACM Transactions on Graphics, vol. 27, no. 3 (2008): 67. http://www.cs.huji.ac.il/~danix/epd/
12. Fattal, Raanan. "Edge-avoiding wavelets and their applications." In ACM Transactions on Graphics vol. 28, no. 3 (2009): 22. http://www.cs.huji.ac.il/~raananf/projects/eaw/
13. Criminisi, Antonio, Toby Sharp, Carsten Rother, and Patrick Pérez. "Geodesic image and video editing." In ACM Transactions on Graphphics (TOG), vol. 29, no. 5 (2010): 134. http://research.microsoft.com/apps/pubs/default.aspx?id=81528
14. He, Kaiming, Jian Sun, and Xiaoou Tang. "Guided image filtering." In Computer Vision–ECCV 2010, pp. 1-14. Springer Berlin Heidelberg, 2010. http://kaiminghe.com/eccv10/index.html {{Webarchive. link. (2017-12-19)
15. Gastal, Eduardo S. L., and Manuel M. Oliveira. "Domain transform for edge-aware image and video processing." In ACM Transactions on Graphics, vol. 30, no. 4 (2011): 69. http://inf.ufrgs.br/~eslgastal/DomainTransform/