Filter Images and Videos

The image filtering is a neighborhood operation in which the value of any given pixel in the output image is determined by applying a certain algorithm to the pixel values ​​in the vicinity of the corresponding input pixel. This technique is commonly used to smooth, sharpen and detect edges of images and videos.

Let's learn the meaning of some terms used when discussing image filtering techniques, kernel and convolution.


Kernel


Kernel is a matrix with an odd height and an odd width. It is also known as convolution matrix, mask or filter. Kernels are named based on their value distribution. Kernel is used to define a neighborhood of a pixel in a image filtering operation. Some of the popular kernels are Normalized box filter, Gaussian kernel, Laplacian kernel, edge detecting kernels etc. Kernels can be defined with different sizes. But large kernels result in a large processing time.

This is a 3 x 3 Normalized Box filter. This kernel is used in homogeneous smoothing (blurring).


This is a 5 x 5 Normalized Box filter. This kernel is also used in the homogeneous smoothing. In the same way, you can create a Normalized Box filter with any size to be used in the homogeneous smoothing .


This is a 3 x 3 Gaussian kernel used in Gaussian smoothing (blurring). 


This is a 5 x 5 Gaussian kernel used in Gaussian smoothing (blurring). In the same way, you may create a Gaussian kernel with any size. The value distribution of the kernel should be calculated using the 2-D Gaussian function.
2-D Gaussian Function
2-D Gaussian Function
x, y is the coordinates of the kernel where the origin is placed at the center. (i.e.  x = 0 and y = 0 at the center element of the kernel.) σ is the standard deviation of the Gaussian distribution. For larger standard deviations, larger kernels are required in order to accurately perform the Gaussian smoothing. The standard deviation of the following 5 x 5 Gaussian kernel is 1.

Convolution


Convolution is a mathematical operation performed on images by sliding a kernel across the whole image and calculating new values for each pixels based on the value of the kernel and the value of overlapping pixels of original image.

Convolution


C22 = (K11 x A11 + K12 x A12 + K13 x A13) + (K21 x A21 + K22 x A22 + K23 x A23) + (K31 x A31 + K32 x A32 + K33 x A33)
C23 = (K11 x A12 + K12 x A13 + K13 x A14) + (K21 x A22 + K22 x A23 + K23 x A24) + (K31 x A32 + K32 x A33 + K33 x A34)
C24 = (K11 x A13 + K12 x A14 + K13 x A15) + (K21 x A23 + K22 x A24 + K23 x A25) + (K31 x A33 + K32 x A34 + K33 x A35)

C32 = (K11 x A21 + K12 x A22 + K13 x A23) + (K21 x A31 + K22 x A32 + K23 x A33) + (K31 x A41 + K32 x A42 + K33 x A43)
C33 = (K11 x A22 + K12 x A23 + K13 x A24) + (K21 x A32 + K22 x A33 + K23 x A34) + (K31 x A42 + K32 x A43 + K33 x A44)



In the same way C34, can be calculated in the convolved image. But other values in the boundary of the convolved image (e.g C11, C12, etc) cannot be calculated in the same way because the kernel is partially overlapped with the original image at the boundary. Therefore non-overlapped non-existing pixel values should be calculated in order to determine the pixel values at the boundaries of the convolved image. There are various techniques to handle this boundary values, but I'm not going to discuss it in this tutorial.




After learning bit of theory of image filtering, let's learn some use cases of image filtering techniques with OpenCV C++ examples.



  • Smooth / Blur Images 

When an image is acquired by a camera or other method, the image may be corrupted by random dots and noises. Smoothing/blurring is one of image processing techniques to eliminate such imperfections and improve the image. Image smoothing techniques can be applied even for each frames of a video to eliminate imperfections and improve the video.
Posted by Shermal Fernando