# Infrared and visible image fusion technology based on directionlets transform

- Xin Zhou
^{1}Email author, - Xin Yin
^{1}, - Rui-An Liu
^{1}and - Wei Wang
^{1}

**2013**:42

https://doi.org/10.1186/1687-1499-2013-42

© Zhou et al.; licensee Springer. 2013

**Received: **15 December 2012

**Accepted: **21 January 2013

**Published: **20 February 2013

## Abstract

The article provides an infrared and visible image fusion algorithm based on directionlets transform. The registered original images were decomposed into the low-frequency and high-frequency coefficients by directionlets transform. Getting the mean of the low-frequency coefficients, applying the local variance maximum principle to the high-frequency coefficients, thereby the fusion coefficients of the fused image can be acquired. Finally, the fused image was obtained using inverse directionlets transform. The experiment indicates that the fusion algorithm can extract the original image features better. Compared with the traditional fusion algorithms, the fusion algorithm presented in this article provides better subjective visual effect, and the standard deviation and entropy value would be somewhat increased.

### Keywords

Infrared image Visible image Image fusion Directionlets transform## 1. Introduction

Infrared and visible image fusion is part of multi-source image fusion. Multi-source image fusion is the process of acquiring unified description towards the formation of high-performance perception system with employing different types of image sensors, and combing two of more kinds of image information effectively. It is the technology with comprehensive and optimized treatment for multi-information’s acquisition, presentation, and internal relations [1].

Because the approaches of the infrared detectors to obtain measured wavelength range of the target information are different and with different imaging modalities, the infrared image and the visible image’s information are very different and complementary. Through the fusion of the infrared image and visible image, plus synthesizing the complementary and redundant information of the two types of images, the fused image’s object’s contour can be clearer than the original one with the characteristics of information-rich, and easy to identify, and the image sensor’s perception towards the environment can also be extended.

Image fusion is mainly divided into three levels, namely the pixel-level fusion, feature fusion, and decision-level fusion. This article only focuses on the pixel-level fusion. There are many existing pixel-level image fusion methods, including the weighted average method [2], the tower decomposition method [3], principal component analysis (PCA) method [4] as well as fusion based on wavelet transform methods. The wavelet transform has become an important tool in image fusion method for its excellent feature of time frequency analysis [5]. However, the advantages of wavelet transform are mainly embodied in the analysis and processing of the one-dimensional piecewise smooth function or bounded variation function. When the wavelet transform was applied to two-dimensional or higher-dimensional domain, the one-dimensional separable wavelet which is spanned by the one-dimensional wavelet has limited direction; therefore, it cannot have optimal representation of high-dimensional function containing the line or surface singularity. Therefore, the wavelet transform can only reflect the signal point singularity, as far as the lines singularity and surface singularity contained in the linear and edge characteristics of the two-dimensional image is concerned, they are hard to be reflected with the wavelet transform approach. In the multi-resolution decomposition fusion algorithm, the adoptions of the wavelet transform fusion algorithms tend to introduce high-frequency noise, which would affect the quality of the fused image.

For the above-mentioned drawbacks of the wavelet transform, directionlets transform is anisotropic transform proposed by Velisavljevic et al. [6], which is based on integer lattice. The directionlets still use the one-dimensional filter group, but with the base function of multi-directional anisotropy, the directionlets have a detachable filter and critical structure, and are able to be fully reconstructed, thus, theoretically it has more advantages than the general wavelet transform, and the other second generation wavelet transform [6].

This article has applied directionlets transform to image fusion experiments. The test results show that it can blend the edges of the image information fairly well, and subjectively more in line with human visual characteristics and the objective evaluation is also superior to other image fusion method.

## 2. Directionlet transform

The directionlets transform proposed by Velisavljevic et al. [7–9] is the multi-directional anisotropy based upon the integer lattice. It adopts multi-directional anisotropy basis functions, therefore, it has more advantages in expressing the image than the average wavelet transform. At the same time, it only uses the one-dimensional filter banks with separable filtering and critical structures, and can be reconstructed totally, thus, as far as the computational complexity is concerned, it has more advantage than other second generation wavelet transform. The directionlets transform is a new multi-scale analysis tool.

When using one-dimensional filter banks to conduct multi-directional two-dimensional separable wavelet transform, we select any two rational slope *r*_{1} = *b*_{1}/*a*_{1} and *r*_{2} = *b*_{
2
}/*a*_{2}’s digital line direction to filtering and down-sampling; however, when the critical sampling is enhanced, two digital lines will have the issue of direction of mutual inductance, that is, along the slope *r*_{1} and *r*_{2}, the concept of the digital line cannot provide a systematic rule for the down-sampling of the repeated filtering and repeat sampling.

*r*

_{1}=

*b*

_{1}/

*a*

_{1}and

*r*

_{2}=

*b*

_{2}/

*a*

_{2}’s directions in grid space

*z*

^{2}, expressed in matrix as

*r*

_{1}of the vector

*r*

_{1}is the change of direction; the direction along the slope

*r*

_{2}of the vector

*d*

_{2}is called the queue direction. Along the skewed collinear transform of the transformation of the lattice in the queue application, it has

*n*

_{1}and

*n*

_{2}(

*n*

_{1}≠

*n*

_{2}) transformation in an iterative steps along the transform direction and queue direction, and it has been marked as

*S*-AWT(

**M**

_{ Λ },

*n*

_{1},

*n*

_{2}) (Anisotropic Wavelet Transform, AWT). From

**M**

_{ Λ }, the integer lattice Λ can be ascertained. According to the case theory,

*z*

^{2}has been divided into the | det

**M**

_{ Λ }|’s co-set which is about the entire integer lattice

*Λ*. The filtering and down-sampling have been conducted in every co-set, and then the remaining pixels belong to the lattice

*Λ*

^{′}of integer lattice

*Λ*, the matrix

The image which has gone through the above-mentioned directionlets transform has a very sparse coefficient, and then can obtain more directional information, which can be better used to describe the edge contour of the infrared image.

## 3. The infrared and visible image fusion algorithm based on directionlets

The article has introduced the directionlets into fusion of the infrared and visible image. The characters of the directionlets can be better used in extracting the geometric features of the source image and provide more information for the fused image. The advantage of applying the multi-scale directional analysis theory in image fusion is that the image can be decomposed into different scale and sub-band. Therefore, when the fusion is processing, the different scales and different directional sub-bands can adopt different fusion rules. The better fused effect can be achieved through sufficiently digging original multi-source image’s visual information.

The high-frequency sub-band after the directionlets decomposition contains a lot of high-frequency information of the image. The bigger coefficient’s absolute value corresponds to the certain directional interval’s significant characteristic, for example, marginal, linear, regional boundary, etc. The coefficients can better depict the image’s structure’s information, and have great influence upon human’s vision. The low-frequency sub-band contains most of low-frequency information of the image, and is the primary perception part of the human’s eyes to the image content. The article judges and processes the fusion according to the characteristics of the low-frequency sub-band and the high-frequency sub-band with corresponding fusion rules, and the sub-band coefficient would thereby be acquired.

According to the human visual system’s characteristics, we can know that the human’s eyes are not sensitive to the individual pixel’s gray value [10]. The distinctness of the image is decided by the all the pixels in certain region. To improve the fused image’s clearness, the pixel’s regional feature should be considered in the design of fusion algorithm. Therefore, the coefficient with the bigger regional variance value should be adopted as the fused image’s high-frequency sub-band coefficient in directionlets transform.

- 1
The directionlets decomposition is applied in the visible image

*V*and the infrared image*I*. The high-frequency sub-band*V*_{H}and*I*_{H}, and the low-frequency sub-band*V*_{L}and*I*_{L}. - 2The coefficient of the low-frequency sub-band remains constant. The article chooses two images’ average value of the coefficient of the low-frequency sub-band as the fused image’s low-frequency sub-band coefficient. Suppose: the low-frequency sub-band’s coefficient as
*F*_{L}, then${F}_{\mathrm{L}}=\frac{{V}_{\mathrm{L}}+{I}_{\mathrm{L}}}{2}$(2) - 3As for the high-frequency sub-band’s coefficient, the maximum principle of the local variance has been adopted, that is to say, in the transform domain, calculating the corresponding point
*N*×*N*neighborhood’s local variance*C*_{ X }(*X*as*V*or*I*), and choosing the highest coefficient of the variance as the fused image’s corresponding point’s coefficient.$\{\begin{array}{c}\hfill {\mathit{F}}_{\mathrm{L}}={V}_{\mathrm{L}},\phantom{\rule{1em}{0ex}}\left|{C}_{V}\right|\ge \left|{C}_{I}\right|\hfill \\ \hfill {F}_{\mathrm{L}}={I}_{\mathrm{L}},\phantom{\rule{2em}{0ex}}\left|{C}_{V}\right|<\left|{C}_{I}\right|\hfill \end{array}$(3) - 4
The directionlets inverse transformation has been applied to the fused image’s coefficient, and we get the fused image

*F*.

## 4. Experimental results and analysis

As can be seen from the figure, the images (c) and (d) have different degrees of blur, such as the marginal information of the runways and the outline of the aircraft is not clear, compared to (c) and (d), the image (e) is clearer as far as the visual effect is concerned. For example, the image (e) contours of aircraft and distant details such as trees and buildings look more clearly.

**Comparison of statistical parameters about fusion results according to different fusion rules**

Entropy | Cross entropy | Standard deviation | Average gradient | |
---|---|---|---|---|

LP | 10.670 | 0.605 | 12.711 | 26.307 |

DWT | 10.592 | 0.871 | 13.427 | 26.221 |

Directionlets | 10.623 | 0.550 | 13.972 | 27.932 |

## 5. Conclusions

The article puts forward an infrared and visible fusion algorithm based on the directionlets transform. Compared with the traditional wavelet transform, directionlets can preserve the infrared and visible image’s feature information, enhance the fused image’s space detail representation ability, and improve the fused image’s information. The experiment proves that the approach in this article is better than the wavelet transform and Laplace pyramid decomposition, etc.

## Declarations

### Acknowledgment

The authors are grateful to the anonymous referees for constructive comments. This study was funded by the Tianjin Normal University Doctoral Fund (52X09008, 52LX14).

## Authors’ Affiliations

## References

- Zhou X, Liu R-A, Chen J: Infrared and visible image fusion enhancement technology based on multi-scale directional analysis.
*IEEE Comput. Soc*2009, 1-3.Google Scholar - Hall DL, Linas J: An introduction to multisencor data fusion.
*Proc IEEE*1997, 85(10):6-23.View ArticleGoogle Scholar - Toet A, Ruyven LV, Velaton J: Merging thermal and visual images by a contrast pyramid.
*Opt. Eng.*1989, 28(7):789-792.View ArticleGoogle Scholar - Yonghong J: Fusion of landsat TM and SAR image based on principal component analysis.
*Remote Sens. Technol. Appl.*1998, 13(1):4649-4654.Google Scholar - Lin YC, Liu QH: An image fusion algorithm based on directionlet transform.
*Nanotechnol. Precision Eng.*2010, 8(6):565-568.Google Scholar - Velisavljevic V, Beferull-Lozano B, Vetterli M: Directionlets: anisotropic multi-directional representation with separable filtering.
*IEEE Trans. Image Process.*2006, 15(7):1916-1933.MathSciNetView ArticleGoogle Scholar - Velisavljevic V: Low-complexity iris coding and recognition based on directionlets.
*IEEE Trans. Inf. Forens. Secur.*2009, 4(3):410-417.View ArticleGoogle Scholar - Velisavljevic V, Beferull-Lozano B, Vetterli M: Space-frequency quantization for image compression with directionlets.
*IEEE Trans. Image Process.*2007, 16(7):1761-1773.MathSciNetView ArticleGoogle Scholar - Velisavljevic V, Beferull-Lozano B, Vetterli M: Efficient image compression using directionlets, in.
*6th International Conference on Information, Communications & Signal Processing*2007, 1-5.Google Scholar - Yang L: B-l Guo, W Ni.
*Multifocus image fusion algorithm based on region statistics in contourlet domain. J. Xi'an Jiaotong Univ.*2007, 41(4):448-452.Google Scholar

## Copyright

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.