- Open Access
Compressive sensing image fusion algorithm based on directionlets
© Zhou et al.; licensee Springer. 2014
- Received: 30 December 2013
- Accepted: 24 January 2014
- Published: 1 February 2014
This paper presented a new image fusion based on compressed sensing (CS). The method decomposes two or more original images using directionlet transform, gets the sparse matrix by the directionlet coefficient sparse representation, and fuses the sparse matrices with the coefficient absolute value maximum scheme. The compressed sample can be obtained through random observation. The fused image is recovered from the reduced samples by solving the optimization. The study demonstrates that the compressive sensing image fusion algorithm based on directionlets has a number of perceived advantages. The simulations show that the proposed algorithm has the advantages of simple structure and easy implementation and also can achieve a better fusion performance.
- Image fusion
- Compressive sensing
- Directionlet transform
The main goal of image fusion is to extract all the important features from all input images and integrate them to form a fused image which is more informative and suitable for human visual perception or computer processing.
There are a number of pixel-level image fusion methods, including the weighted average method [1, 2], the pyramid transform method , principal component analysis (PCA) method , as well as fusion based on wavelet transform methods. The wavelet transform has become an important tool in image fusion method for its excellent feature of time-frequency analysis . However, wavelet bases are isotropic and of limited directions and fail to represent high anisotropic edges and contours in images well. For the drawbacks of the wavelet transform, directionlet transform is an anisotropic transform proposed by Vladan, which is based on the integer lattice. The directionlet still uses the one-dimensional filter group, but with the base function of multi-directional anisotropy, the directionlet has a detachable filter and critical structure and is able to be fully reconstructed. Thus, theoretically, it has more advantages than the general wavelet transform and the other second-generation wavelet transform .
In recent years, inspired by the ideas of 'sparse’ approximation, a novel theory called compressed sensing (CS) has been developed [7–9]. The CS principle claims that if a signal is compressive or sparse in a certain transform domain, it can be projected onto a low-dimensional space using a measurement matrix which is irrelevant with transform basis while still enabling reconstruction at high probability from these small numbers of random linear measurements via solving an optimization problem. Therefore, it is expected to provide a new idea for image fusion by combined directionlets with CS.
This article proposes a new scheme for image fusion; in our scheme, directionlet transform first decomposes each source image into two components, i.e., dense and sparse components. Then, the dense components are fused by the selection method according to the manifestations of defocus, while the sparse components are fused under the frame of CS via fusing a few linear measurements by solving the problem of l1 norm minimization which is based on the two-step iterative shrinkage reconstruction algorithm. The proposed fusion scheme is applied to infrared and visible image fusion experiments, and the performance is evaluated in terms of computational efficiency, visual quality, and quantitative criterion.
The test results show that it can blend the edges of the image information fairly well, and subjectively more in line with human visual characteristics and the objective evaluation is also superior to other image fusion methods.
The directionlet transform proposed by the German researcher Vladan is the multi-directional anisotropy based upon the integer lattice [10–12]. It adopts multi-directional anisotropy basis functions; therefore, it has more advantages in expressing the image than the average wavelet transform. At the same time, it only uses the one-dimensional filter banks with separable filtering and critical structures and can be reconstructed totally; thus, as far as the computational complexity is concerned, it has more advantage than the other second-generation wavelet transform. The directionlet transform is a new multi-scale analysis tool.
When using one-dimensional filter banks to conduct multi-directional two-dimensional separable wavelet transform, we select any two rational slope r1 = b1/a1 and r2 = b2/a2's digital line direction to filtering and down-sampling; however, when the critical sampling is enhanced, two digital lines will have the issue of direction of mutual inductance, that is, along the slope r1 and r2, the concept of the digital line cannot provide a systematic rule for the down-sampling of the repeated filtering and repeated sampling.
The image which has gone through the above mentioned directionlet transform has a very sparse coefficient and then can obtain more directional information, which can be better used to describe the edge contour of the infrared image.
Compressive sensing enables a sparse or compressible signal to be reconstructed from a small number of non-adaptive linear projections, thus significantly reducing the sampling and computation costs . CS has many promising applications in signal acquisition, compression, and medical imaging. In this paper, we investigate its potential application in the image fusion.
where ψ is the N × N matrix and s is the coefficient component column vector of dimension N × 1.
The advantage that the CS theory has is that the data obtained via the projection measurement is much smaller than the conventional sampling methods, breaking the bottleneck of the Shannon sampling theorem, so that the high-resolution signal acquisition becomes possible. The attraction of CS theory is that it is for applications in many fields of science and engineering and has important implications and practical significance, such as statistics, information theory, coding theory, computer science theory, and other theories.
Compared with the traditional fusion algorithms, the CS-based image fusion algorithm theory has shown significant superiority: the image fusion can be conducted in the non-sampling condition of the image with the CS technique, the quality of image fusion can be improved by increasing the number of measurements, and this algorithm can save storage space and reduce the computational complexity. The main ideas of the CS-based image fusion algorithm theory are as follows: first of all, the two images which need to deal with should undergo the directionlet transform, the sparse matrix can be obtained after the directionlet coefficients are processed with the sparse treatment, then the fusion rules for the sparse matrix integration are determined, compressive sampling through random sampling matrix is obtained, and finally, the fused image can be obtained in the best optimized way.
The practical function of wavelet transform is the signal decorrelation, and all the information of the signal are concentrated into the wavelet coefficients with large amplitude. These large wavelet coefficients contain far more energy than that contained in small coefficient so that in the reconstruction of the signal, a large coefficient is more important than the smaller one.
where D f is the fusion wavelet coefficient, D M is the wavelet coefficient whose absolute value is the largest of the wavelet coefficients in the same location in different images, and I is the number of the source image.
The directionlet transform is used to deal with the source image; directionlet coefficients are obtained with the sparse treatment: the small coefficient (or coefficient of close to zero) is set to zero to obtain an approximate sparse coefficient matrix.
When the source image is conducted via sparse transformation, the wavelet is used as the sparse basis. To reconstruct the image with less measurement value, we must ensure that the sparse basis ψ and the measurement matrix ϕ are irrelevant, because any random sparse matrix has superiority that it is irrelevant to any sparse basis. That is the reason why it can be used as a measurement measure matrix.
For each m × n pixel image, conduct directionlet transform to obtain the directionlet coefficient matrix.
The directionlet coefficients are processed with the sparse treatment and then fused according to the larger absolute value rule.
For the fused directionlet coefficients, the random matrix is selected as the measurement matrix ϕ; after the measurement, the measured value y can be obtained.
By solving the linear programming of the l 1 norm, the approximate solution can be acquired.
Conduct the inverse transform to the obtained directionlet coefficients and thus the fusion image can be acquired.
Comparison of statistical parameters about fusion results according to different fusion rules
As can be seen from Figure 3, the mutual information values of the compressive sensing image fusion algorithm are the best among the three fusion methods.
The paper put forward a fusion algorithm based on the compressed sensing. Compared with the traditional wavelet transform, the proposed CS-based image fusion algorithm can preserve the image feature information, enhance the fused image space detail representation ability, and improve the fused image information. The experiment proves that the approach in this paper is better than the wavelet transform and Laplace pyramid decomposition, etc.
The authors are grateful to the anonymous referees for the constructive comments.
- Zhou X, Liu R-A, Chen J: Infrared and visible image fusion enhancement technology based on multi-scale directional analysis. IEEE Computer Society. Piscataway: IEEE; 2009.Google Scholar
- Hall DL, Linas J: An introduction to multisensor data fusion. Proceedings of the IEEE 1997, 85(10):6-23.View ArticleGoogle Scholar
- Toet A, van Ruyven LJ, Velaton JM: Merging thermal and visual images by a contrast pyramid. Optical Engineering 1989, 28(7):789-792.View ArticleGoogle Scholar
- Yonghong J: Fusion of landsat TM and SAR images based on principal component analysis. Remote Sensing Technology and Application 1998, 13(1):4649-4654.Google Scholar
- Lin YC, Liu QH: An image fusion algorithm based on directionlet transform. Nanotechnology and Precision Engineering 2010, 8(6):565-568.Google Scholar
- Velisavljevic V, Beferull-Lozano B, Vetterli M: Directionlets: anisotropic multi-directional representation with separable filtering. IEEE Transactions on Image Processing 2006, 15(7):1916-1933.MathSciNetView ArticleGoogle Scholar
- Jin Wei F, Ran-di YM: Multi-focus fusion using dual-tree contourlet and compressed sensing. Opto-Electronic Engineering 2011, 38(4):87-94.Google Scholar
- Candes E, Wakin MB: An introduction to compressive sampling. IEEE Signal Processing Magazine 2008, 48(4):21-30.View ArticleGoogle Scholar
- Provost F, Lesage F: The application of compressed sensing for photo-acoustic tomography. IEEE Trans. on Med. Imaging 2009, 28(4):585-594.View ArticleGoogle Scholar
- Velisavljevic V: Low-complexity iris coding and recognition based on directionlets. IEEE Transactions on Information Forensics and Security 2009, 4(3):410-417.View ArticleGoogle Scholar
- Velisavljevic V, Beferull-Lozano B, Vetterli M: Space-frequency quantization for image compression with directionlets. IEEE Transactions on Image Processing 2007, 16(7):1761-1773.MathSciNetView ArticleGoogle Scholar
- Velisavljevic V, Beferull-Lozano B, Vetterli M: Efficient image compression using directionlets. 2007 6th International Conference Information, Communications & Signal Processing. Piscataway: IEEE; 2007:1-5.Google Scholar
- Wan T, Canagarajah N, Achim A: Compressive image fusion. IEEE International Conference on Image Processing. Piscataway: IEEE; 2008:1308-11.Google Scholar
- Huang XS, Dai QF, Cao YQ: Compressive sensing image fusion algorithm based on wavelet sparse basis. Application Research of Computers 2012, 29(9):3581-3583.Google Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.