A hierarchical propelled fusion strategy for SAR automatic target recognition

Abstract

Synthetic aperture radar (SAR) automatic target recognition (ATR) is playing a very important role in military and civil field. Much work has been done to improve the performance of SAR ATR systems. It is well-known that ensemble methods can be used for improving prediction performance. Thus recognition using multiple classifiers fusion (MCF) has become a research hotspot in SAR ATR. Most current researchers focus on the fusion methods by parallel structure. However, such parallel structure has some disadvantages, such as large time consumption, features attribution conflict and low capability on confuser recognition. A hierarchical propelled strategy for multi-classifier fusion (HPSMCF) is proposed in this paper. The proposed HPSMCF has the characters both of series and parallel structure. Features can be used more effective and the recognition efficiency can be improved by extracting features and fusing the probabilistic outputs in a hierarchical propelled way. Meanwhile, the confuser recognition can be achieved by setting thresholds for the confidence in each level. Experiments on MSTAR public data demonstrate that the proposed HPSMCF is robust for variant recognition conditions. Compared with the parallel structure, HPSMCF has better performance both on time consumption and recognition rate.

1. Introduction

SAR is playing an important role both in national defense and civil applications, because SAR can work in all weather and day/night conditions. For better taking advantage of SAR data, the problem of target recognition should be overcome. In 1997–2000, L.M. Novak publishes a number of articles on SAR ATR [1–3]. After that many researchers have done lots of work to improve the performance of SAR ATR systems.

It is well-known that ensemble methods can be used for improving prediction performance [4]. The main idea behind the ensemble methodology is to combine several individual classifiers in order to obtain a classifier that outperforms every one of them. In 1998, J. Kittler .etc. develop a common theoretical framework for combining classifiers [5]. In 2010, Lior Rokach proposes an ensemble system which is composed of several independent base-level models [4]. These base-level classifiers are respectively constructed using different techniques and methods. With the development of MCF technology, MCF has been extensively applied in many areas, such as character recognition [6], multi-sensor data classification [7] and SAR ATR [8, 9].

It can be seen that MCF has become a research hotspot. Thus many new theories in pattern recognition are gradually introduced to MCF. For example, Jorge Sánchez and Javier Redolfi propose a novel approach for the combination of classifiers based on a graph defined in the space of concepts and a Markov chain defined on that graph [10]. Some researchers consider the contributions of various types of feature sets and classifiers, so weights are given to features or classifiers [11–13]. It is believed that confidence-weighted learning is more consistent with human’s prediction model. In [14], authors use clustering ensemble for classifiers combination, because the use of cluster analysis techniques in supervised classification tasks has shown that they can enhance the quality of the classification results.

All the methods above belong to parallel structure of MCF (PSMCF) [5]. The flow of PSMCF is shown in Figure 1. PSMCF can indeed improve the decision accuracy by complementing the disadvantages between different classifiers. However, such parallel structure has some disadvantages. Firstly, if the decision-making strategy is not suitable, the accuracy will be reduced. Secondly, all the features should be extracted in a parallel structure, so large time-consuming is needed. Thirdly, the parallel structure can only make a decision on the final fusion result, so it has low capability on confuser recognition.

Figure 1

The common flow of multiple classifiers fusion.

In order to overcome such disadvantages of the PSMCF, a hierarchical propelled strategy for multi-classifier fusion (HPSMCF) is proposed in this paper. HPSMCF has the characters both of series and parallel structure. Features can be used more effective and the recognition efficiency can be improved by extracting features and fusing the probabilistic outputs in a hierarchical propelled way. Determining whether to go to the next level depends on the comparison between confidence and empirical threshold, so HPSMCF can reduce the time consumption without lower the recognition rate. Also the confuser recognition can be achieved by setting thresholds for making decisions in each level. Experiments on MSTAR public data demonstrate that the proposed HPSMCF gets better performance than PSMCF when applied to SAR ATR.

The rest of the paper is organized as follows. Section 2 introduces the flow of the proposed fusion strategy, the definition of classification confidence and classification weight in each level. In section 3, the detail description of three levels HPSMCF is given. Then Section 4 presents experimental results and analysis on MSTAR database. Finally in section 5 the conclusion and future work are stated.

2. Hierarchical propelled fusion strategy

If there are L levels, the basic framework of HPSMCF is shown in Figure 2.

Figure 2

The basic flow chart of hierarchical propelled fusion strategy.

If there are c classes samples, the classifier in level l (1 ≤ l ≤ L) can get posterior probability output of each class shown by P^l = {p₁, p₂, ...p _c }.

1) Classification confidence

The classification confidence can be defined as:

$$\begin{array}{l}\mathit{\text{conf}}\left(P^l\right)=\frac{\underset{c}{\text{max}}\left(p_i\right)-\text{max}\left\{P^l\backslash \underset{c}{\text{max}}\left(p_i\right)\right\}}{\underset{c}{\text{max}}\left(p_i\right)}\\ s.t.{\displaystyle \sum _{i=1}^c{p}_i}=1\end{array}$$

(1)

$\left\{P^l\backslash \underset{c}{\text{max}}\left(p_i\right)\right\}$ means the set P^l except $\underset{c}{\text{max}}\left(p_i\right)$. If conf (P^l) is greater than a determined value, it means that the class represented by $\underset{c}{\text{max}}\left(p_i\right)$ is acceptable.

In HPSMCF, if the confidence in each level is bigger than the empirical threshold T, the process is ended and the system can output the predicted class of the unknown image. If not, the process will go to the next level. A new feature is extracted and fed to another classifier. It is worth to remind that the confidences in each level are different because different features are used. All the thresholds can be written as:

$$T=\left\{T_1,\left(T_2,T_2^{*}\right),..\left(T_L,T_L^{*}\right)\right\}$$

In level l (l > 1), there are two thresholds (T _l , T _l ^*). T _l indicates the threshold in level l before fusion, and T _l ^* is used for the fusion result.

2) Weights of P^l

If in level l, the confidence of the probability output is less than the empirical threshold, it means that this classification result is probably wrong. So in this level, the classification is not given and the process will go to next level. However, the probability output in level l P^lwill be fused in the following levels with a weight. The weight of P^l can be expressed as:

$$w_{\mathit{\text{lj}}}\left(1\le l\le L,2\le j\le L,l\le j\right)$$

Where j indicates the level in which P^l is fused. Weights used for fusion in each level are shown in Table 1.

Table 1 The confused matrix of weights

The weight w _lj has a character as follows:

$$w_{\mathit{\text{lj}}}=\{\begin{array}{l}1l=j\\ \mu w_{l\left(j-1\right)}l<j\end{array}$$

(2)

where 0 < μ < 1. It means that if the probability output P^l cannot give enough evidence to get the classification result in level j-1, the weight of P^l should be reduced when P^l is used in level j and the reduced coefficient is μ.

3) Basic flow

The detailed process at each level is described as follows:

In level 1, extracting the first feature, and feeding the feature to classifier. Computing the confidence of P¹. If conf(P¹) > T₁, output the classification result; otherwise the flow goes to level 2;

In level 2, extracting the second feature, and feeding the feature to classifier. Computing the confidence of P². If conf(P²) > T₂, output the classification result; otherwise fusion w₁₂P¹ and w₂₂P². Computing the confidence of the fusion result. If conf(fusion(w₁₂P¹, w₂₂P²)) > $T_2^{*}$, output the classification result; else go to next level;

The processes in the following levels are the same as level 2, except in level L. The last level L makes the final decision. If in level L, the classification result cannot be got, all the process will be ended, and output ‘cannot recognition’.

It can be seen that if the threshold T _L ^* is set to 0 and all of the other thresholds are set to 1, our HPSMCF is simplified as PSMCF; if the fusion processes are cancelled, HPSMCF is simplified as series structure.

3. Application of the proposed three-tier HPSMCF to SAR ATR

In order to verify the feasibility of the strategy proposed in this study, three-tier HPSMCF is applied to SAR images automatic target recognition. This work includes three important parts: Feature Extracting, Classification, and Decision Fusion.

1) Feature extracting

In our current research, three projection features are used. They are Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), and Non-negative Matrix Factor (NMF).

A. PCA

PCA is a common feature extraction method in pattern recognition, which has been widely used for target classification in SAR images. PCA is based on the assumption that high information corresponds to high variance.

If X = {x₁, x₂, ...x _n }, X ∊ ℜ^m×n is the original data, the mean of X is:

$$\overline{X}=\frac{1}{n}{\displaystyle \sum _{i=1}^n{X}_i}$$

(3)

The covariance matrix of X:

$$Q={\displaystyle \sum _{i=1}^n{\left(X_i-\overline{X}\right)}^T\left(X_i-\overline{X}\right)}$$

(4)

Eigenvalues and eigenvectors of Q:

$$\left[V,D\right]=\mathit{\text{EIG}}\left(Q\right)$$

(5)

Function EIG can get all the eigenvalues and eigenvectors of Q. It is proved that the first few principal components account for most of the variation. They can be used to describe the data, leading to a reduced-dimension representation. Choose the eigenvectors correspond to the k largest positive eigenvalues to form the transformation matrix. Then the PCA feature of X is extracted by multiplying the transformation matrix. The detailed information about PCA is introduced in reference [15, 16].

B. LDA

LDA uses the discrimination information between different classes, so the class information of samples should be known [15, 17]. Denote N sampled images as X = {x₁, x₂, ...x _N }, the within-class scatter matrix is defined as:

$$S_w={\displaystyle \sum _{j=1}^c{\displaystyle \sum _{i=1}^{N_j}\left(x_i^j-{\mu }_j\right){\left(x_i^j-{\mu }_j\right)}^T}}$$

(6)

Where x _i ^j is the i th sample of class j, μ _j is the mean of class j, c is the number of classes, and N _j is the number of samples in class j.

The between-class scatter matrix is defined as:

$$S_b={\displaystyle \sum _{j=1}^c\left({\mu }_j-\mu \right){\left({\mu }_j-\mu \right)}^T}$$

(7)

where μ represents the mean of all classes.

The goal of LDA is to maximize the between-class measure while minimizing the within-class measure. One way to do it is to maximize the ratio det |S _b |/ det |S _w |. If S _w is a nonsingular matrix, when the column vectors of the projection matrix W = PCA(S _w ⁻¹S _b ), the ratio is maximized. It should be noted that the number of eigenvectors of W is at most c-1.

C. NMF

NMF has been successfully used for matrix decomposition and dimensionality reduction. The non-negativity constraint leads to a part-based representation because it allows only additive, not subtractive, combinations of the original data [18, 19].

Assume matrix $D\in {\mathbb{R}}^{n\times m}$ is decomposed into two matrices $W\in {\mathbb{R}}^{n\times r}$ and $H\in {\mathbb{R}}^{r\times m}$, so that:

$$\mathrm{D}\approx {\mathrm{WH}\mathrm{D}}_{i,j},W_{i,\mu },H_{\mu ,j}\ge 0$$

(8)

with 0 ≤ i < n − 1, 0 ≤ j < m − 1 and 0 ≤ μ < r − 1.

Define cost function based on the Square Euclidian Distance:

$$\underset{W,H}{\text{arg}\text{min}}{‖,D-\mathit{\text{WH}}‖}^2={\displaystyle \sum _{0\le i<n,0\le j<m}{\left(D_{i,j}-{\left(\mathit{\text{WH}}\right)}_{i,j}\right)}^2}$$

(9)

The Square Eculidian Distance measure in (9) is non-increasing under the following iterative update rules:

$$\begin{array}{l}{H}_{\mathit{a\mu }}\leftarrow H_{\mathit{a\mu }}\frac{{\left(W^{T}D\right)}_{\mathit{a\mu }}}{{\left(W^T\mathit{\text{WH}}\right)}_{\mathit{a\mu }}}\\ W_{\mathit{ia}}\leftarrow W_{\mathit{ia}}\frac{{\left(D{H}^T\right)}_{\mathit{ia}}}{{\left(\mathit{\text{WH}}{H}^T\right)}_{\mathit{ia}}}\end{array}$$

(10)

for 0 ≤ a < r, 0 ≤ μ < m and 0 ≤ i < n. Appropriate W, H can be found by iteration.

D. Feature ordering

In our current research, the principle of feature ordering is the computational complexity for extracting each feature. The computational complexity of PCA is the smallest, and the computational complexity of NMF is the largest. So in the three-tier HPSMCF, PCA is used in level 1, LDA is used in level 2 and NMF is used in level 3.

2) Classifier

In order to make the metric of probability output same in each level, Support Vector Machine (SVM) is used in all levels. SVM classification method has extraordinary potential capacity. Using kernel function, SVM can well solve the non-linear classification problem [6, 20].

SVM discriminates two classes by fitting an optimal linear separating hyperplane (OSH). The optimization principle is based on structural risk minimization (SRM). SRM aims to maximize the margins between the OSH and the closest training samples. These closest training samples are called support vectors. The details of solving the optimization problem in SVM are introduced in [21].

For a test sample, SVM can get the probability output in the following form:

$$P=\left\{p_1,p_2,\dots p_i,\dots p_c\right\}$$

where 0 < j ≤ c, c is the number of class. P _j is the probability that the test sample belongs to class j.

The kernel of SVM used in this paper is radial basis function (RBF):

$$k\left(x,y\right)=exp\left(-\gamma {‖x-y‖}^2\right)$$

and the parameters of C and y are set to C = 32, γ = 1/32.

3) Fusion theory

Dempster-Shafer Evidence Theory is close to human decision principle and has been widely used in information fusion and classifier fusion [22, 23].

Assuming Θ is a mutually exclusive and exhaustive finite set, which is called frame of discernment. The mapping m : 2^Θ → [0, 1] called Basic Probability Assignment Function (BPAF) is defined as:

$$\{\begin{array}{l}m\left(\Theta \right)=0\\ {\displaystyle \sum _{A\in \Theta }m\left(A\right)=1}\end{array}$$

(11)

If A ⊆ Θ and m(A) > 0, A is called a Focal Element.

If Θ has n Focal Elements: A = {A₁, A₂, …A _i , …A _n }, Dempster’s rule of combination is described as follows:

$$\{\begin{array}{l}m\left(\Theta \right)=0\\ m\left(A\right)=\frac{{\displaystyle \sum _{\cap A_i=A}{\displaystyle \prod _{i=1}^n{m}_i\left(A_i\right)}}}{1-{\displaystyle \sum _{\cap A_i=\Theta }{\displaystyle \prod _{i=1}^n{m}_i\left(A_i\right)}}}=\frac{{\displaystyle \sum _{\cap A_i=A}{\displaystyle \prod _{i=1}^n{m}_i\left(A_i\right)}}}{1-k}\end{array}$$

(12)

where k is a measure of conflict between n evidences and can be got by

$$\begin{array}{l}K={\displaystyle \sum _{A_1\cap \dots \cap A_n\ne \varnothing }{m}_1\left(A_1\right)}.m_2\left(A_2\right)\dots m_n\left(A_n\right)\\ =1-{\displaystyle \sum _{A_1\cap \dots \cap A_n=\varnothing }{m}_1\left(A_1\right)}.m_2\left(A_2\right)\dots m_n\left(A_n\right)\end{array}$$

(13)

In level l, there are l evidences need to be fused. So in the second and third level of our three-tier HPSMCF, the value of n in (13) is 2 and 3, respectively.

4. Experiment analysis

The Moving and Stationary Target Acquisition and Recognition (MSTAR) program was initiated by the U.S. Defense Advanced Research Projects Agency (DARPA) and the U.S. Air Force Research Laboratory (AFRL) in the summer of 1995. The SAR images used in our experiments are taken from MSTAR public release database. The database consists of X-band SAR images with 1 ft. × ft. resolution for multiple targets. The SAR target images were captured at two different depression angles 15° and 17° over 360° aspect angles [24]. Figure 3 shows some sample images with both optical images and SAR images. The statistics of the MSTAR public database is summarized in Table 2.

Figure 3

Some samples with both optical images and SAR images in MSTAR. (a)-(b) Optical images for BMP2, BTR70, T72, 2S1, D7, ZIL131 and ZSU23-4. (c)-(d) Corresponding SAR images for 7 targets.

Table 2 Part of MSTAR public database

The format of the data is RAW including amplitude and phase information. In the following experiments, only amplitude information is used. All the images are cropped by extracting 64×64 patches from the center of the image but without any other preprocessing.

(1) 3-Class recognition and confuser recognition

1) 3-Class recognition

In this experiment, 3 classes targets (BMP2, BTR70, T72) are used. BMP2 and T72 have three series as shown in Table 2. Only the images of BMP2 sn-c9563, BTR70 and T72 sn-132 at depression 17° are used for training data. All of the images of these three classes at depression 15° are used as the testing data.

Table 3 is the confusion matrix of the classification results by HPSMCF. The overall recognition rate (RR), especially the BMP2, is not as high as the state-of-the-art recognition rate. This is caused by the images without any preprocessing. Figure 4 shows the comparison between PCA+SVM, LDA+SVM, NMF+SVM, PSMCF and HPSMCF. It can be clearly seen that our HPSMCF has the best performance on this 3-class classification problem.

Table 3 The confusion matrix of HPSMCF performance

Figure 4

Recognition performance of different methods: PCA+SVM, LDA+SVM, NMF+SVM, PSMCF and HPSMCF.

2) Confuser recognition

In this experiment four nontarget vehicles (2S1, D7, ZIL131, and ZSU23-4), are added to the testing set in Table 3 as confusers.

The rejection rates are listed in Table 4. Our HPSMCF gets the average rejection rate 70.2% while PSMCF rejects 52.4% of confusers. It can be seen that no matter 3-class recognition or confuser recognition, our HPSMCF has better performance.

Table 4 Confuser rejection rate

(2) Depression angle and configuration variance

In order to test our designed method’s robustness of processing different condition data, the data in 3-class problem is divided into one training set and three testing sets. The images of BMP2-c21, BTR70-c71, and T72-132 in depression angles of 17° are employed as training set. The other images are used for testing sets. The detail of the data sets is shown in Table 5. This new mode of partition the data sets can get variance in depression angle and configuration.

Table 5 New division of 3-class data

The recognition results of five methods on three testing sets are presented in Table 6 and Figure 5. They clearly show that tests on set 2 have the highest recognition rate and tests on set 4 have the lowest recognition rate. The recognition rate of set 4 is the lowest, which means both target type and depression angle can have effect on the recognition performance. However, the recognition rate of set 2 is higher than set 3, which means the variance of target configuration in set 2 has more effect on recognition than the variance of depression angle in set 3.

Table 6 Recognition results on new division

Figure 5

The recognition results on different test set.

Meanwhile, Table 6 and Figure 5 also show that the PSMCF indeed outperforms the method which uses single feature, but the proposed hierarchy framework HPSMCF in this paper has the best results.

The number of levels which a process reaches is called hierarchy depth. The hierarchy depth of three testing sets is presented in Table 7. For most samples of set 2, only the first level is used for recognition. Just a few recognition processes can reach the third hierarchy. However, the number of recognition processes which use all three levels has obvious growth when recognizing the samples of set 3 and set 4, which means that set 3 and set 4 have more difficult recognition condition than set 2. Therefore our system can choose different hierarchy depth according to the complexity of recognition processes.

Table 7 The hierarchy depth of three testing sets

The comparison on average recognition rate and time consumption with PSMCF when testing set 2~set 4 is shown in Table 8. It proves that the proposed method outperforms PSMCF both on recognition rate and time consumption.

Table 8 Comparison with on average RR and used time

5. Conclusion and future work

In order to overcome the disadvantages of the common fusion method by parallel structure, a hierarchical propelled strategy of multiple classifiers fusion (HPSMCF) is proposed in this paper. The recognition efficiency can be improved by extracting features and fusing the probabilistic outputs in the hierarchical propelled way. Also the confuser recognition can be achieved by computing confidence and making decisions in each level. Experiments on MSTAR public data set demonstrate the effectiveness of the proposed hierarchical propelled fusion strategy. Compared to the single classifier based recognition processes, HPSMCF has higher recognition rate. Meanwhile, the proposed method outperforms the traditional parallel structure on both time consumption and recognition rate.

The next step in our research work will consist in selecting the threshold T adaptively and using more features and more classifiers to evaluate the feasibility of this system. On these bases, our goal is to build a recognition framework based on human cognition theory. Meanwhile, the proposed strategy can also be considered for the multiple sensors fusion, etc.