Cyclostationarity-based DOA estimation algorithms for coherent signals in impulsive noise environments

Abstract

Estimating direction of arrival (DOA) is important in a variety of practical applications. Conventional cyclostationarity-based coherent DOA estimation algorithms are not robust to non-Gaussian α-stable impulsive noise. Additionally, fractional lower-order statistics (FLOS)-based algorithms are tolerant to impulsive noise; however, they experience performance degradation for coherent signals and interference. To overcome these drawbacks, two types of fractional lower-order cyclostationarity-based subspace DOA estimation methods are proposed for coherent signals in the presence of interference and α-stable impulsive noise. The new proposed algorithms exploit the fractional lower-order cyclostationarity properties of the signals and are immune to the impulsive noise and interference. Moreover, they can provide more accurate DOA estimates of coherent signals than conventional cyclostationarity-based and FLOS-based methods. The simulation results illustrate the robustness and effectiveness of the proposed methods for coherent signals based on a comparison with traditional methods. The new algorithms can be used in the presence of a wide range of interference, Gaussian noise, and α-stable distribution impulsive noise environments.

1 Introduction

The direction of arrival (DOA) is the base problem in array signal processing and is the core of many civilian and military applications, such as communication regulation enforcement, search-and-rescue operations, and military reconnaissance [1,2,3]. In realistic radio environments, correlated signals are not negligible. Evans and Shane et al. proposed derived spatial smoothing techniques for the DOA estimation of coherent signals; in the study, a uniform linear array was divided into several subarrays, and the covariance matrices of the subarrays were then calculated and averaged together [4]. Additionally, subspace smoothing, modified smoothing, and other smoothing methods have been proposed [5,6,7,8]. A new formulation of the Khatri-Rao has been used to estimate DOA which can cope with more coherent signals than classical multiple signal classification (MUSIC) with the spatial smoothing [9]. The unitary estimation of signal parameters via rotational invariance techniques (U-ESPRIT) incorporates forward-backward averaging that can lead to higher resolution than classical ESPRIT [10,11,12]. Since the performance of subspace-based algorithms is limited by the number of snapshots, the sparse reconstruction-based methods have been proposed to deal with the DOA estimation of coherent signals [13,14,15,16].

Many artificial modulated communications signals and nature signals have the cyclostationarity properties, which can break through previous limitations and further improve the DOA estimation performance of subspace algorithms. If received signals can be divided into the signal of interest (SOI) and interference (including noise) based on their cyclostationary characteristics, the signals of interest can be selected and the noise and interfering signals can be removed by utilizing cyclostationarity [17]. Therefore, there are many advantages of DOA estimation using cyclostationary characteristics compared to conventional methods [18, 19]. These advantages include selective direction finding, interference and noise suppression, and breakthrough-limited multi-signal processing. Therefore, DOA estimation based on cyclostationarity has been widely studied [20,21,22,23].

Noise, interference, and coherent signals can affect the performance of DOA estimation in wireless communication, radar, sonar, and other systems. The noise and interference can affect the estimation accuracy, and the coherent and correlated signals caused by multipath propagation can, in turn, cause the correlation function to produce several peaks and widen the main peak, which may lead to erroneous estimations by the corresponding algorithms [24]. However, most of the traditional cyclostationarity-based DOA methods for coherent signals assume that the additive noise obeys a Gaussian distribution. Many theoretical studies and experiments have shown that underwater acoustic noise, atmospheric noise, artificial noise, and other noise types contain numerous impulsive components. Previous studies [25,26,27] demonstrated that it is inappropriate to model these noises as Gaussian. In fact, the phenomena with impulsive components encountered in practice are appropriate to be modeled as non-Gaussian processes. Although cyclostationarity-based methods are tolerant to coherent signals and Gaussian noise, a significant degradation in their performance can occur when non-Gaussian impulsive noise is present. To suppress the impulsive components, robust fractional lower-order statistics (FLOS)-based DOA estimation methods have been developed [26,27,28,29]. The FLOS-based algorithms can provide accurate DOA estimates for Gaussian and non-Gaussian impulsive noises; however, they suffer from poor performance when interfering signal is present which occupies the same spectral band as the source signal. Recent advances on communications, array processing, and identification have indicated that non-Gaussian impulsive noise can be modeled by α-stable distributions [29, 30]. According to the Generalized Central Limit Theorem, the Gaussian distribution is the limiting case of the α-stable distribution (α = 2). Therefore, the α-stable distribution is a more realistic class of distribution than the Gaussian distribution in communication, radar, sonar, and similar application. In real applications, we are interested in developing DOA algorithms that are robust to interference, Gaussian noise, and non-Gaussian α-stable distribution noise and that account for the coherent signals in real-world environments.

In this paper, we address the issue of the DOA estimation of cyclostationary coherent signals in impulsive noise environments. Two novel types of robust fractional lower-order cyclostationarity-based signal-selective subspace DOA estimation algorithms for coherent signals are developed. Compared with conventional cyclic subspace DOA estimation algorithms, the proposed methods are not only immune to the interference and Gaussian noise, but can account for impulsive noise and coherent signals. Moreover, because the proposed methods are applicable to coherent signals in the presence of impulsive noise and interference, they are superior to the methods that are merely robust against impulsive noise or suitable for coherent sources. The proposed DOA estimation algorithms have three notable advantages over traditional methods: (1) in the presence of interfering signals, the proposed methods exhibit signal selectivity and performance improvements over the FLOS-based MUSIC and ESPRIT algorithms; (2) in impulsive environments, the proposed methods are more robust to impulsive noise than the cyclic MUSIC and cyclic ESPRIT algorithms; and (3) the proposed methods are effective for cyclostationary coherent signals in impulsive noise and interference environments, while conventional coherent cyclic methods are limited to DOA estimation from Gaussian noise, and FLOS-based coherent methods exhibit degradation in the presence of interference.

2 Methods

DOA estimation has been widely used in various important applications (e.g., radar, sonar, communications, and wireless sensor networks) and has garnered considerable attention in recent years. The conventional subspace DOA estimation methods are not robust against non-Gaussian α-stable impulsive noise. Although FLOS-based algorithms are robust to the impulsive noise, they do not achieve satisfactory performance in the presence of coband interference. Conventional cyclostationarity-based methods are highly immune to interference; however, they are severely degraded in the presence of α-stable impulsive noise and useless for coherent signals. Therefore, we address the problem of DOA estimation for coherent signals in the presence of impulsive noise and interference based on cyclostationarity.

First, we briefly introduce the signal model, cyclic statistics, and α-stable distribution. Based on these concepts, we formulate a fractional lower-order cyclostationarity. Since the conventional cyclostationary-based subspace DOA estimation methods are not robust against impulsive noise and coherent signals, we propose a type of spatial smoothing fractional lower-order cyclic algorithm. The proposed spatial smoothing fractional lower-order cyclic algorithms use pth-order cyclic statistics to exploit cyclostationarity in impulsive noise and employ spatial smoothing technique to account for coherent signals. Thus, they are applicable to coherent signals in impulsive noise. To further circumvent the coherent signals in impulsive noise, we propose a type of modified fractional lower-order cyclic DOA estimation method. Compared with spatial smoothing fractional lower-order cyclic algorithms, the modified fractional lower-order cyclic algorithms combine the cyclic and conjugate-cyclic statistics together to improve the performance for coherent signals in impulsive noise. Finally, we perform experiments to evaluate the performance of the new proposed algorithms. The designed methods are implemented by using MATLAB. The parameters in the experiments are introduced in Section 5.

3 Preliminary

The DOA problem involves estimating the signal arrival angle using the receive array, which is also called spatial spectral estimation. In this paper, the system is based on a uniform linear array. To simplify the problem, it is assumed that the received signal can be treated as a parallel plane wave and that the distance between any two adjacent elements is not greater than half of the signal wavelength. Assuming that the array spacing of the antenna array is d, the number of elements is M, and the wavelength of the signal is λ, the array output can be expressed as

$$ {x}_m(t)=\sum \limits_{i=1}^K{a}_m\left({\theta}_i\right){s}_i(t)+{n}_m(t),\kern0.75em m=1,2,\cdots, M $$

(1)

where s_i(t) is the ith signal received at the sensor, ${a}_m\left({\theta}_i\right)={e}^{-j\frac{2\pi }{\lambda}\left(m-1\right)d\sin \left({\theta}_i\right)}$ is the mth steering coefficient of the sensor towards the angle θ_i, and n_m(t) is the noise at the mth sensor.

In general, the signal model for DOA estimation can be rewritten in vector form as,

$$ \mathbf{X}(t)=\mathbf{AS}(t)+\mathbf{N}(t) $$

(2)

where S(t) = [s₁(t) s₂(t) ⋯ s_K(t)]^T is the signal vector, $\mathbf{X}(t)={\left[{x}_1(t)\kern0.5em {x}_2(t)\kern0.5em \cdots \kern0.5em {x}_M(t)\right]}^T$ is the vector of signals received by the array sensors, $\mathbf{A}=\left[{a}_1\left({\theta}_1\right)\kern0.5em {a}_2\left({\theta}_2\right)\kern0.5em \cdots \kern0.5em {a}_M\left({\theta}_K\right)\right]$ is the matrix of array steering vectors, and $\mathbf{N}(t)={\left[{n}_1(t)\kern0.5em {n}_2(t)\kern0.5em \cdots \kern0.5em {n}_M(t)\right]}^T$ is the noise vector.

In wireless communications, multipath signals are ubiquitous phenomena in radio propagation that can result in signals reaching the antenna array via two or more paths. For example, if a transmitted wave or echo propagates along multiple paths, the received signals are coherent in wireless communications systems. At the same time, the complexity of the propagation path can result in multiple propagation paths between the base station and users are reached, thereby creating a multipath problem. Therefore, it is important to consider efficient signal processing techniques to limit coherent signals.

For two stationary signals s_i(t) and s_j(t), the correlation coefficients can be written as follows,

$$ {\rho}_{ij}=\frac{E\left[{s}_i(t){s}_j^{\ast }(t)\right]}{\sqrt{E\left[{\left|{s}_i(t)\right|}^2\right]E\left[{\left|{s}_j^{\ast }(t)\right|}^2\right]}} $$

(3)

The correlation of the signals is defined as follows:

$$ \left\{\begin{array}{l}{\rho}_{ij}=0,\kern2em \mathrm{uncorrelated}\\ {}0<\left|{\rho}_{ij}\right|<1,\mathrm{correl}\mathrm{ate}\\ {}\left|{\rho}_{ij}\right|=1,\kern1.75em \mathrm{coherent}\end{array}\right. $$

(4)

According to (3) and (4), coherent signals can be expressed in the following form [4].

$$ {s}_i(t)={u}_i{s}_j(t) $$

(5)

where u_i is a complex.

Many manmade signals encountered in communications, radar, and sonar systems are from a special class of processes whose statistical functions are periodic functions of time, such as signals with amplitude modulation (AM), binary phase shift keying (BPSK), and quaternary phase shift keying (QPSK). These signals are referred to as cyclostationary signals, which exhibit inherent cyclostationarity properties. The cyclostationarity can be used for effective signal processing and make the signals immune to interference and noise, which have different cyclostationarity as that of the signals of interest or are not cyclostationary signals.

The cyclic autocorrelation of x(t) is defined by

$$ {R}_x^{\varepsilon}\left(\tau \right)=\left\langle x\left(t+\tau /2\right){x}^{\ast}\left(t-\tau /2\right){e}^{-j2\pi \varepsilon t}\right\rangle $$

(6)

where $\left\langle \cdot \right\rangle =\underset{T\to \infty }{\lim}\left(1/T\right){\int}_{-T/2}^{T/2}\left(\cdot \right) dt$ is the time-averaging operation, ε represents all the harmonics of the fundamental cycle frequencies of x(t), and “∗” denotes the conjugate operation. The cyclic autocorrelation function is a characteristic property of second-order cyclostationarity. The signal x(t) is said to contain cyclostationarity if and only if ${R}_x^{\varepsilon}\left(\tau \right)\ne 0$ for some non-zero cycle frequencies ε. The cyclic spectrum (also called the spectral correlation function) is defined as the Fourier transform of the cycle autocorrelation function:

$$ {S}_x^{\varepsilon }(f)={\int}_{-\infty}^{\infty }{R}_x^{\varepsilon}\left(\tau \right){e}^{-j2\pi f\tau} d\tau $$

(7)

Furthermore, when ε = 0, the cyclic correlation becomes a conventional correlation function, and the cyclic spectrum is the same as the power spectrum.

The analytic properties of the Gaussian distribution have made it the most significant statistical distribution for noise modeling. However, there are many noises in communication, radar, and sonar systems that are decidedly non-Gaussian and inherently impulsive in nature. The α-stable distribution is appropriate for describing these non-Gaussian noises and has been effectively used to characterize impulsive phenomena. However, the α-stable distribution only has a closed form probability density function in some special cases. The characteristic function of the α-stable distribution is given as follows:

$$ \varphi (t)={e}^{\left\{ jat-\gamma {\left|t\right|}^{\alpha}\left[1+ j\beta \operatorname{sgn}(t)\omega \left(t,a\right)\right]\right\}} $$

(8)

where

$$ \omega \left(t,\alpha \right)=\left\{\begin{array}{cc}\tan \left(\pi \alpha /2\right)& \alpha \ne 1\\ {}\left(2/\pi \right)\log \left|t\right|& \alpha =1\end{array}\right. $$

(9)

and

$$ \operatorname{sgn}(t)=\left\{\begin{array}{c}1,\kern0.5em t>0\\ {}\begin{array}{cc}0,& t=0\end{array}\\ {}\begin{array}{cc}-1,& t<0\end{array}\end{array}\right. $$

(10)

We find that the stable distribution can be determined by four parameters, the characteristic exponent α (0 < α ≤ 2), symmetric parameter β (−1 ≤ β ≤ 1), scale parameter γ (γ > 0), and location parameter a (− ∞  < α <  + ∞). When β = 0, the stable distribution is called symmetric α-stable distribution (SαS). The smaller the characteristic exponent α is, the heavier the tails of the SαS distribution. The Gaussian distribution is a special case of the SαS family (α = 2).

Compared with Gaussian noise which has an exponential tail, α-stable distributions have algebraic tails. This property makes the second-order or higher order moments of stable distribution non-existence (α > 2). In particular, even the first-order moment of it is non-existent when α > 1. Thus, the α-stable impulsive components of x(t) make the second-order autocorrelation function R_x(t, τ) non-existent. As a result, the cyclic autocorrelation function ${R}_x^{\varepsilon}\left(\tau \right)$ and spectral correlation function ${S}_x^{\varepsilon }(f)$ become useless. Therefore, the conventional second-order statistics-based subspace DOA algorithms and the cyclostationarity-based subspace DOA algorithms degrade in the presence of impulsive noise.

The conventional MUSIC, ESPRIT, cyclic MUSIC, and cyclic ESPRIT algorithms measured with interfering signal and noise are shown in Figs. 1 and 2. The number of arrays is M = 8, the signal-to-noise ratio (SNR) is 0 dB, the number of snapshots is N = 1024, and the sampling frequency is f_s = 10⁹ Hz. The SOI is a BPSK signal. The carrier frequency of the BPSK signal is f₁ = 0.25f_s, and the symbol rate is ε₁ = 0.025f_s. The interfering signal is a QPSK signal with a carrier frequency of f₂ = 0.2f_s and a symbol rate of ε₂ = 0.02f_s. The angles of incidence of the BPSK and QPSK signals are 20^∘ and 50^∘, respectively. The cycle frequency exploited by the cyclic algorithms is ε = 2f₁, which is the cycle frequency of the BPSK signal.

Fig. 1

MUSIC and cyclic MUSIC method results: a Gaussian noise and b impulsive noise (α = 1.6). — represents estimation results of cyclic MUSIC method. -- represents estimation results of MUSIC method

Fig. 2

ESPRIT and cyclic ESPRIT algorithms results: a Gaussian noise and b impulsive noise (α = 1.6). о represents estimation results of cyclic ESPRIT method. + represents estimation results of ESPRIT method

The simulation results indicate that the traditional MUSIC, ESPRIT, cyclic MUSIC, and cyclic ESPRIT algorithms work normally in Gaussian noise. However, the cyclic MUSIC and cyclic ESPRIT are signal selective and can suppress the interfering QPSK signal for only one peak corresponding to the measured DOA. In contrast, MUSIC and ESPRIT cannot suppress the effects of interference. There are two peaks for MUSIC and ESPRIT, one for the DOA of the SOI and one for the DOA of the interference. Although all the algorithms are robust to Gaussian noise, Figs. 1b and 2b show that all the algorithms exhibit severe degradation when impulsive noise is encountered. Moreover, they are not robust to α-stable distributed impulsive noise.

Figure 3 shows the DOA estimation results of the cyclic subspace methods for coherent BPSK signals. In this simulation, the parameters of the array are the same as those in Figs. 1 and 2. The carrier frequency and keying rate of the coherent BPSK signals are the same as those for the BPSK signal in Figs. 1 and 2. The interference and noise are absent in this case. The DOAs of the two coherent BPSK signals are 20^∘ and 40^∘, respectively. All estimation results are obtained from 10 realizations. Figure 3 shows that cyclic MUSIC and cyclic ESPRIT are not applicable for coherent signals, and they cannot provide accurate DOA estimates for coherent signals.

Fig. 3

DOA estimation results of coherent signals: a Cyclic MUSIC and b Cyclic ESPRIT. a Blue — represents estimation results of cyclic MUSIC. Red -- represents real azimuth. b * represents estimation results of cyclic ESPRIT. · represents real angle

The simulation results (Figs. 1, 2, and 3) indicate that the cyclic MUSIC and cyclic ESPRIT algorithms can suppress Gaussian noise and interference signals, but cannot be applied to coherent sources and impulsive noise. To overcome the limitations of these cyclic subspace DOA estimation algorithms, it is necessary to exploit the additional signal properties and develop novel signal processing techniques that enable the use of cyclostationarity-based subspace algorithms and are tolerant to coherent signals and robust to impulsive noise.

4 Methods of cyclostationarity-based DOA estimation for coherent signals

Because of the uselessness of the conventional cyclic correlation function and the spectral correlation function, conventional cyclic DOA algorithms cannot be effectively applied to impulsive noise. Furthermore, conventional cyclic DOA algorithms cannot account for coherent signals. To circumvent these issues, we develop new DOA estimation algorithms based on fractional lower-order cyclostationarity. The new proposed algorithms share the signal selectivity traits that make them immune to the interference, Gaussian noise, and impulsive noise and are applicable to coherent signals.

4.1 The proposed smoothing fractional lower-order cyclic DOA methods

The performance of conventional cyclic DOA algorithms degrades in impulsive noise and coherent signal scenes. It is necessary to develop new cyclostationarity-based DOA estimation algorithms that are tolerant to coherent signals and interference and are robust to both Gaussian noise and impulsive noise.

In realistic communication scenarios, the case of correlated or coherent signals is inevitable, and the measurement vector at the receiver can be written as

$$ \mathbf{X}(t)=\mathbf{A}\left(\mathbf{C}s(t)\right)+\mathbf{N}(t)=\mathbf{A}\mathbf{S}(t)+\mathbf{N}(t) $$

(11)

where C is the K × 1 complex constant attenuation vector and S(t) = Cs(t). The K sources are coherent, and thus, the signals can be described as

$$ \mathbf{S}(t)=\left[\begin{array}{c}{c}_1s(t)\\ {}{c}_2s(t)\\ {}\vdots \\ {}{c}_Ks(t)\end{array}\right]=\left[\begin{array}{c}1\\ {}{c}_2\\ {}\vdots \\ {}{c}_K\end{array}\right]s(t) $$

(12)

where c₁ = 1. The covariance matrix of X(t) can be obtained as follows:

$$ \mathbf{R}=E\left[{\mathbf{XX}}^H\right]=\mathbf{A}E\left[{\mathbf{SS}}^H\right]{\mathbf{A}}^H+{\sigma}^2\mathbf{I} $$

(13)

From (12), the covariance matrix of S(t) can be given by,

$$ E\left[{\mathbf{SS}}^H\right]=\overline{{\left|{s}_1(t)\right|}^2}\left[\begin{array}{c}1\\ {}{c}_2\\ {}\vdots \\ {}{c}_K\end{array}\right]\left[1\kern0.5em {c}_2\kern0.5em \cdots \kern0.5em {c}_K\right] $$

(14)

The coherent signals have a certain relationship, that is, they change in a unified way with time, causing the information to be hidden. Therefore, the signal and noise subspaces cannot be constructed by eigenvalue decomposition, and direction estimates based on the conventional feature structure subspace are invalid for coherent signals.

To address cyclostationary signal coherence phenomena in the impulsive noise environment, we introduce two spatial smoothing-based fractional lower-order cyclic algorithms. The performance of the MUSIC and ESPRIT algorithms can be drastically improved compared to that of the ordinary MUSIC and ESPRIT algorithms by using the Hermitian sample covariance in DOA estimation [7, 8]. The M (M > K) element uniform line array is divided into L uniformly overlapping subarrays of dimension Q (Q > K), in such a way that each sub-array shares all but one of its sensors with an adjacent sub-array [6]. The signal of the lth sub-array is

$$ {\mathbf{x}}_l^f(t)={\left[{x}_l(t),{x}_{l+1}(t),\cdots, {x}_{l+N-1}(t)\right]}^T={\mathbf{A}}_M{\mathbf{D}}^{l-1}s(t)+{n}_l(t),1\le l\le L $$

(15)

where A_M is the direction matrix of Q × K dimension, which is the Q dimensional steering vector a_M(θ_i) (i = 1, 2, ⋯, K), and $\mathbf{D}=\operatorname{diag}\left({e}^{j\frac{2\pi }{\lambda}\sin {\theta}_1},{e}^{j\frac{2\pi }{\lambda}\sin {\theta}_2},\cdots, {e}^{j\frac{2\pi }{\lambda}\sin {\theta}_K}\right)$.

Since the conventional cyclic statistics are useless for exploiting cyclostationarity in impulsive noise, it is necessary to use other properties of signals to develop robust coherent DOA estimation algorithms. It has been shown in [30] that FLOS-based cyclic statistics can be applied to suppress the impulsive noise for cyclostationary signals, and a new type of pth-order cyclostationarity has been developed. Considering a cyclostationary signal x(t), the pth-order correlation of x(t) is defined by

$$ {R}_x^p\left(t,\tau \right)=E\left(x\left(t+\tau /2\right){\left[{x}^{\ast}\left(t-\tau /2\right)\right]}^{\left\langle p\right\rangle}\right),\kern0.5em 1\le p<\alpha $$

(16)

where the pth-order phased fractional lower-order moment (PFLOM) is defined as z^〈p〉 = |z|^p − 1z. The PFLOM can be re-expressed in a polar form (z = re^jθ) as follows:

$$ {z}^{\left\langle p\right\rangle }={r}^p{e}^{j\theta} $$

(17)

Note that the PFLOM acts only on the magnitude of the operand and preserves the corresponding phase; thus, z^〈p〉 has the same period as z, and x(t) and (x(t))^〈p〉 have the same period.

According to the PFLOM properties, the Fourier series of ${R}_x^p\left(t,\tau \right)$ can be represented as

$$ {R}_x^p\left(t,\tau \right)=\sum \limits_{\varepsilon }{R}_x^{\varepsilon, p}\left(\tau \right){e}^{j2\pi \varepsilon t} $$

(18)

where ε represents all cycle frequencies of x(t). The Fourier coefficient ${R}_x^{\varepsilon, p}\left(\tau \right)$ is referred to as the pth-order cyclic correlation function and given as follows:

$$ {R}_x^{\varepsilon, p}\left(\tau \right)=\left\langle x\left(t+\tau /2\right){\left[{x}^{\ast}\left(t-\tau /2\right)\right]}^{\left\langle p\right\rangle }{e}^{-j2\pi \varepsilon t}\right\rangle $$

(19)

According to the pth-order and stable process theory [31], the covariation 〈x(t + τ/2)[x^∗(t − τ/2)]^〈p〉〉 is robust to the impulsive noise when 1 ≤ p < α; thus, the pth-order cyclic correlation function can refrain the effects of impulsive noise. In fact, the cyclic correlation function can be represented as the cross-correlation of frequency-shifted versions of ${R}_x^{\varepsilon}\left(\tau \right)=\left\langle u\left(t+\tau /2\right){v}^{\ast}\left(t-\tau /2\right)\right\rangle$ where u(t) = x(t)e^−jπεt and v(t) = x(t)e^jπεt. By using the properties of z^〈k〉e^±jπεt = (ze^±jπεt)^〈k〉 and (z^∗)^〈k〉 = (z^〈k〉)^∗, the pth-order cyclic correlation function (19) can be rewritten as

$$ {R}_x^{\varepsilon, p}\left(\tau \right)=\left\langle {u}^{\prime}\left(t+\tau /2\right)\left[{v}^{\prime}\left(t-\tau /2\right)\right]\right\rangle $$

(20)

where

$$ {u}^{\hbox{'}}(t)=x(t){e}^{- j\pi \varepsilon t} $$

(21a)

$$ {v}^{\hbox{'}}(t)={\left(x(t)\right)}^{\left\langle p\right\rangle }{e}^{j\pi \varepsilon t} $$

(21b)

Based on Eq. (17), the PFLOM acts only on the magnitude of v^'(t), and v^'(t) has the same period as that of v(t). Therefore, the pth-order cyclic correlation function is an alternative but equivalent characterization of the second-order cyclic function. Furthermore, compared to conventional second-order cyclic correlation function, pth-order cyclic correlation function is robust to SαS distributed (1 < α ≤ 2) impulsive noise. When p = 2, the pth-order cyclic autocorrelation function ${R}_x^{\varepsilon, p}\left(\tau \right)$ becomes the traditional cyclic autocorrelation function ${R}_x^{\varepsilon}\left(\tau \right)$. When the cycle frequency ε = 0, the pth-order cyclic autocorrelation function ${R}_x^{\varepsilon, p}\left(\tau \right)$ reduces to the pth-order correlation function ${R}_x^p\left(\tau \right)$. The pth-order cyclic correlation function plays an essential role in high-resolution direction finding.

It is assumed that interference does not exhibit the same cyclostationarity as the SOI and that noise is not a cyclostationary process. Thus, the fractional lower-order covariance matrix of the lth sub-array is obtained as

$$ {\mathbf{R}}_{X_l}^{\varepsilon, p}\left(\tau \right)={\mathbf{A}}_M{\mathbf{D}}^{l-1}{\mathbf{R}}_S^{\varepsilon, p}\left(\tau \right){\left({\mathbf{A}}_M{\mathbf{D}}^{l-1}\right)}^H $$

(22)

where ε is one cycle frequency of s(t). The spatial smoothing fractional lower-order cyclic covariance matrix is given by

$$ {\mathbf{R}}_X^{\varepsilon, p}\left(\tau \right)=\frac{1}{L}\sum \limits_{l=1}^L{\mathbf{R}}_{X_l}^{\varepsilon, p}\left(\tau \right) $$

(23)

Furthermore, we can use singular value decomposition (SVD) to decompose the covariance matrix ${\mathbf{R}}_X^{\varepsilon, p}\left(\tau \right)$,

$$ {\mathbf{R}}_X^{\varepsilon, p}\left(\tau \right)=\left[{\mathbf{U}}_S\kern0.5em {\mathbf{U}}_Q\right]\left[\begin{array}{cc}{\boldsymbol{\Sigma}}_S& 0\\ {}0& {\boldsymbol{\Sigma}}_Q\end{array}\right]\left[\begin{array}{c}{\mathbf{V}}_S^H\\ {}{\mathbf{V}}_Q^H\end{array}\right] $$

(24)

From (22), we can obtain U_S and U_Q. The spectral estimation is achieved based on the orthogonality of A(θ) and U_Q,

$$ P\left(\theta \right)=\frac{1}{a^H\left(\theta \right){\mathbf{U}}_Q{\mathbf{U}}_Q^Ha\left(\theta \right)} $$

(25)

Then, the DOA of the proposed smoothing fractional lower-order cyclic MUSIC algorithm can be obtained from a search for peaks in the spectrum of (25).

Due to the computational complexity of the MUSIC algorithm, we further propose an improved cyclic ESPRIT algorithm called the smoothing fractional lower-order cyclic ESPRIT algorithm. As in the cyclostationary MUSIC algorithm, the received signal can be expressed as in (12). According to Eq. (22) and the spatial smoothing method, we get the following equation:

$$ {\displaystyle \begin{array}{l}{\mathbf{R}}_{X_l}^{\varepsilon, p}\left(\tau \right)=\left\langle \left[{\mathbf{X}}_l\left(t+\tau /2\right)\right]{\left[{\mathbf{X}}_l^{\left\langle p\right\rangle}\left(t-\tau /2\right)\right]}^H{e}^{-j2\pi \varepsilon t}\right\rangle \\ {}\kern3em ={\mathbf{A}}_l{\mathbf{D}}^{l-1}{\mathbf{R}}_S^{\varepsilon, p}\left(\tau \right){\left({\mathbf{D}}^{l-1}\right)}^H{\mathbf{A}}_l^H\end{array}} $$

(26)

Then, using spatial smoothing and SVD, the covariance matrix ${\mathbf{R}}_X^{\varepsilon, p}\left(\tau \right)$ can be rewritten as

$$ {\mathbf{R}}_X^{\varepsilon, p}\left(\tau \right)=\frac{1}{L}\sum \limits_{l=1}^L{\mathbf{R}}_{X_l}^{\varepsilon, p}\left(\tau \right)=\left[{\mathbf{U}}_S\kern0.5em {\mathbf{U}}_N\right]\left[\begin{array}{cc}{\boldsymbol{\Sigma}}_S& 0\\ {}0& {\boldsymbol{\Sigma}}_N\end{array}\right]\left[\begin{array}{c}{\mathbf{V}}_S^H\\ {}{\mathbf{V}}_N^H\end{array}\right] $$

(27)

U_S and U_N can be obtained from (25). To develop the smoothing fractional lower-order cyclic ESPRIT algorithm, U_S is divided into U_S1 and U_S2. Specifically, the relation between U_S1 and U_S2 can be determined by

$$ {\mathbf{U}}_{S2}={\mathbf{U}}_{S1}{\mathbf{T}}^{-1}\boldsymbol{\Phi} \mathbf{T}={\mathbf{U}}_{S1}\boldsymbol{\Psi} $$

(28)

where Ψ = T⁻¹ΦT. Additionally,

$$ \boldsymbol{\Phi} =\operatorname{diag}\left\{{e}^{j{\mu}_1},{e}^{j{\mu}_2},\dots, {e}^{j{\mu}_k}\right\} $$

(29)

where μ_k = ω₀Δ sin θ_k/c(k = 1, 2,  … , K). Note that (28) indicates that the characteristic value of Ψ is a diagonal element of Φ. Therefore, after obtaining $\boldsymbol{\Psi} ={\mathbf{U}}_{S1}^{+}{\mathbf{U}}_{S2}$, the eigenvalues and diagonal elements of Φ can be obtained. The DOA estimate can be obtained by,

$$ \overset{\wedge }{\theta_k}=\arcsin \left\{c\cdot \mathrm{angle}\left({\lambda}_k\right)/\left({\omega}_0\varDelta \right)\right\} $$

(30)

where λ_k is the eigenvalue of Ψ.

4.2 The proposed modified fractional lower-order cyclic DOA methods

Both cyclic and conjugate-cyclic statistics can be combined to improve the performance of the signal selective DOA estimation method [23]. To circumvent the coherent signals, we define a new matrix based on the conjugate-cyclic statistics,

$$ \mathbf{Y}(t)={\mathbf{JX}}^{\ast }(t) $$

(31)

where X^∗(t) is the complex conjugation of X(t), and

$$ \mathbf{J}=\left[\begin{array}{ccccc}0& 0& \cdots & 0& 1\\ {}0& 0& \cdots & 1& 0\\ {}\vdots & \vdots & \cdots & \vdots & \vdots \\ {}0& 1& \cdots & 0& 0\\ {}1& 0& \cdots & 0& 0\end{array}\right] $$

(32)

From (32), we find that J² = I; therefore, the correlation matrix of Y(t) = JX^∗(t) is given by

$$ E\left({\mathbf{YY}}^H\right)=\mathbf{J}{\left[{\mathbf{R}}_X\left(\tau \right)\right]}^{\ast }{\mathbf{J}}^H $$

(33)

Then, the fractional lower-order cyclic covariance matrix of Y(t) is defined in the following form:

$$ {\mathbf{R}}_Y^{\varepsilon, p}\left(\tau \right)=\mathbf{JA}{\left[{\mathbf{R}}_S^{\varepsilon, p}\left(\tau \right)\right]}^{\ast }{\mathbf{A}}^H{\mathbf{J}}^H $$

(34)

To develop new DOA estimation methods, we define a modified fractional lower-order cyclic covariance matrix,

$$ {\mathbf{R}}^{\varepsilon, p}\left(\tau \right)={\mathbf{R}}_X^{\varepsilon, p}\left(\tau \right)+{\mathbf{R}}_Y^{\varepsilon, p}\left(\tau \right) $$

(35)

Because R^{ε, p}(τ) is not a Hermitian matrix, we can use SVD to decompose the covariance matrix R^{ε, p}(τ),

$$ {\mathbf{R}}^{\varepsilon, p}\left(\tau \right)={\mathbf{USV}}^H $$

(36)

From (36), we can obtain the K_a non-zero singular values of R^{ε, p}(τ), and the other M − K_a singular values equals to zero. For rank(R^{ε, p}(τ)) = K_a, U and V can then be divided,

$$ {\mathbf{R}}^{\varepsilon, p}\left(\tau \right)=\left[{\mathbf{U}}_S\kern0.5em {\mathbf{U}}_N\right]\left[\begin{array}{cc}{\boldsymbol{\Sigma}}_S& 0\\ {}0& {\boldsymbol{\Sigma}}_N\end{array}\right]\left[\begin{array}{c}{\mathbf{V}}_S^H\\ {}{\mathbf{V}}_N^H\end{array}\right] $$

(37)

U_S and U_N can be obtained from (35). The modified fractional lower-order cyclic MUSIC method is obtained using the orthogonality of A(θ) and U_N,

$$ P\left(\theta \right)=\frac{1}{a^H\left(\theta \right){\mathbf{U}}_N{\mathbf{U}}_N^Ha\left(\theta \right)} $$

(38)

Thus, the DOA can be obtained from a search for the peaks in the spectrum of (38).

We further propose an improved ESPRIT algorithm called the modified fractional lower-order cyclic ESPRIT algorithm. As in the cyclostationary MUSIC algorithm, the fractional lower-order covariance matrix ${\mathbf{R}}_X^{\varepsilon, p}\left(\tau \right)$ is obtained, and Y(t) is defined as the product of J and the complex conjugate of X^∗(t). Based on the SVD of R^{ε, p}(τ), the signal subspace U_S and noise subspace U_N are determined. Then, we can extract U_S1 and U_S2 from U_S. According to the property of rotational invariance, we obtain,

$$ {\mathbf{U}}_{S2}={\mathbf{U}}_{S1}{\mathbf{T}}^{-1}\boldsymbol{\Phi} \mathbf{T}={\mathbf{U}}_{S1}\boldsymbol{\Psi} $$

(39)

where Ψ is the transformation matrix $\boldsymbol{\Psi} ={\mathbf{U}}_{S1}^{+}{\mathbf{U}}_{S2}$.

When R^{ε, p}(τ) is nonsingular, the space of A and U_S is the same, and the linear transformation matrix of A and U_S is T. By establishing the relation between the rotationally invariant factor Φ and the transformation matrix Ψ, we obtain the following equation,

$$ \boldsymbol{\Phi} ={\mathbf{T}\boldsymbol{\Psi } \mathbf{T}}^{-1} $$

(40)

Finally, we can determine the diagonal elements Φ by finding the eigenvalue of ψ, that is the DOA estimates is obtained as

$$ {\overset{\wedge }{\theta}}_k=\arcsin \left\{c\cdot \mathrm{angle}\left({\lambda}_k\right)/\left({\omega}_0\Delta \right)\right\} $$

(41)

5 Simulation results and discussion

In this section, we present various simulation results that are used to assess the performance of the proposed algorithms. We verify the effectiveness and robustness of the proposed smoothing fractional lower-order cyclic MUSIC (SFCyclic-MUSIC), smoothing fractional lower-order cyclic ESPRIT (SFCyclic-ESPRIT), modified fractional lower-order cyclic MUSIC (MFCyclic-MUSIC), and modified fractional lower-order cyclic ESPRIT (MFCyclic-ESPRIT) algorithms and compare their performance to the performance of four classes of existing FLOS-based, cyclostationarity-based, sparse representation algorithms, namely, the fractional lower-order moment-based MUSIC (FLOM-MUSIC) and ESPRIT (FLOM-ESPRIT), the modified MUSIC (M-MUSIC) and ESPRIT (M-ESPRIT), the cyclic MUSIC (Cyclic-MUSIC) and ESPRIT (Cyclic-ESPRIT), the Bayes-optimal algorithm [24], and conventional sparse Bayesian learning (SBL) algorithms for those frequently encountered communication signals (e.g., BPSK and QPSK).

Because the α-stable distribution process has finite variance only for α = 2, the traditional SNR is inappropriate for determining the power of SαS noise. Thus, we use a generalized signal-to-noise ratio (GSNR) which is defined as [27],

$$ \mathrm{GSNR}=10{\log}_{10}\left(\frac{1}{\gamma \times N}\sum \limits_{n=1}^N{\left|s(t)\right|}^2\right) $$

(42)

According to the GSNR metric, the SαS impulsive noise samples are power scaled by the dispersion parameter γ. The performance of the algorithms is evaluated by the root-mean-square error (RMSE) of the DOA estimates and the probability of resolution. The RMSE is defined by

$$ \mathrm{RMSE}\left[{\theta}_k\right]=\sqrt{\frac{1}{N}\sum \limits_{n=1}^N{\left({\widehat{\theta}}_n-{\theta}_k\right)}^2} $$

(43)

The probability of resolution is also called the success probability which is defined as the ratio of the number of successful runs to the total number of Monte Carlo runs. The number of arrays is M = 10, the number of the snapshots is N = 1024, and the sampling frequency is f_s = 10⁹ Hz.

5.1 DOA estimation of the proposed methods

In this case, the incident signals are s₁(t), s₂(t), s₃(t), and s₄(t) with DOA(θ₁, θ₂, θ₃, θ₄) = (10^∘, 15^∘, 40^∘, 50^∘). The carrier frequency and symbol rate of the coherent BPSK signals of s₁(t) and s₂(t) are f₁ = 0.25f_s and ε₁ = 0.025f_s, respectively. s₃(t) is a BPSK signal with a carrier frequency of f₂ = 0.25f_s and keying rate of ε₂ = 0.025f_s. s₄(t) is a QPSK signal with a carrier frequency and symbol rate of f₃ = 0.1f_s and ε₃ = 0.05f_s, respectively. s₃(t) and s₄(t) are independent of s₁(t) and s₂(t). The cycle frequency used by the cyclic algorithms is ε = 2f₁, which is the cycle frequency of s₁(t), s₂(t), and s₃(t).

The performance of the proposed methods for coherent signals in impulsive noise with α = 1.6 (GSNR = 10 dB) is shown in Figs. 4 and 5. The simulation results indicate that all the proposed methods are robust to impulsive noise. However, the MFCyclic-MUSIC and MFCyclic-ESPRIT methods make better use of fractional lower-order cyclostationarity and conjugate fractional lower-order cyclic cyclostationarity and are superior to the SFCyclic-MUSIC and SFCyclic-ESPRIT methods. Figure 5 illustrates that the MFCyclic-MUSIC and MFCyclic-ESPRIT methods can still suppress extremely impulsive noise (α = 1.6) and provide accurate DOA estimates for cyclostationary coherent sources in the presence of impulsive noise and interference signals.

Fig. 4

DOA estimation of proposed algorithms for coherent sources in Gaussian noise: a SFCyclic-MUSIC and b SFCyclic-ESPRIT. a — represents estimation results of SFCyclic-MUSIC. b represents estimation results of SFCyclic-ESPRIT

Fig. 5

DOA estimation of proposed algorithms for coherent sources in impulsive noise: a MFCyclic-MUSIC and b MFCyclic-ESPRIT. a — represents estimation results of MFCyclic-MUSIC. b represents estimation results of MFCyclic-ESPRIT

5.2 Influence of impulsive noise

We consider a BPSK communication signal representing the SOI. The carrier frequency of the BPSK signal is f_c = 0.25f_s, the keying rate is ε_k = 0.0625f_s, and the DOA of the signal is 10^∘. The interference in this case is a BPSK signal with a carrier frequency of f₁ = 0.2f_s, keying rate of ε₁ = 0.025f_s, and a DOA of 20^∘. The signal-to-interference ratio (SIR) is 3 dB. We use a cycle frequency of ε = ε_k = 0.0625/T_s and p = 1.2. The estimation results are obtained from 1000 Monte Carlo realizations.

A comparison of the proposed smoothing fractional lower-order cyclic methods and modified fractional lower-order cyclic methods for different values of the characteristic exponent α is conducted. The simulation scenario includes a GSNR = 10 dB and p = 1.1. As shown in Fig. 6a and 6b, the impulsive characteristics of the stable distribution are weakened as α increases, and the performance of all the algorithms improved. Moreover, the simulation results indicate that MFCyclic-MUSIC and MFCyclic-ESPRIT slightly outperformed SFCyclic-MUSIC and SFCyclic-ESPRIT when the impulsive characteristics are strong. When α ≥ 1.9, the two types of algorithms are basically the same.

Fig. 6

DOA estimation accuracy as a function of the α: a RMSE and b success probability. a —□—represents DOA estimation accuracy of SFCyclic-MUSIC. —+— represents DOA estimation accuracy of SFCyclic-ESPRIT. ◇— represents DOA estimation accuracy of MFCyclic-MUSIC. о— represents DOA estimation accuracy of MFCyclic-ESPRIT. b —□—represents success probability of SFCyclic-MUSIC. —+— represents success probability of SFCyclic-ESPRIT. ◇— represents success probability of MFCyclic-MUSIC. о— represents success probability of MFCyclic-ESPRIT

Simulations that compare the performance of the proposed MFCyclic-MUSIC and MFCyclic-ESPRIT algorithms with FLOM-MUSIC, FLOM-ESPRIT, cyclic MUSIC, and cyclic ESPRIT in impulsive noise and interference are given in Fig. 7. The results in Fig. 7a and 7b indicate that the second-order cyclic statistics-based methods display the worst performance for the impulsive noise, because these second-order cyclic statistics methods fail with impulsive noise, especially the GSNRs is less than 13 dB. Although the impulsive noise is suppressed by the FLOM-based methods, due to the interfering signal, the performance of FLOM-MUSIC and FLOM-ESPRIT remains essentially unchanged when the GSNR is greater than approximately 10 dB. Compared to the conventional FLOM-based and cyclostationarity-based algorithms, the proposed algorithms can simultaneously suppress the effects of impulsive noise and interfering signals; thus, the performance of the proposed algorithms is better than that of the other methods studied.

Fig. 7

DOA estimation accuracy versus GSNR in impulsive noise: a RMSE and b success probability. a □— represents DOA estimation accuracy of MFCyclic-MUSIC.·-- represents DOA estimation accuracy of MFCyclic-ESPRIT. о— represents DOA estimation accuracy of FLOM-MUSIC. Δ-- represents DOA estimation accuracy of FLOM- ESPRIT. *— represents DOA estimation accuracy of cyclic MUSIC. ◇--represents DOA estimation accuracy of cyclic ESPRIT. b □— represents DOA success probability of MFCyclic-MUSIC.·-- represents DOA success probability of MFCyclic-ESPRIT. о— represents DOA success probability of FLOM-MUSIC. Δ-- represents DOA success probability of FLOM-ESPRIT. *— represents DOA success probability of cyclic MUSIC. ◇--represents DOA success probability of cyclic ESPRIT

The performance of the proposed algorithms and the exiting methods in Gaussian noise and interference is shown in Fig. 8. Figure 8 shows that although the FLOM-based methods can effectively suppress Gaussian and non-Gaussian impulsive noises, the performance does not improve as the GSNR increases due to the interfering signal. Therefore, the proposed fractional lower-order cyclostationarity-based methods outperform the FLOM-based methods in the presence of interference. Compared Figs. 7 with 8, we can see that, although the proposed algorithms are equivalent in performance to the second-order cyclostationarity-based methods in Gaussian noise and interference environment, they perform much better performance in the presence of impulsive noise. Therefore, the proposed methods display good adaptability to Gaussian noise and non-Gaussian impulsive noise.

Fig. 8

DOA estimation accuracy versus GSNR in Gaussian noise: a RMSE and b success probability. a □— represents DOA estimation accuracy of MFCyclic-MUSIC.·-- represents DOA estimation accuracy of MFCyclic-ESPRIT. о— represents DOA estimation accuracy of FLOM-MUSIC. Δ-- represents DOA estimation accuracy of FLOM-ESPRIT. *— represents DOA estimation accuracy of cyclic MUSIC. ◇-- represents DOA estimation accuracy of cyclic ESPRIT. b □— represents success probability of MFCyclic-MUSIC. -- represents success probability of MFCyclic-ESPRIT. о— represents success probability of FLOM-MUSIC. Δ-- represents success probability of FLOM-ESPRIT. *— represents success probability of cyclic MUSIC. ◇--represents success probability of cyclic ESPRIT

5.3 Influence of coherent signals

In this case, we consider two coherent QPSK communication signals. The carrier frequency of the QPSK signals is f_c = 0.2f_s, and the keying rate is ε_k = 0.025f_s. The DOA of the signals are 10^∘ and 15^∘, and the angle to be estimated is 10^∘. The coband interference is an AM signal that has the same carrier frequency and bandwidth as that of the QPSK signals. The DOA of the AM signal is 20^∘. The cycle frequency exploited by the cyclic algorithm is set to ε = ε_k and GSNR = 0 dB. The estimation results are obtained from 1000 Monte Carlo realizations.

A simulation is conducted to compare the RMSE versus GSNR for the proposed algorithms, modified MUSIC, and modified ESPRIT, considering both impulsive noise and Gaussian noise for the coherent sources. The accuracy of the algorithms in impulsive noise and Gaussian noise environments is shown in Fig. 9. Although the second-order cyclic statistics-based M-MUSIC and M-ESPRIT are suitable for coherent signals, they cannot suppress impulsive noise. Thus, both the two methods degrade severely when impulsive noise is encountered. The M-MUSIC and M-ESPRIT are effective for coherent signals in Gaussian noise; however, because of the existence of AM interference, the performance of traditional modified algorithms is not further improved. These algorithms failed in the case in which coband interference is present. In contrast, the proposed algorithms are superior to the traditional modified algorithms, and they effectively suppressed impulsive noise. It can be seen from Fig. 9 that the proposed algorithms are immune to the effects of the coband interfering signal and can provide accurate DOA estimates for coherent signals in Gaussian noise and impulsive noise.

Fig. 9

DOA estimation accuracy as a function of GSNR: a impulsive noise and b Gaussian noise. □— represents DOA estimation accuracy of MFCyclic-MUSIC. ·-- represents DOA estimation accuracy of MFCyclic-ESPRIT. о— represents DOA estimation accuracy of M-MUSIC. ◇-- represents DOA estimation accuracy of M-ESPRIT

The sparse representation methods can estimate coherent sources in impulsive noise; we compare the performance of the proposed MFCyclic-MUSIC algorithm with Bayes-optimal algorithm, conventional SBL algorithms, and FLOM-MUSIC method in the impulsive noise (α = 1.5) and in the coband interfering environments, respectively. Because the FLOM-MUSIC cannot estimate coherent signals, it can be seen from Fig. 10 that the FLOM-MUSIC achieves the worst performance. Moreover, comparing Fig. 10a with b, we can see that the Bayes-optimal algorithm achieves the best suppression performance for the impulsive noise and the proposed method outperforms Bayes-optimal algorithm against interference.

Fig. 10

RMSE as a function of GSNR at different cycle frequencies in impulsive noise. a □— represents DOA success probability of MFCyclic-MUSIC. *— represents DOA success probability of FLOM-MUSIC. Δ— represents DOA success probability of Bayes-optimal. о— represents DOA success probability of SBL. b □— represents DOA success probability of MFCyclic-MUSIC. *— represents DOA success probability of FLOM-MUSIC. Δ— represents DOA success probability of Bayes-optimal. о— represents DOA success probability of SBL

Cycle frequency is one of the most important representation of cyclostationarity. Typical cycle frequencies for many communications signals include the double carrier frequency, the keying rate and the corresponding harmonics, as well as the sums and differences of these cycle frequencies. For example, for a BPSK signal or a QPSK signal, the cycle frequencies are ±2f_c, ε_k, and ε_i =  ± 2f_c + iε_k (i = 0, ± 1, ± 2, …), where f_c is the carrier frequency and ε_k represents the keying rate. If we use the cycle frequency at which the spectral correlation is stronger, the performance of the proposed cyclic algorithms can be improved.

The estimation accuracy of the proposed algorithms at different cycle frequencies is shown in Fig. 11. The impulsive noise with α = 1.8 is added only in the simulation. The simulation results indicate that the proposed methods perform effectively at the two cycle frequencies. However, for the QPSK signal, the cyclostationarity characteristic is stronger at ε = 2f_c than at ε = ε_k, so the performance of the two proposed algorithms is slightly better at ε = 2f_c than at ε = ε_k.

Fig. 11

RMSE as a function of GSNR at different cycle frequencies in impulsive noise. □— represents DOA estimation accuracy of MFCyclic-MUSIC when ε = ε_k. +— represents DOA estimation accuracy of MFCyclic-ESPRIT when ε = ε_k. —□— represents DOA estimation accuracy of MFCyclic-MUSIC when ε = 2f_c. —+— represents DOA estimation accuracy of MFCyclic-ESPRIT when ε = 2f_c

6 Conclusions

In this paper, new fractional lower-order cyclostationarity-based DOA estimation algorithms for coherent signals in the presence of α-stable impulsive noise and interference are introduced. Traditional second-order cyclostationarity-based and FLOS-based DOA estimation methods severely degrade for coherent signals, under α-stable distributed impulsive noise and interfering signals. By exploiting the fractional lower-order cyclostationarity properties of signals, new smoothing fractional lower-order cyclic subspace algorithms and modified fractional lower-order cyclic subspace algorithms are proposed. The new proposed algorithms not only provide the signal selectivity in the presence of impulsive noise, but also are highly tolerant to coherent signals. Moreover, these algorithms yield more reliable DOA estimates of cyclostationary coherent signals than do cyclostationarity-based and FLOS-based methods in the presence of interference and impulsive noise. The effectiveness and robustness of the proposed algorithms are evaluated via simulations, and the results indicate that the new algorithms are applicable to a wide range of interference, Gaussian noise, and non-Gaussian impulsive noise environments for coherent signals.