On the performance of hybrid carrier system with spectrum precoding based on WFRFT

Abstract

High out-of-band (OOB) power emission has become the main shortcoming of multicarrier scheme in regard to the requirements and challenges of 5G. In this paper, two effective OOB power reduction methods, previously applied to traditional orthogonal frequency division multiplexing (OFDM) system, are proposed in hybrid carrier (HC) scheme based on weighted-type fractional Fourier transform (WFRFT). Simulation results demonstrate that, due to the flexible selection of WFRFT order in the HC system, combined bit error rate (BER) and peak-to-average power ratio (PAPR) performance advantages are gained without sidelobe impact and significant complexity increase in comparison with the OFDM scheme with spectrum precoding. In projection precoding, a smaller error vector magnitude of precoder G _χ is obtained in the proposed hybrid carrier scheme. In SVD precoding with an orthogonal decoding at the receiver, better BER performance could also be acquired at WFRFT order over the fading channels. The proposed two WFRFT-based structures are complementary, and their potential scenarios are given. In addition, applying the feature of projection precoding to channel estimation, a novel pilot structure based on WFRFT is posed to reduce the introduced error of spectrum precoding and finally helps to improve the BER performance.

1 Introduction

Recently, the efficient utilization of idle spectrum becomes the main means to solve the shortage of limited spectrum resources in modern wireless communication system. Cognitive radio and spectrum sensing [1–3] are usually the mainstream technologies to detect the free spectrum. In addition, reducing the unnecessary protection interval between sub-bands through suppressing the out-of-band (OOB) power of in-band data is another effective method to relieve the increasingly scarce spectrum resources. High OOB emission is always the shortcomings of traditional multicarrier schemes and becomes the opposed target of novel waveform candidates of the fifth generation (5G) with multi-antenna [4, 5]. For instance, traditional orthogonal frequency division multiplexing (OFDM) system suffers high OOB leakage due to slow tailing attenuation of sinc function and needs the protected subcarriers to inhibit OOB power radiation. Current 5G candidate waveforms such as generalized frequency division multiplexing (GFDM) and filterbank-based multicarrier (FBMC) in [6–8] focus more on the low OOB radiation through pulse shaping of the transmitted signal.

Existing methods for reducing sidelobe of multicarrier scheme are proposed, involving time domain windowing, subcarrier weighting, insertion of cancelation carriers (CCs), and precoding. In [9], the used subcarriers are multiplied by subcarrier weights for sidelobe suppression based on an optimization algorithm with several optimization constraints. In [10], a multicarrier spectrum sculpting precoder is put forward with flexible choice of the notching frequencies and lower transmitter complexity than that in [9] but suffers bit error rate (BER) performance degradation due to introduced error. An orthogonal multiplexing is proposed in [11] to improve the BER performance according to the knowledge of a precoder at the transmitter. However, how to achieve the channel estimation at the receiver becomes a problem as a result of precoded data and pilot symbols. In [12], two kinds of structures of pilot carriers are proposed on the basis of the precoder in [10]. The practical design for out-of-band emission reduction and adjacent channel interference rejection in the OFDM system is put forward in [13]. N-continuous OFDM is proposed in [14] through precoding the information symbols and achieves obvious out-of-band power reduction with slight decline of reliability. A precoding scheme which focuses on suppressing out-of-subband emission of the DFT-based OFDM system is proposed in [15], whose BER performance is improved as a result of frequency diversity. In addition, there are also other technologies to shape the OFDM signals for sidelobe suppression. The method of inserting several CCs at each side of the OFDM spectrum is posed in [16] with a small degradation of system performance. In [17], the sidelobe of the licensed user in cognitive radio system is reduced through adding extended active interference cancelation signals. The optimal orthogonal precoding of DFT-based system is derived in [18] through accounting the leaked power as a matrix Frobenius norm minimization problem. In [19], an orthogonal projection matrix is given with balanced complexity and out-of-band power suppression performance. Without significant BER degradation, a prescribed mask is used to control the OOB emissions beneath a particular power level in [20] with more distortion on edge subcarriers. An overview of existing OOB reduction techniques is proposed in [21]. All in all, the OOB power radiation is always reduced accompanied by deteriorated BER performance.

The high PAPR is another ubiquitous problem of multicarrier schemes including OFDM and other novel waveform candidates such as FBMC and GFDM. An overview of PAPR reduction techniques for OFDM signals has been given in [22], including clipping, coding, partial transmission sequence (PTS), selective mapping (SLM), nonlinear companding transforms, and tone reservation. However, few literatures focus on the PAPR reduction methods from the aspect of the carrier scheme. Moreover, joint PAPR reduction and sidelobe suppression is achieved in [23] through dynamically extending part of the constellation points on the secondary user subcarriers and adding several signal cancelation symbols on the primary user subcarriers. The optimal cancelation signal is obtained by solving a quadratically constrained quadratic program. A suppressing alignment approach is posed in [24] to achieve OOB power and PAPR suppression by generating a suppressing signal.

As a trade-off of OFDM and single carrier (SC) schemes, the hybrid carrier (HC) scheme based on weighted-type fractional Fourier transform (WFRFT) adapts to diverse scenes due to its flexible and configurable parameters. Generally, the OFDM and SC systems are two special cases of the HC system. Compared to the OFDM/SC system, the merits of the HC system includes better BER performance in narrowband interference suppression in [25]. Moreover, the HC system possesses superior performance over doubly dispersive channels in [26–28] in comparison with the OFDM/SC system. In the techniques applied to the OFDM/SC system such as power allocation, coding are also available in the HC system due to its strong compatibility. In [29], the ICI coefficients introduced by carrier frequency offset is derived in the HC system according to the conclusions in the OFDM and SC systems. In [30], the joint PAPR and sidelobe reduction is achieved in the HC scheme via time domain windowing and frequency domain pulse shaping, where the WFRFT order is proven to be a beneficial regulation of PAPR and BER.

The contribution of this paper includes the application of two spectrum precoding methods in hybrid carrier system based on WFRFT. The interference introduced by spectrum precoding in the HC scheme is analyzed, and a smaller error vector magnitude is obtained in comparison with the traditional OFDM system. In addition, current literatures focus more on OOB suppression and PAPR reduction with BER loss. However, in this paper, both BER and PAPR performances are upgraded simultaneously over AWGN and fading channels without sidelobe regeneration and significant complexity increase. Furthermore, a novel WFRFT-based pilot structure with smaller estimation error is posed according to the characteristics of spectrum precoders.

2 WFRFT

The αth order WFRFT is defined as a linear summation of discrete time signal x(n) and its discrete Fourier transform (DFT) X(n), which is expressed as

(1)

where x(−n) and X(−n) are the reversed form of x(n) and X(n). Note that there is no significant complexity increase when the DFT is substituted by WFRFT in DFT-based multicarrier system.

A group of homologous weighting coefficients is denoted by

$$ \begin{aligned} w_{l}(\alpha) \,=\,\cos\left[\frac{(\alpha\,-\,l)\pi}{4}\right]\cos\left[\frac{2(\alpha\,-\,l)\pi}{4}\right] \exp\left[\pm\mathrm{i}\frac{3(\alpha-l)\pi}{4}\right] \end{aligned} $$

(2)

where the ± in exp[ ·] represents the rotate direction of coefficients in a complex plane as α increases. w _l (α) with exp[ +] rotates in a clockwise direction and exp[ −] does in an opposite trend. The cycle of w _l (α) is 4, and the representative selection of WFRFT order α is in the interval of [ 0,1].

For a vector x, its WFRFT result x _α is calculated as

$$ \mathbf{x}_{\alpha}=\mathbf{W}_{\alpha}\mathbf{x} $$

(3)

which is a vector expression of (1) and the WFRFT matrix W _α is constituted by four power transformations of Fourier matrix, shown as

$$ {{} \begin{aligned} \mathbf{W}_{\alpha}\,=\,\sum\limits_{l=0}^{3}w_{l}(\alpha)\mathbf{F}^{l} \,=\,w_{0}(\alpha)\mathbf{I}\,+\,w_{1}(\alpha)\mathbf{F}\,+\,w_{2}(\alpha)\mathbf{\Gamma}\,+\,w_{3}(\alpha)\mathbf{\Gamma}\mathbf{F} \end{aligned}} $$

(4)

where the shift matrix Γ is defined by

$$ \mathbf{\Gamma}=\left(\begin{array}{ccccc} 1 & 0 & 0 & 0 & 0 \\\ 0 & 0 & 0 & 0 & 1 \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & 1& 0 & 0 \\ 0 & 1 & 0& 0 & 0 \\ \end{array}\right) $$

(5)

The Fourier matrix has the property of F ³=Γ F=F ⁻¹=F ^H. I denotes an identity matrix and N×N normalized Fourier matrix is defined by $[\mathbf {F}]_{m,n}=1/\sqrt {N}\cdot \exp [-j2\pi (m-1)(n-1)/N]$ with m,n=1,⋯,N.

Particularly, (4) could be simplified to W ₀=I and W ₁=F, so the Fourier matrix as well as its inverse matrix is equivalently substituted by W ₁ and W ₋₁. Furthermore, the WFRFT matrix W _α is the generalized form of Fourier matrix F and identity matrix I, and thus DFT is the special case of WFRFT.

In addition, W _α is a unitary matrix and obeys the following rules

$$\begin{array}{*{20}l} &\mathbf{W}_{-\alpha} = \mathbf{W}_{\alpha}^{-1} = \mathbf{W}_{\alpha}^{H} \end{array} $$

(6a)

$$\begin{array}{*{20}l} &\mathbf{W}_{\alpha+\beta} = \mathbf{W}_{\alpha}\mathbf{W}_{\beta} = \mathbf{W}_{\beta}\mathbf{W}_{\alpha} \end{array} $$

(6b)

3 HC system with spectrum precoding

As shown in Fig. 1, a hybrid carrier communication system based on WFRFT is depicted with two kinds of spectrum precoders to reduce the out-of-band radiation. Specially, the proposed HC system can be simplified to the OFDM scheme at α=0 and the DFT-s-OFDM scheme at α=1. The two precoders are applied in frequency domain, and the input signal belongs to 1−α fractional domain.

Fig. 1

Framework of HC system with two spectrum precoding structures

In projection precoding, the precoder G _χ is used at the transmitter without related decoding operation. Conversely, necessary information interaction is needed at the receiver to decode the precoder G _o in singular value decomposition (SVD) precoding. As revealed in [10] and [11], precoder G _χ achieves OOB radiation without BER constraint, which could be applied to low cost or fast response scenario. On the other hand, precoder G _o inhibits OOB power without BER performance degradation. Therefore, the required precoder can be selected according to the applications, specific requirements, and potential scenarios.

3.1 OOB power analysis

In the OFDM system, the data transmitted on the total K subcarriers are expressed as $\mathbf {d}=[d_{k_{1}},d_{k_{2}},\ldots,d_{k_{K}}]^{T}$. The spectrum at frequency f with spectrum precoder can be shown as

$$ S(f)=\mathbf{a}^{T}(f)\cdot \mathbf{G}\mathbf{d} $$

(7)

where $\mathbf {a}(f)=(a_{k_{1}}(f),a_{k_{2}}(f),\ldots,a_{k_{K}}(f))^{T}$. Specifically, a _k (f) denotes the spectral component contributed by the data on the kth subcarrier indexed by k∈κ={k ₁,k ₂,…k _K }, which is given as

$$ a_{k}(f)=T_{0}\cdot \frac{\sin\left(\pi(f-f_{k})T_{0}\right)}{\pi(f-f_{k})T_{0}} $$

(8)

where f _k and T ₀ denote the frequency of the kth subcarrier and the length of a symbol interval, respectively. Now, we consider the spectrum precoder G in the HC system and thus the transmitted data are pretreated by WFRFT matrix W _α . Thereby, the spectrum of the HC system can be expanded from (7), expressed as

$$ S_{\alpha}(f)=\mathbf{a}^{T}(f)\cdot \mathbf{G}\mathbf{d}_{\alpha}=\mathbf{a}^{T}(f)\cdot \mathbf{G}\mathbf{W}_{\alpha}\mathbf{d} $$

(9)

where d _α =W _α d is obtained in the HC system based on WFRFT. According to (9), the power spectrum of S _α (f) is calculated as

$$ \begin{aligned} P_{\alpha}(f) &=\frac{1}{T_{0}}E\left[|S_{\alpha}(f)|^{2}\right]\\ &=\frac{1}{T_{0}}\mathbf{a}^{T}(f) E\left\{\mathbf{G}\mathbf{W}_{\alpha} \mathbf{d} \mathbf{d}^{H} \mathbf{W}_{\alpha}^{H} \mathbf{G}^{H}\right\}\mathbf{a}^{*}(f)\\ &=\frac{1}{T_{0}}\|\mathbf{G}^{H}\mathbf{a}^{*}(f)\|^{2}_{2}=\frac{1}{T_{0}}E[|S(f)|^{2}]=P(f) \end{aligned} $$

(10)

where we assume that the data in d are memoryless, identically distributed, and thus subject to E[d d ^H]=I. As shown in (10), due to orthogonality and unitarity of W _α , the OOB emission of the proposed HC system remains the same as that of the OFDM scheme regardless of the change of WFRFT order α.

3.2 Precoder G _χ

In projection precoding, as given in (10), the power spectrum of the HC system with precoder G _χ is revealed as

$$ \begin{aligned} P_{\alpha,\text{pro}}(f)=P_{\text{pro}}(f)=\frac{1}{T_{0}}\left\|\mathbf{G}_{\chi}^{H}\mathbf{a}^{*}(f)\right\|^{2}_{2} \end{aligned} $$

(11)

This spectrum sculpting method firstly chooses several frequencies at each side of the frequency band and then reduces the spectrum of the considered frequencies to zero via data precoding, i.e.,

$$ S_{\alpha}(f_{i})=0,\ i=0,1,\ldots,M-1 $$

(12)

The set of the selected frequencies is defined as χ=f ₀,f ₁,…,f _M−1. Further, combining (9) and (12), the spectrum expression is rewritten as

$$ \mathbf{A}\mathbf{G}_{\chi}\mathbf{d}_{\alpha}=0 $$

(13)

where A=(a(f ₀),a(f ₁),…,a(f _M−1))^T is a matrix with M rows and K columns, shown as

$$ \mathbf{A}=\left(\begin{array}{cccc} a_{k_{1}}(f_{0}) & a_{k_{2}}(f_{0}) & \ldots & a_{k_{K}}(f_{0}) \\ a_{k_{1}}(f_{1}) & a_{k_{2}}(f_{1}) & \ldots & a_{k_{K}}(f_{1}) \\ \vdots & \vdots & \ddots & \vdots \\ a_{k_{1}}(f_{M-1}) & a_{k_{2}}(f_{M-1}) & \ldots & a_{k_{K}}(f_{M-1}) \\ \end{array}\right) $$

(14)

Particularly, G _χ d _α resides in the nullspace of A according to (13), which is shown as

$$ \begin{aligned} \mathcal{N}(\mathbf{A})=\left\{\mathbf{x}\in \mathbb{C}^{K}|\mathbf{A}\mathbf{x}=0\right\} \end{aligned} $$

(15)

where M≪K is the use case and the dimension of $\mathcal {N}(\mathbf {A})$ is K−M. The solving process of sculpting matrix G _χ equals to an optimization problem.

$$ \begin{aligned} &\mathbf{G}_{\chi}=\text{arg} \ \mathrm{\min} \|\mathbf{d}_{\alpha}-\mathbf{G}_{\chi}\mathbf{d}_{\alpha}\|_{2} &s.t. \ \mathbf{A}\mathbf{G}_{\chi}\mathbf{d}_{\alpha}=0 \end{aligned} $$

(16)

The constrained least-squares problem in (16) can be solved as

$$ \begin{aligned} &\hat{\mathbf{d}}_{\alpha}=\mathbf{G}_{\chi}\mathbf{d}_{\alpha}, &\mathbf{G}_{\chi}=\mathbf{I}-\mathbf{A}^{H}(\mathbf{A}\mathbf{A}^{H})^{-1}\mathbf{A}\doteq \mathbf{I}-\tilde{\mathbf{{A}}} \end{aligned} $$

(17)

The K×K spectrum precoder G _χ is only related to A as revealed in (17).

At the receiver, the received signal of precoder G _χ over the AWGN channel is given as

$$ \begin{aligned} \hat{\mathbf{d}}_{\mathbf{G}_{\chi}} &=\mathbf{W}_{-\alpha}\mathbf{W}_{1}\left(\mathbf{W}_{-1}\mathbf{G}_{\chi}\mathbf{W}_{\alpha}\mathbf{d}_{\mathbf{G}_{\chi}}+\nu\right)\\ &=\mathbf{W}_{-\alpha}\mathbf{G}_{\chi}\mathbf{W}_{\alpha}\mathbf{d}_{\mathbf{G}_{\chi}}+\mathbf{W}_{1-\alpha}\nu \end{aligned} $$

(18)

where ν denotes the additive white Gaussian noise (AWGN) with variance $\sigma _{\nu }^{2}$. The process of adding and removing cyclic prefix (CP) is simplified in (18). The average signal power at the receiver is calculated as

$$ \begin{aligned} \sigma^{2}_{\text{pro},s} &=E\left[\mathbf{W}_{-\alpha}\mathbf{G}_{\chi}\mathbf{W}_{\alpha}\mathbf{d}_{\mathbf{G}_{\chi}}\mathbf{d}_{\mathbf{G}_{\chi}}^{H}\mathbf{W}_{-\alpha}\mathbf{G}_{\chi}^{H}\mathbf{W}_{\alpha}\right]\\ &=\mathbf{W}_{-\alpha}\mathbf{G}_{\chi}\mathbf{G}_{\chi}^{H}\mathbf{W}_{\alpha} \end{aligned} $$

(19)

As a result of the properties of WFRFT, we can obtain $\sigma ^{2}_{\text {pro},\nu }=E[\mathbf {W}_{1-\alpha } \nu \nu ^{H} \mathbf {W}_{\alpha -1}]=\sigma ^{2}_{\nu }$. Therefore, the signal-to-noise ratio (SNR) of the proposed system with precoder G _χ is not same at different WFRFT orders.

The obtained data after spectrum precoding are $\bar {\mathbf {d}}_{\alpha }=\mathbf {G}_{\chi }\mathbf {d}_{\alpha }=\mathbf {G}_{\chi }\mathbf {W}_{\alpha }\mathbf {d}$, and the error introduced by G _χ is shown as

$$ \begin{aligned} \varepsilon_{\alpha} &=\mathbf{d}_{\alpha}-\bar{\mathbf{d}}_{\alpha}=(\mathbf{I}-\mathbf{G}_{\chi})\mathbf{W}_{\alpha}\mathbf{d}\\ &=\mathbf{A}^{H}(\mathbf{A}\mathbf{A}^{H})^{-1}\mathbf{A}\cdot\mathbf{W}_{\alpha}\mathbf{d}=\tilde{\mathbf{{A}}}\mathbf{W}_{\alpha}\mathbf{d} \end{aligned} $$

(20)

According to some mathematic calculations, the mean squared error (MSE) is shown as

$$ \begin{aligned} E\|\varepsilon_{\alpha}\|_{2}^{2} &=E\left[\mathbf{d}^{H}\mathbf{W}_{\alpha}^{H}\tilde{\mathbf{{A}}}^{H}\tilde{\mathbf{{A}}}\mathbf{W}_{\alpha}\mathbf{d}\right] =E\left[\mathbf{d}^{H}\mathbf{W}_{\alpha}^{H}\tilde{\mathbf{{A}}}\mathbf{W}_{\alpha}\mathbf{d}\right]\\ &\neq E[\mathbf{d}^{H}\tilde{\mathbf{{A}}}\mathbf{d}]=E\|\varepsilon\|_{2}^{2} \end{aligned} $$

(21)

As shown in (21), the relationship between $E\|\varepsilon _{\alpha }\|_{2}^{2}$ and $E\|\varepsilon \|_{2}^{2}$ depends on the selections of W _α and $\tilde {\mathbf {A}}$. The error produced by precoder G _χ is distinct at different WFRFT orders. Therefore, we can search the optimal WFRFT order to receive a relatively small error in regard to the projection precoder. Considering the unitarity of WFRFT matrix, we obtain $E\|\mathbf {d}_{\alpha }\|_{2}^{2}=E\|\mathbf {d}\|_{2}^{2}$. Since BER performance is related with the error vector magnitude (EVM) $E\|\varepsilon _{\alpha }\|_{2}^{2}/E\|\mathbf {d}_{\alpha }\|_{2}^{2}$ given in [10], the optimal order of the HC system over the AWGN channel is the one with least error introduced by precoder G _χ , expressed as

$$ \alpha_{\text{opt}}=\text{arg} \ \mathrm{\min_{\alpha}} \ E\|\varepsilon_{\alpha}\|_{2}^{2} $$

(22)

As shown in Fig. 2, for QPSK or 16QAM modulation, we can search an optimal or suboptimal α with least MSE due to the closed interval of WFRFT order in [0, 1]. However, we should note that the selected α also depends on the selections of zero frequencies numbers, which decides the matrix A and G _χ . Here, two zeroing subcarriers at both sides of the frequency band are considered.

Fig. 2

$\|\varepsilon _{\alpha }\|_{2}^{2}$ of precoder G _χ with traversal α (normalized by modulation order)

Further, when fading channels are taken into consideration, the received signal is expressed as

$$ \begin{aligned} \hat{\mathbf{d}}_{\mathbf{G}_{\chi}} &=\mathbf{W}_{-\alpha}\mathbf{Q}\mathbf{W}_{1}\left(\mathbf{H}\mathbf{W}_{-1}\mathbf{G}_{\chi}\mathbf{W}_{\alpha}\mathbf{d}_{\mathbf{G}_{\chi}}+\nu\right)\\ &=\mathbf{W}_{-\alpha}\mathbf{Q}\boldsymbol{\Lambda}\mathbf{G}_{\chi}\mathbf{W}_{\alpha}\mathbf{d}_{\mathbf{G}_{\chi}}+\mathbf{W}_{-\alpha}\mathbf{Q}\mathbf{W}_{1}\nu \end{aligned} $$

(23)

where the channel matrix H could be diagonalized by Fourier matrix, i.e., Λ=F H F ^H=W ₁ H W ₋₁, and Q denotes the channel equalization matrix. In this case, the optimal order can be selected according to the analysis of error introduced by G _χ , channel matrix H, and equalization matrix Q.

3.3 Precoder G _o

Compared to G _χ , the precoder G _o given in [11], obtained from the singular-value decomposition of A, achieves orthogonality at the cost of higher complexity and spectral efficiency loss. The matrix A can be factorized as

$$ \mathbf{A}=\mathbf{U}\mathbf{\Phi} \mathbf{V}^{H} $$

(24)

where Φ is a diagonal M×K matrix with the singular values of A on the diagonal. U and V are two unitary matrices with the size M×M and K×K, respectively. Thus, there is U ^H U=I _M and V ^H V=I _K . V is constituted by K column vectors, i.e., v ₀,v ₁,…,v _K−1. An orthogonal basis of $\mathcal {N}(\mathbf {A})$ in (15) is obtained via the collection of the last K−M columns of V, expressed as

$$ \mathbf{G}_{o}=\left[\mathbf{v}_{M},\mathbf{v}_{M+1},\ldots,\mathbf{v}_{K-2},\mathbf{v}_{K-1}\right] $$

(25)

From (25), we can see that G _o is a K×(K−M) matrix and there is

$$ \mathbf{G}_{o}^{H}\mathbf{G}_{o}=\mathbf{I}_{K-M} $$

(26)

Thus, $\mathbf {G}_{o}^{H}$ could be considered as a decoding matrix at the receiver. In addition, the relationship between precoder G _χ and G _o is given in [11],

$$ \mathbf{G}_{\chi}=\mathbf{G}_{o}\mathbf{G}_{o}^{H} $$

(27)

Combining (26) and (27), we get

$$ \begin{aligned} \mathbf{G}_{\chi}\mathbf{G}_{\chi}^{H}=\mathbf{G}_{o}\mathbf{G}_{o}^{H}\mathbf{G}_{o}\mathbf{G}_{o}^{H}=\mathbf{G}_{o}\mathbf{G}_{o}^{H} \end{aligned} $$

(28)

As shown from (9) and (10), the OOB radiation performance of the hybrid carrier system with precoder G _o is calculated as

$$ \begin{aligned} P_{\alpha,\text{SVD}}(f) &=\frac{1}{T_{0}}E\left[|S_{\alpha,\text{SVD}}(f)|^{2}\right]\\ &=\frac{1}{T_{0}}\mathbf{a}^{T}(f) E\left\{\mathbf{G}_{o}\mathbf{W}_{\alpha} \mathbf{d} \mathbf{d}^{H} \mathbf{W}_{\alpha}^{H} \mathbf{G}_{o}^{H}\right\}\mathbf{a}^{*}(f)\\ &=\frac{1}{T_{0}}\mathbf{a}^{T}(f) \mathbf{G}_{o}\mathbf{G}_{o}^{H}\mathbf{a}^{*}(f)=\frac{1}{T_{0}}\left\|\mathbf{G}_{o}^{H}\mathbf{a}^{*}(f)\right\|^{2}_{2}\\ &=\frac{1}{T_{0}}\mathbf{a}^{T}(f) \mathbf{G}_{\chi}\mathbf{G}_{\chi}^{H}\mathbf{a}^{*}(f)=\frac{1}{T_{0}}\left\|\mathbf{G}_{\chi}^{H}\mathbf{a}^{*}(f)\right\|^{2}_{2} \end{aligned} $$

(29)

Then, the OOB radiation performance of precoder G _χ and G _o of the HC scheme based on WFRFT is summarized as

$$ \begin{aligned} P_{\text{pro}}(f)=P_{\alpha,\text{pro}}(f)=P_{\text{SVD}}(f)=P_{\alpha,\text{SVD}}(f) \end{aligned} $$

(30)

Therefore, considering the same matrix A, the two frequency domain precoders achieve identical OOB attenuations in the HC system in spite of different WFRFT order α.

Distinguished from precoder G _χ , the orthogonality is maintained via the decoder at the receiver in SVD precoding revealed in (26). The received signal after AWGN channel is expressed as

$$ \begin{aligned} \hat{\mathbf{d}}_{\mathbf{G}_{o}} &=\mathbf{W}_{-\alpha}\mathbf{G}_{o}^{H}\mathbf{W}_{1}\left(\mathbf{W}_{-1}\mathbf{G}_{o}\mathbf{W}_{\alpha}\mathbf{d}_{\mathbf{G}_{o}}+\nu\right)\\ &=\mathbf{d}_{\mathbf{G}_{o}}+\mathbf{W}_{-\alpha}\mathbf{G}_{o}^{H}\mathbf{W}_{1}\nu \end{aligned} $$

(31)

The signal power $\sigma ^{2}_{\text {SVD},s}=\sigma ^{2}_{s}$ is unrelated with the WFRFT order and precoder G _o . The noise power is shown as

$$ \begin{aligned} \sigma^{2}_{\text{SVD},\nu} =E\left[\mathbf{W}_{-\alpha}\mathbf{G}_{o}^{H}\mathbf{W}_{1}\nu\nu^{H}\mathbf{W}_{-1}\mathbf{G}_{o}\mathbf{W}_{\alpha}\right] =\sigma^{2}_{\nu} \end{aligned} $$

(32)

Therefore, the SNR of the proposed system with precoder G _o equals that of the OFDM system. Furthermore, the same BER performance can be obtained in the hybrid carrier system (including the traditional OFDM system) with different WFRFT orders over the AWGN channel.

Similarly, we apply frequency domain equalization (FDE) to offset the channel loss at the receiver if the fading channel is premeditated. The received signal is expressed as

$$ \begin{aligned} \hat{\mathbf{d}}_{\mathbf{G}_{o}} &=\mathbf{W}_{-\alpha}\mathbf{G}_{o}^{H}\mathbf{Q}\mathbf{W}_{1}\left(\mathbf{H}\mathbf{W}_{-1}\mathbf{G}_{o}\mathbf{W}_{\alpha}\mathbf{d}_{\mathbf{G}_{o}}+\nu\right)\\ &=\mathbf{W}_{-\alpha}\mathbf{G}_{o}^{H}\mathbf{Q}\boldsymbol{\Lambda}\mathbf{G}_{o}\mathbf{W}_{\alpha}\mathbf{d}_{\mathbf{G}_{o}}+ \mathbf{W}_{-\alpha}\mathbf{G}_{o}^{H}\mathbf{Q}\mathbf{W}_{1}\nu \end{aligned} $$

(33)

where the optimal SNR depends on the selections of W _α , G _o , and Q for a definite H.

4 An application in channel estimation

4.1 OFDM scheme

In classical OFDM scheme, the certain subcarriers are selected as the pilot symbols known at the receiver. However, the pilot information is corrupted at the transmitter in spectrum precoding system. In projection precoding, the precoding error is introduced by precoder G _χ as given in (20). In SVD precoding, the spectral efficiency is reduced as a result of precoder G _o . Therefore, the effective way to realize channel estimate at the receiver is to protect the pilot subcarriers from the damage of the precoder G.

A precoder structure for channel estimator at the transceiver is proposed in [12], where two precoding methods of pilot subcarriers are revealed. In precoder A, the pilot information is not precoded and thus protected from corruption. Compared to the structure A, the precoder B realizes the optimal spectrum shaping with lower EVM of data information at the cost of interfered pilot information.

The subcarriers of the OFDM system can be divided into two parts, one for data-carrying subcarriers and another for pilot-carrying subcarriers. The total K=I+P subcarriers is partitioned into two disjoint sets κ _i =k ₀,k ₁,…k _I−1 and κ _p =k _I ,k _I+1,…k _K−1, where κ _i and κ _p denote the subcarrier index of data and pilot symbols respectively.

In precoder A, the object is to achieve the unchanged data modulation on the pilot-carrying subcarriers, i.e., $\bar {\mathbf {{d}}}_{p}=\mathbf {d}_{p}$. The optimization problem can be expressed as

$$ \begin{aligned} \mathbf{A}\bar{\mathbf{{d}}} = \left[\begin{array}{cc} \mathbf{A}_{i} & \mathbf{A}_{p} \end{array}\right] \left[\begin{array}{c} \bar{\mathbf{{d}}}_{i} \\ \bar{\mathbf{{d}}}_{p} \end{array}\right] =0\\ s.t. \left[\begin{array}{c} \bar{\mathbf{{d}}}_{i} \\ \bar{\mathbf{{d}}}_{p} \end{array}\right] =\left[\begin{array}{cc} \mathbf{G}_{i} & \mathbf{G}_{p} \\ \mathbf{0} & \mathbf{I}_{p} \end{array}\right] \left[\begin{array}{c} \mathbf{d}_{i} \\ \mathbf{d}_{p} \end{array}\right] \end{aligned} $$

(34)

where A _i is denoted as the set of I columns of A and A _p is constituted by the remaining P columns of A. d _i denotes the data information, and d _p represents the pilot information. To find the closest approximation $\bar {\mathbf {{d}}}_{i}$ of d _i , the constraint problem in (34) becomes

$$ \begin{aligned} \bar{\mathbf{{d}}}_{i}=\text{arg} \ \mathrm{\min} \|\mathbf{d}_{i}-\bar{\mathbf{{d}}}_{i}\|_{2} \quad s.t. \quad \mathbf{A}_{i}\bar{\mathbf{{d}}}_{i}+\mathbf{A}_{p}\mathbf{d}_{p}=0 \end{aligned} $$

(35)

The constrained least-squares problem can be solved by introducing a Lagrange vector, same as the way in (16). Therefore, the solution of (35) is given as

$$ \begin{aligned} \mathbf{G}_{i}=\mathbf{I}-\mathbf{A}_{i}^{H}\left(\mathbf{A}_{i}\mathbf{A}_{i}^{H}\right)^{-1}\mathbf{A}_{i}\doteq \mathbf{I}-\tilde{\mathbf{{A}}}_{i} \end{aligned} $$

(36)

$$ \begin{aligned} \mathbf{G}_{p}=-\mathbf{A}_{i}^{H}\left(\mathbf{A}_{i}\mathbf{A}_{i}^{H}\right)^{-1}\mathbf{A}_{p} \end{aligned} $$

(37)

Although the undamaged pilot symbols are obtained on the basis of precoder A, the transmitted data q before IFFT suffers the interference from not only the precoding but also the pilot data, which can be expressed as

$$ \begin{aligned} \mathbf{q} &=\mathbf{G}_{i}\mathbf{d}_{i}+\mathbf{G}_{p}\mathbf{d}_{p}\\ &=\mathbf{d}_{i}-\mathbf{A}_{i}^{H}\left(\mathbf{A}_{i}\mathbf{A}_{i}^{H}\right)^{-1}\mathbf{A}_{i}\mathbf{d}_{i}-\mathbf{A}_{i}^{H}\left(\mathbf{A}_{i}\mathbf{A}_{i}^{H}\right)^{-1}\mathbf{A}_{p}\mathbf{d}_{p} \end{aligned} $$

(38)

In this case, the precoding is transparent to the pilots, but the information data is interfered in a larger scale. Thus, we seek a precoder without interference from pilots at the price of slight distortion of pilots. Then, the constraints in (34) is reduced to

$$ \begin{aligned} \mathbf{A}\bar{\mathbf{{d}}} = \left[\begin{array}{cc} \mathbf{A}_{i} & \mathbf{A}_{p} \end{array}\right] \left[\begin{array}{cc} \hat{\mathbf{{G}}}_{i} & \mathbf{0} \\ \mathbf{0} & \hat{\mathbf{{G}}}_{p} \end{array}\right] \left[\begin{array}{c} \mathbf{d}_{i} \\ \mathbf{d}_{p} \end{array}\right] =0 \end{aligned} $$

(39)

where

$$ \begin{aligned} \hat{\mathbf{{q}}}=\hat{\mathbf{{G}}}_{i}\mathbf{d}_{i} \end{aligned} $$

(40)

and the solution of (39) is given as

$$ \begin{aligned} \hat{\mathbf{{G}}}_{i}=\mathbf{I}-\mathbf{A}_{i}^{H}\left(\mathbf{A}_{i}\mathbf{A}_{i}^{H}\right)^{-1}\mathbf{A}_{i}=\mathbf{G}_{i} \end{aligned} $$

(41)

$$ \begin{aligned} \hat{\mathbf{{G}}}_{p}=\mathbf{I}-\mathbf{A}_{p}^{H}\left(\mathbf{A}_{p}\mathbf{A}_{p}^{H}\right)^{-1}\mathbf{A}_{p}\doteq \mathbf{I}-\tilde{\mathbf{{A}}}_{p} \end{aligned} $$

(42)

The data precoding matrix of the two precoder structures is the same as the precoder G _χ in (17) in this paper and thus they hold the same out of band power suppression performance.

Although the pilot of precoder A is completely received without information loss, its BER performance is related with the distribution of pilot subcarriers, and the received signal is interfered by the pilot shown in (38). On the contrary, the second precoding structure could also achieve the traditional channel estimate at the receiver in spite of slight deviation due to the spectrum shaping of pilot-carrying subcarriers at the transmitter. In addition, the precoder $\hat {\mathbf {{G}}}_{p}$ achieves same BER performance free from the influence of the pilot positions. As given in [12], the EVM of the transmitted data of the two structures are calculated as

$$ \begin{aligned} \text{EVM}_{A}>\text{EVM}_{B}=\sqrt{M/I}>\sqrt{M/K} \end{aligned} $$

(43)

However, the BER performance should be analyzed on the basis of introduced pilot interference and estimation error. The EVM in (43) does not count the BER decline due to the channel estimation error. In structure B, as a result of the spectrum precoding of pilots, the performance degradation still exists despite the perfect channel estimation. Therefore, we attempt to utilize the WFRFT-based structure to reduce the spectrum shaping error introduced by $\hat {\mathbf {{G}}}_{p}$.

4.2 HC scheme with WFRFT-based pilot structure

The framework of the hybrid carrier scheme with spectrum precoding and channel estimation is put forward in Fig. 3. The proposed scheme is a combination of the channel estimation system in [12] and the hybrid carrier spectrum precoding system in Section 3. The proposed hybrid carrier system in Fig. 3 could achieve different WFRFT precoding orders of data and pilot. Nevertheless, the out-of-band power suppression performance remains unchanged due to irrelevance between OOB and WFRFT order proven in (30). As given in (22), a relatively optimal WFRFT order α could be picked out according to the spectrum precoding G. Considering this principle, the pilot precoder $\hat {\mathbf {{G}}}_{p}$ in (42) is processed by β order WFRFT, expressed as

$$ \begin{aligned} {} \mathbf{G}_{\beta,p} &=\mathbf{W}_{\beta}\hat{\mathbf{{G}}}_{p}\mathbf{W}_{-\beta} =\mathbf{W}_{\beta}\left(\!\mathbf{I}\,-\,\mathbf{A}_{p}^{H}\left(\mathbf{A}_{p}\mathbf{A}_{p}^{H}\right)^{-1}\mathbf{A}_{p}\right)\mathbf{W}_{-\beta}\\ &=\mathbf{I}-\mathbf{W}_{\beta}\tilde{\mathbf{{A}}}_{p}\mathbf{W}_{-\beta} \end{aligned} $$

(44)

Fig. 3

Framework of HC scheme with WFRFT-based pilot structure and channel estimation

where (44) degrades to (42) at β=0. The deviation of the pilots introduced by G _β,p is shown as

$$ \begin{aligned} \varepsilon_{\beta,p}=\mathbf{d}_{p}-\mathbf{G}_{\beta,p}\mathbf{d}_{p}=\mathbf{W}_{\beta}\tilde{\mathbf{{A}}}_{p}\mathbf{W}_{-\beta}\mathbf{d}_{p} \end{aligned} $$

(45)

According to the expression in (21), the mean squared error can be calculated as

$$ \begin{aligned} E\|\varepsilon_{\beta,p}\|_{2}^{2} &=E\left[\mathbf{d}^{H}\mathbf{W}_{-\beta}^{H}\tilde{\mathbf{{A}}}^{H}_{p}\mathbf{W}_{\beta}^{H}\mathbf{W}_{\beta}\tilde{\mathbf{{A}}}_{p}\mathbf{W}_{-\beta}\mathbf{d}\right]\\ &=E\left[\mathbf{d}^{H}\mathbf{W}_{-\beta}^{H}\tilde{\mathbf{{A}}}^{H}_{p}\tilde{\mathbf{{A}}}_{p}\mathbf{W}_{-\beta}\mathbf{d}\right]\\ &=E\left[\mathbf{d}^{H}\mathbf{W}_{-\beta}^{H}\tilde{\mathbf{{A}}}_{p}\mathbf{W}_{-\beta}\mathbf{d}\right] \end{aligned} $$

(46)

Since the A _p is the set of P columns of A, there is still $\tilde {\mathbf {{A}}}^{H}_{p}\tilde {\mathbf {{A}}}_{p}=\tilde {\mathbf {{A}}}_{p}$. Compared to the OFDM scheme at α=0, a smaller MSE at fractional order $\breve {\beta }$ can be obtained through flexible order selections in different modulations shown in Fig. 2. Then, (46) can be extended as

$$ \begin{aligned} E\left\|\varepsilon_{\breve{\beta},p}\right\|_{2}^{2} < E\left\|\varepsilon_{p}\right\|_{2}^{2} \end{aligned} $$

(47)

where $\breve {\beta }$ belongs to a set in (0, 1). Therefore, the channel estimation error of the method in [12] can be reduced through the precoder G _β,p given in (44). At the receiver, the obtained pilot signal can be revealed as

$$ \begin{aligned} \mathbf{r}_{p} &=\mathbf{W}_{1}\left(\mathbf{H}\mathbf{W}_{-1}\mathbf{W}_{-\beta}\mathbf{G}_{\beta,p}\mathbf{W}_{\beta}\mathbf{d}_{p}+\nu\right)\\ &=\boldsymbol{\Lambda}\mathbf{W}_{-\beta}\mathbf{G}_{\beta,p}\mathbf{W}_{\beta}\mathbf{d}_{p}+\mathbf{W}_{1}\nu \end{aligned} $$

(48)

Here, the traditional channel estimation methods can be employed to reckon the channel matrix Λ. If the zero forcing estimation is employed, the channel state information can be calculated as

$$ \bar{\boldsymbol{\Lambda}}=\frac{\mathbf{r}_{p}}{\mathbf{d}_{p}} $$

(49)

5 Numerical results and discussion

In this section, OOB radiation and BER and PAPR simulation results are revealed with various WFRFT orders to display the performance improvement of the HC scheme in comparison with the OFDM system.

5.1 OOB power results

Simulation results show the OOB radiation with two and four zeroing subcarriers at both sides of the frequency band. As verified in (30), same OOB power leakage is obtained due to the identical A of G _χ and G _o . In addition, the WFRFT orders are unrelated with the OOB power suppression performance. Therefore, we only give the OOB performance of precoder G _χ at α=0.7 in Fig. 4. Further, lower OOB radiation could be achieved if more nulling frequencies M are considered. However, the bigger M obviously means the increase of complexity due to the increased rows of A and possibly leads to deteriorated system performance.

Fig. 4

OOB of precoder G _χ with different M at α=0.7

5.2 BER results

For the projection precoding and SVD precoding methods, we set 200 subcarriers in one frequency band and two zeroing subcarriers at both sides of the frequency band. As shown in Fig. 5, α=0.7 is selected as a comparatively optimal WFRFT order with best BER performance in the AWGN channel model, much better than that in the OFDM scheme at α=0 and the DFT-s-OFDM scheme at α=1. The optimal order mainly depends on the selection of χ, and the value of M/K is revealed in (21). In contrast, the precoder G _o remains same performance at different WFRFT orders in the HC scheme due to the quadrature decoder at the receiver given in (26).

Fig. 5

BERs over the AWGN channel with QPSK modulation

Furthermore, the relatively optimal order can also be picked out in Fig. 6 considering the fading channel with minimum mean squared error equalizer at the receiver. The tap delay and power of the fading channel are [0 50 110 170 290 310] ns and [0 −3 −10 −18 −26 −32] dB, respectively. The maximum Doppler shift is 33 Hz versus 10 Mcps chip rate. Compared to the OFDM and DFT-s-OFDM systems, we can also obtain better BER performance in the HC scheme no matter with the precoder G _χ or G _o . In addition, the optimal performance of precoder G _χ in the HC scheme approaches to that of precoder G _o employed in the OFDM scheme.

Fig. 6

BERs over fading channels (QPSK of G _χ and BPSK of G _o )

In Fig. 7, BER performances of the HC system with different WFRFT processing orders of data and pilot are given. To illustrate the smaller precoding error of the pilot given in (45), we employ the same channel estimation method and here, ZF estimation revealed in (49) is selected. The channel information and equalization strategy are same as that in Fig. 6. The BER performances at α=0.7 with different pilot precoding orders are totally superior to that at α=0. On the one hand, the fractional order α indeed appears a better BER performance in the proposed DFT-based hybrid carrier system. On the other hand, we set enough pilot subcarriers to implement an accurate channel estimation at the receiver and aim to prove the superiority of the proposed WFRFT-based pilot structure at the transmitter, so there is little performance divergence between different β. Specifically, we can obtain a relatively optimal pilot precoding in fractional order $\breve {\beta }$ at α=0 or α=0.7, which indicates that less precoding error can be obtained in the pilot structure of (44).

Fig. 7

BERs of the proposed WFRFT-based pilot structure with different WFRFT orders

5.3 PAPR results

As shown in Fig. 8, the averaged PAPR of the OFDM system at α=0 with precoder G _χ and G _o is the same after 50 times of simulation. However, there are PAPR performance divergences of precoder G _χ and G _o at α=0.7 after the same times of simulation. Compared to precoder G _o , lower PAPR results are obtained with precoder G _χ at α=0.7, but the PAPR performance in the HC scheme with both precoders is still superior to that in the OFDM system. Therefore, combined BER and PAPR performance advantages can be gained in the HC scheme in comparison with the results in the OFDM system at α=0.

Fig. 8

PAPRs of HC system after spectrum precoding

In this paper, compared to the HC system itself, joint PAPR and sidelobe suppression is not achieved because there is no PAPR reduction methods employed as the way in [23, 24]. However, a joint PAPR and sidelobe reduction method is put forward in [30], which proves the feasibility in the HC system. Considering the strong compatibility of the HC system, we could combine spectrum precoding with PAPR suppression to achieve a joint suppression in future work.

5.4 Scenario analysis

Combined signal processing optimization via balancing conflicting metrics is considered as a feature in 5G. In this paper, better performances could be obtained through the spread of the OFDM scheme into the HC scheme. The combined multi-object optimization including BER, OOB, and PAPR is achieved in the HC system with spectrum precoders. The obtained beneficial results meet the requirements of 5G with combined multi-object and multiscenario.

Specifically, the projection precoding can be reckoned as an OOB suppression method without BER constraint (with BER loss). Therefore, it can be employed in those scenarios such as low-speed Internet of things (IoT), whose feathers include low cost, fast response, narrow band, and insensitivity to BER degradation. In contrast, the SVD precoding realizes OOB reduction without BER performance loss. It could be claimed in precision scenes with a higher tolerance of complexity and requirement of reliability, but this precoder still suffers the cost of spectral efficiency and needs the information interaction at the transceiver.

6 Conclusions

In this paper, compared to the OFDM scheme employing spectrum precoder to reduce the OOB radiation, a combined multi-object gain is achieved in the HC scheme based on WFRFT. The proposed scheme achieves better BER and PAPR performances without OOB change and remarkable complexity increase. The relatively optimal order can be singled out from the set of WFRFT order considering different OOB suppression parameters and channel conditions. In addition, a novel pilot structure based on WFRFT is proposed to obtain better BER performance after channel estimation.