Interference alignment and power allocation for the MIMO X channels with multiple layers of relays

Abstract

In this paper, we study a system consisting of a single source, a single destination, and multiple layers of parallel relays. The system is considered as a combination of broadcast channels, multiple input multiple output (MIMO) X channels, and multiple-access channels. Interference alignment (IA) is used throughout the whole system, and perfect channel state information (CSI) is assumed available at all the transmitters and receivers. Iterative algorithms are applied to find the proper transmit and receive matrices in the broadcast channels and multiple-access channels. For the MIMO X channels, based on IA technique, we design the transmit and receive beamforming matrices. Finally, in order to maximize the total transmission rate of the whole system, we optimize the power allocation of each transmitter.

1 Introduction

In recent years, due to the rapid increase of mobile data traffic and the demand for ubiquitous connectivity, wireless communication has become a hot research area. Two key problems in this area are how to increase the transmission rate and how to deal with the large path loss of long distance transmission.

Great attention has been put on relay networks for long distance transmission. In a relay network, a number of nodes are assigned to help the source to forward signals to the destination, in order to provide reliable long distance transmission [1, 2]. Different kinds of relay networks have been studied. A single-input-single-output network is studied in [3]. In [4, 5], the authors mentioned two models of single-layer relay networks, including single-antenna relays and multi-antenna relays. In [6], a multi-layer relay network was proposed, in which there is a single antenna at each relay. Motivated by these studies, we use relay networks to solve the path loss problem in the long distance transmission.

In our work, we consider a network with multiple layers of relays, a single source, and a single destination. In order to increase the overall transmission rate, we assume that in each layer, there are multiple relays (we denote the number of relays can be used in layer n as K _n ). We assume that the relays in this network are operated in the half-duplex mode and equipped with multiple antennas.

Although applying multiple relays in each layer can improve the performance, the improvement is significantly limited by interference signals. However, since each node are equipped with multiple antennas, we can apply interference alignment (IA) ([7–9]), which is an efficient approach to degrade the effect of interferences. IA is a method to align interferences to a certain space, so that they can be distinguished from the desired signal. IA is considered in different kinds of networks to maximize interference-free dimensions remaining for the desired signal. In [10, 11], IA is used to analyze the communication over multiple-input-multiple-output (MIMO) X channel. Suh et al. and Ntranos et al. studied IA in cellular networks [12, 13].

Through providing additional dimension of space for transmitting signals, multi-antenna systems are believed to be an efficient method to substantially increase the overall throughput of wireless communication [14, 15]. MIMO systems have much higher capacity than single-input-single-output (SISO) systems. Foschini and Gans and Zheng and TSe [16, 17] stated that the multiplexing gain (MG) of point-to-point MIMO systems with M transmit antennas and N receive antennas is min{M, N}. MIMO broadcast channel was studied in [18]. In [19], for a two-user MIMO Gaussian interference channel with M ₁ and M ₂ antennas at transmitters 1 and 2, respectively, and N ₁ and N ₂ antennas at the corresponding receivers, the degrees of freedom (DoF) is min {M ₁+M ₂,N ₁+N ₂,m a x{M ₁,N ₂},m a x{M ₂,N ₁}}, with perfect channel knowledge at all transmitters and receivers. In our work, we employ multiple antennas at each relay with interference alignment to increase the DoF, and hence to increase the data rate of our system.

In our previous work [20], we consider a two-layer relay network, where there are equal number of relays in each layer. The communication channel between the two layer of relays is MIMO interference channel, in which each first layer relay selects one distinct second layer relay to transmit signals. In this paper, we extend it to a more general case, in which there are multiple layers of relays and the numbers of relays in each layer can be different. In this case, the distinct selection will require the equalization of the number of relays selected in each layer. As a result, the number of relays used in each layer is min{K ₁,K ₂,…}, assuming K _n relays are available in layer n. In this way, the overall throughput of the system will be reduced due to the under-utilization of the relays in some layers. Moreover, the selection process requires exhaustive search, which leads to a high complexity. To deal with this problem, in this work, we consider the transmission between the relays in different layers as MIMO X channel.

There have been lots of researches on the X channel, which is a kind of network that each transmitter has a distinct signal for each receiver. Using the X channel may reduce the complexity of exhaustive search and release the constraint of equal relay numbers. The DoF of M×N MIMO channel with a single antenna at each transmitter and receiver has been shown to be upper bounded by $\frac {MN}{M+N-1}$ in [21]. In [22, 23], a two-user MIMO X channel was studied. In our work, we consider a K ₁×K ₂ MIMO X channel with M antennas at each node. This is a communication channel with K ₁ transmitters and K ₂ receivers, where each transmitter has a distinct signal to each of the K ₂ receivers. In this way, we can take full use of all the available relays in each layer and increase the overall throughput.

The main results of the paper are as follows.

• 1. We consider a system with multiple layers of relays, and each of the relays has multiple antennas. We apply the iterative algorithms to find the proper transmit and receive matrices in the broadcast channels and multiple-access channels under the condition of IA.

• 2. For the K _n ×K _n+1 MIMO X channel, we propose a construction of an interference alignment scheme.

• 3. We evaluate the system with a layer level power constraint and optimize the power allocation of all nodes.

• 4. We compare the performance of our system with two benchmark systems: (i) a benchmark system with interference channel as shown in our previous work [20] and (ii) a benchmark system with a single relay in each layer. The simulation results show that the performance of the system in this paper is better than the benchmark systems in terms of total transmission rate.

The remainder of this paper is organized as follows. Section 2 provides an overview of the system model. We show the analysis of the model and the process of perfectly aligning the interferences of each part of the system in Section3. In section4, the power allocation problem is established and solved. Performance evaluation is in Section5. Finally, Section6 concludes the paper.

Notations: All boldface letters indicate matrices (upper case) or vectors (low case). A ^H denotes the conjugate transpose of A, and A ^† is pseudo-inverse of A. The null space of the matrix A is denoted by null(A). The subspace spanned by columns of A is represented by span(A). The identity matrix is represented by I. The det(A) and tr(A) are the determinant and trace of the matrix A, respectively.

2 System model

We consider a wireless network that includes a single source, N layers of parallel relays and a single destination. There are K _n relays available in layer n. It is assumed that every relay in our system adopts the decode-and-forward (DF) protocol and each node operates in the half-duplex mode. We denote the source as layer 0 and the destination as layer (N+1). It is assumed that the nodes in the nth layer can only receive signals from the nodes in the (n−1)th layer and transmit signals to the nodes in the (n+1)th layer. There is no direct link from the (n−1)th layer nodes to the (n+1)th layer nodes, since the distance between the (n−1)th layer and the (n+1)th layer is too long and there may be some obstacles between them. We denote the source as S, destination as D, and the ith relay in the nth layer (n∈{1,2,…,N}) as ${R^{n}_{i}}$. Besides, we assume that there are N+1 time slots for the end-to-end transmission. The system model is shown in Fig. 1.

Fig. 1

A single-source single-destination multi-layer relay MIMO network. Illustration of a single-source single-destination multi-layer relay MIMO network

In the first time slot, the source transmits to the first layer relays using broadcasting protocol. In the nth (1<n<N+1) time slot, the relays in the (n−1)th layer decode and forward signals to the nth layer relays. We consider a K _n−1×K _n X channel in the transmission. Finally, the Nth layer relays decode and forward the signals to the destination using multiple-accessing, and the destination could directly decode the signals. The schemes of IA and zero-forcing are used to precode and decode the signals, respectively.

We assume that there are K _n relays available in layer n. The communication channel between the nth layer relays and the (n+1)th layer relays is a K _n ×K _n+1 MIMO X channel where each of the K _n transmitters sends a distinct message to each of the K _n+1 receivers. For the cases that (i) K _n =1 and K _n+1≠1, (ii) K _n ≠1 and K _n+1=1, and (iii) K _n =1 and K _n+1=1, the communication channel between them is a broadcasting channel, multiple-access channel and point-to-point channel, respectively.

We assume each relay is equipped with M antennas. Let the number of antennas at the source and the destination to be M _s and M _d , respectively. We denote $\mathbf {H}_{sj} \in \mathbb {C}^{M \times M_{s}}$ as the channel matrix between the source to the jth relay in the first layer. The channel matrix between the ith relay in the Nth layer to the destination is denoted as H _id , where $\mathbf {H}_{id} \in \mathbb {C}^{M_{d} \times M}$. In the middle part of the system, we denote the channel matrix between the ith relay in the nth layer to the jth relay in the (n+1)th layer as $\mathbf {H}^{n}_{ij} \in \mathbb {C}^{M \times M}$. Finally, we assume that the entries of H _sj , $\mathbf {H}^{n}_{ij}$, and H _id are i.i.d. complex Gaussian random variables with zero-mean and the total power constraint of each layer is P.

3 Analysis of interference alignment

We divide the analysis of the system into three parts. In the first part, we consider K _n ×K _n+1 MIMO X channels (for the cases K _n >1 and K _n+1>1). In the second part, we consider the broadcast channels, which are (i) from the source to the first layer relays, if K ₁>1; (ii) from the nth layer to the (n+1)th layer, if K _n =1 and K _n+1>1. In the third part, we consider the multiple-access channels, which are (i) from the Nth layer relays to the destination, if K _N >1; (ii) from the nth layer to the (n+1)th layer, if K _n >1 and K _n+1=1. For the point-to-point MIMO channels (the case K _n =1 and K _n+1=1), since there is no interference signal to be aligned, we will not consider the point-to-point MIMO channels in this section.

3.1 K _n ×K _n+1 MIMO X channel

3.1.1 Interference alignment scheme

Now we consider the K _n ×K _n+1 MIMO X channel between layer n and layer n+1 as shown in Fig. 2. We denote the ith relay in layer n as ${R^{n}_{i}}$ (i=1,2,…,K _n ). If K _n or K _n+1 equals one, the problem reduces to a broadcasting or a multiple-accessing problem to be solved in Sections 3.2 and 3.3 In this section, we focus on the case that K _n >1 and K _n+1>1.

Fig. 2

A K _n ×K _n+1 MIMO X channel. Illustration of a K _n ×K _n+1 MIMO X channel

We assume the relay ${R^{n}_{i}}$ is supposed to transmit message $\mathbf {x}^{n}_{ij}$ to the relay $R^{n+1}_{j}$, where $\mathbf {x}^{n}_{ij} \in \mathbb {C}^{T_{sig} \times 1}$ is a vector with unit power of each component (i.e., $\mathrm {E}\left (\mathbf {x}^{n}_{ij}{\mathbf {x}^{n}}^{H}_{ij}\right) = \mathbf {I}_{T_{\text {sig}}}$). T _sig denotes the degree of freedom achieved by each message.

We use $\tilde {\mathbf {V}}^{n}_{ij} \in \mathbb {C}^{M \times T_{\text {sig}}}$ to denote the transmit beamforming matrix, which is to map the T _sig symbols in $\mathbf {x}^{n}_{ij}$ to M antennas by the transmitter ${R^{n}_{i}}$. $\tilde {\mathbf {V}}^{n}_{ij}$ is a full rank matrix with orthonormal columns. We rewrite the transmit beamforming matrix as $\tilde {\mathbf {V}}^{n}_{ij} = \mathbf {Q}^{n}\mathbf {V}^{n}_{ij}$, where the matrix $\mathbf {Q}^{n} \in \mathbb {C}^{M \times T}$ is also with orthonormal columns. The matrix Q ⁿ is used to (i) confine the transmit signals from each transmitter to a T-dimensional subspace; (ii) exploit the null spaces of the channel matrices to achieve higher multiplexing gain [10]. Thus, we call Q ⁿ as space confining matrix. Since both $\tilde {\mathbf {V}}^{n}_{ij}$ and Q ⁿ are matrices with orthonormal columns and $\tilde {\mathbf {V}}^{n}_{ij} = \mathbf {Q}^{n}\mathbf {V}^{n}_{ij}$, the columns of $\mathbf {V}^{n}_{ij} \in \mathbb {C}^{T \times T_{sig}}$ are also normalized and orthogonal with each other. On the MIMO X channel, each transmitter has a distinct message for each receiver, and there are K _n+1 receivers on this MIMO X channel. The signal vector transmitted by transmitter ${R^{n}_{i}}$ is thus:

$$\mathbf{t}^{n}_{i} = \sum_{j=1}^{K_{n+1}}\tilde{\mathbf{V}}^{n}_{ij}\mathbf{P}^{n}_{ij}\mathbf{x}^{n}_{ij} = \sum_{j=1}^{K_{n+1}}\mathbf{Q}^{n}\mathbf{V}^{n}_{ij}\mathbf{P}^{n}_{ij}\mathbf{x}^{n}_{ij}, $$

where $\mathbf {P}^{n}_{ij} \in \mathbb {C}^{T_{\text {sig}} \times T_{\text {sig}}}$ is the transmission power matrix. The received signal vector for receiver $R^{n+1}_{j}$ can be expressed as:

$$\begin{array}{*{20}l}{} \mathbf{Y}^{n+1}_{j} &= \sum_{i=1}^{K_{n}}\mathbf{H}^{n}_{ij}\mathbf{t}^{n}_{i} + \mathbf{z}^{n}_{n} \end{array} $$

(1a)

$$\begin{array}{*{20}l} &=\sum_{i=1}^{K_{n}}\mathbf{H}^{n}_{ij}\mathbf{Q}^{n}\mathbf{V}^{n}_{ij}\mathbf{P}^{n}_{ij}\mathbf{x}^{n}_{ij} + \sum_{i=1}^{K_{n}}\sum_{m=1,m \neq n}^{K_{n+1}}\\ &\quad\mathbf{H}^{n}_{ij}\mathbf{Q}^{n}\mathbf{V}^{n}_{im}\mathbf{P}^{n}_{im}\mathbf{x}^{n}_{im} + \mathbf{z}^{n}_{j}. \end{array} $$

(1b)

The vector $\mathbf {z}_{j}^{n}$ is a unit variance, zero mean, complex additive white Gaussian noise.

The first part of Eq. (1b) represents the desired signals from all K _n relays (transmitters) in the nth layer. The second part represents the interference signals which are supposed to be sent to the relays (receivers) other than $R^{n+1}_{j}$ in the (n+1)th layer. For each receiver $R^{n+1}_{j}$, there are K _n desired signals and K _n (K _n+1−1) interference signals.

In this subsection, we try to find how to select transmit beamforming matrix $\tilde {\mathbf {V}}^{n}_{ij}$ for the fixed space confining matrix Q ⁿ.

As shown in Fig. 3, the signals received by each receiver can be classified into two categories: desired signals and interference signals. The first part contains all the desired signals and the second part contains K _n interference blocks, where each block includes all the interference signals coming from the same transmitter. For receiver $R^{n+1}_{j}$, all the interference signals should be aligned into a subspace distinct from the desired signals, such that the interference signals can be completely canceled. There are two steps to achieve this goal.

Fig. 3

Interference alignment on the K _n ×K _n+1 MIMO X channel. Illustration of the interference alignment scheme on the K _n ×K _n+1 MIMO X channel

Step I: For each receiver, we first align the interference signals coming from the same transmitter into a T _int -dimensional subspace¹. In other words, according to Fig. 3, we need to align the interference signals in the same block first.

Step II: By step I, we have K _n T _int-dimensional interference subspaces for each receiver, since there are K _n blocks in a row. In this step, we let these K _n subspaces to lie in the same T _int-dimensional subspace. In other words, according to Fig. 3, we align the aligned blocks in a row into the same subspace. In this way, all the interference signals are aligned into the same T _int-dimensional subspace.

We know that $\mathbf {H}^{n}_{ij}\mathbf {Q}^{n}\mathbf {V}^{n}_{im}$ represents the subspace occupied by the signal transmitted from the transmitter ${R^{n}_{i}}$ and desired by receiver $R^{n+1}_{m}$ but received by receiver $R^{n+1}_{j}$. To realize step I, for each receiver $R^{n+1}_{j}$, we define a series of new matrices $\mathbf {W}^{n}_{ij} \in \mathbb {c}^{T \times T_{\text {int}}}$ for all i∈{1,2,…,K _n }, and align all the interference signals coming from the transmitter ${R^{n}_{i}}$ (i.e. H _ij Q ⁿ V _im , ∀m∈{1,…,K _n+1}, m≠j) into the subspace $\mathbf {H}^{n}_{ij}\mathbf {Q}^{n}\mathbf {W}^{n}_{ij}$. The matrix $\mathbf {W}^{n}_{ij}$ is named as interference confining matrix. The selection of matrices $\mathbf {V}^{n}_{im}$ should therefore satisfy the following conditions:

$$ \begin{aligned} \text{span}(\mathbf{H}^{n}_{ij}\mathbf{Q}^{n}\mathbf{V}^{n}_{im}) \subset \text{span}(\mathbf{H}^{n}_{ij}\mathbf{Q}^{n}\mathbf{W}^{n}_{ij}), \forall m\\ \in \{1,2,\ldots,K_{n+1}\},~m\neq j. \end{aligned} $$

(2)

These conditions restrict the subspace occupied by interference signals from the same transmitter ${R^{n}_{i}}$ for receiver $R^{n+1}_{j}$.

After step I, for each receiver, we have aligned all the interference signals from the same transmitter into a T _int-dimensional subspace. Since there are K _n transmitters on a K _n ×K _n+1 MIMO X channel, all the interference signals for each receiver $R^{n+1}_{j}$ have been aligned into K _n T _int-dimensional subspaces, which are $\mathbf {H}^{n}_{1j}\mathbf {Q}^{n}\mathbf {W}^{n}_{1j}$, $\mathbf {H}^{n}_{2j}\mathbf {Q}^{n}\mathbf {W}^{n}_{2j}$, …, $\mathbf {H}^{n}_{K_{n}j}\mathbf {Q}^{n}\mathbf {W}^{n}_{K_{n}j}$. The second interference alignment step is for each receiver $R^{n+1}_{j}$, we let the subspaces of interference signals from different transmitters lie in the same T _int-dimensional subspace². Hence, $\mathbf {W}^{n}_{ij}$ should satisfy the following equation:

$$\begin{array}{*{20}l} \mathbf{H}^{n}_{1j}\mathbf{Q}^{n}\mathbf{W}^{n}_{1j} &= \mathbf{H}^{n}_{2j}\mathbf{Q}^{n}\mathbf{W}^{n}_{2j} = \ldots = \mathbf{H}^{n}_{K_{n}j}\mathbf{Q}^{n}\mathbf{W}^{n}_{K_{n}j},\\ &\forall j \in \{1,2,\ldots,K_{n+1}\}. \end{array} $$

(3)

From Eq. (2), we know

$$ \text{span}(\mathbf{V}^{n}_{im}) \subset \text{span}(\mathbf{W}^{n}_{ij}), ~\forall m \in \{1,2, \ldots K_{n+1}\}, m \neq j. $$

(4)

Since Eq. (4) should be satisfied for all receivers $R^{n+1}_{j}$, where j∈{1,…,K _n+1}, we can find that:

$$\text{span}(\mathbf{V}^{n}_{im}) \subset \text{span}(\mathbf{W}^{n}_{ij}), ~~\forall j \in \{1,2,\ldots,K_{n+1}\}, j \neq m. $$

This means, for fixed $\mathbf {V}^{n}_{im}$, there should be some matrices $\mathbf {E}^{n}_{imj} \in \mathbb {C}^{T_{int} \times T_{sig}}$ satisfying:

$${} \begin{aligned} &\mathbf{V}^{n}_{im} = \mathbf{W}^{n}_{i1}\mathbf{E}^{n}_{im1} = \ldots = \mathbf{W}^{n}_{i(m-1)}\mathbf{E}^{n}_{im(m-1)} = \mathbf{W}^{n}_{i(m+1)}\\ &\mathbf{E}^{n}_{im(m+1)} = \ldots = \mathbf{W}^{n}_{{iK}_{n+1}}\mathbf{E}^{n}_{{imK}_{n+1}}. \end{aligned} $$

(5)

for all i∈{1,…,K _n }.

We will use a 3×3 MIMO X channel as an example to illustrate how the IA scheme works in Section 3.1.2.

Since our goal is to find the matrix $\mathbf {V}^{n}_{ij}$ with fixed space confining matrix Q ⁿ and generic channel matrices $\mathbf {H}^{n}_{ij}$ by solving Eq. (3) and Eq. (5), we should guarantee that (i) for any $\mathbf {H}^{n}_{ij}$ and Q ⁿ, the set of equations in (3) always have feasible solutions $\mathbf {W}^{n}_{ij}$ and (ii) for any series of interference confining matrices $\mathbf {W}^{n}_{ij}$, the set of equations in (5) always have feasible solutions $\mathbf {E}^{n}_{imj}$.

By solving equations in (3) and (5), we can find that $\mathbf {W}^{n}_{ij}$ is the null space of $\mathbf {A}^{n}_{j(-i)}$ (i.e., $\mathbf {A}^{n}_{j(-i)}\mathbf {W}^{n}_{ij} = 0$), where

$$\begin{array}{@{}rcl@{}} \mathbf{A}^{n}_{j(-i)} = \left[ \begin{array}{c} (\mathbf{I}-(\mathbf{H}^{n}_{1j}\mathbf{Q}^{n})(\mathbf{H}^{n}_{1j}\mathbf{Q}^{n})^{\dagger})\mathbf{H}^{n}_{ij}\mathbf{Q}^{n}\\ \vdots\\ (\mathbf{I}-(\mathbf{H}^{n}_{(i-1)j}\mathbf{Q}^{n})(\mathbf{H}^{n}_{(i-1)j}\mathbf{Q}^{n})^{\dagger})\mathbf{H}^{n}_{ij}\mathbf{Q}^{n}\\ (\mathbf{I}-(\mathbf{H}^{n}_{(i+1)j}\mathbf{Q}^{n})(\mathbf{H}^{n}_{(i+1)j}\mathbf{Q}^{n})^{\dagger})\mathbf{H}^{n}_{ij}\mathbf{Q}^{n}\\ \vdots\\ (\mathbf{I}-(\mathbf{H}^{n}_{K_{n}j}\mathbf{Q}^{n})(\mathbf{H}^{n}_{K_{n}j}\mathbf{Q}^{n})^{\dagger})\mathbf{H}^{n}_{ij}\mathbf{Q}^{n} \end{array} \right], \end{array} $$

and $\mathbf {E}^{n}_{imj}$ (m≠j) is the null space of $\mathbf {B}^{n}_{i(-m,-j)}$ (i.e., $\mathbf {B}^{n}_{i(-m,-j)}\mathbf {E}^{n}_{imj} = 0$), where $\mathbf {B}^{n}_{i(-m,-j)}$ is resulted by removing the mth and jth sub-matrices (i.e., $(\mathbf {I}-\mathbf {W}^{n}_{im}{\mathbf {W}^{n}_{im}}^{\dagger })\mathbf {W}^{n}_{ij}$ and $(\mathbf {I}-\mathbf {W}^{n}_{ij}{\mathbf {W}^{n}_{ij}}^{\dagger })\mathbf {W}^{n}_{ij}$) from $\mathbf {B}^{n}_{i}$, and

$$\begin{array}{@{}rcl@{}} \mathbf{B}^{n}_{i} = \left[ \begin{array}{c} (\mathbf{I}-\mathbf{W}^{n}_{i1}{\mathbf{W}^{n}_{i1}}^{\dagger})\mathbf{W}^{n}_{ij}\\ (\mathbf{I}-\mathbf{W}^{n}_{i2}{\mathbf{W}^{n}_{i2}}^{\dagger})\mathbf{W}^{n}_{ij}\\ \vdots\\ (\mathbf{I}-\mathbf{W}^{n}_{{iK}_{n+1}}{\mathbf{W}^{n}_{{iK}_{n+1}}}^{\dagger})\mathbf{W}^{n}_{ij} \end{array} \right]. \end{array} $$

Notice that in the previous notation, (·)^† denotes the pseudo-inverse.

For each receiver, there are K _n desired signals and M antennas. The number of dimensions of the whole space is therefore M. We can see the numbers of dimensions occupied by all the desired signals and interference signals are K _n T _sig and T _int, respectively. To perfectly align all the interference signals into a subspace, which is distinct from the subspace occupied by the desired signals, the number of dimensions of the whole space should be no less than the number of dimensions occupied by all the desired signals and interference signals. Thus, we have the following condition:

$$ M \geq T_{\text{int}}+K_{n}T_{\text{sig}}, $$

(6)

The rank of $\mathbf {A}^{n}_{j(-i)} \in \mathbb {C}^{(K_{n}-1)M \times T}$ is less than or equal to (K _n −1)(M−T) and the rank of $\mathbf {B}^{n}_{i(-m,-j)} \in \mathbb {C}^{(K_{n+1}-2)T \times T_{\text {int}}}$ is less than or equal to (K _n+1−2)(T−T _int), since the rank of $(\mathbf {I}-(\mathbf {H}^{n}_{1j}\mathbf {Q}^{n})(\mathbf {H}^{n}_{1j}\mathbf {Q}^{n})^{\dagger })\mathbf {H}^{n}_{ij}\mathbf {Q}^{n}$ and $(\mathbf {I}-\mathbf {W}^{n}_{i1}{\mathbf {W}^{n}_{i1}}^{\dagger })\mathbf {W}^{n}_{ij}$ are less than or equal to M−T and T−T _int respectively³. As mentioned previously, $\mathbf {W}^{n}_{ij} \in \mathbb {c}^{T \times T_{\text {int}}}$ and $\mathbf {E}^{n}_{imj} \in \mathbb {C}^{T_{\text {int}} \times T_{\text {sig}}}$ are in the null spaces of $\mathbf {A}^{n}_{j(-i)}$ and $\mathbf {B}^{n}_{i(-m,-j)}$ correspondingly, the feasibility of equations in (3) and (5) can be always guaranteed if the size of Q ⁿ satisfies Eq. (6) and the following conditions:

$$\begin{array}{*{20}l} &T-(K_{n}-1)(M-T) \geq T_{\text{int}}, \end{array} $$

(7)

$$\begin{array}{*{20}l} &T_{\text{int}}-(K_{n+1}-2)(T-T_{\text{int}}) \geq T_{\text{sig}}. \end{array} $$

(8)

In following analysis and simulation, we choose the equality for Eqs. (6), (7) and (8) to maximize the degree of freedom.

Therefore, we can rewrite the received signal vector of the relay $R^{n+1}_{j}$ in Eq. (1b) as:

$$\begin{array}{*{20}l}{} \mathbf{Y}^{n+1}_{j} &=\sum_{i=1}^{K_{n}}\mathbf{H}^{n}_{ij}\mathbf{Q}^{n}\mathbf{V}^{n}_{ij}\mathbf{P}^{n}_{ij}\mathbf{x}^{n}_{ij} \end{array} $$

(9a)

$$\begin{array}{*{20}l} &+ \mathbf{H}^{n}_{1j}\mathbf{Q}^{n}\mathbf{W}^{n}_{1j}\!\!\!\!\sum_{m=1,m \neq j}^{K_{n+1}}\mathbf{E}^{n}_{1mj}\mathbf{P}^{n}_{1m}\mathbf{x}^{n}_{1m} +\mathbf{H}^{n}_{2j}\mathbf{Q}^{n}\mathbf{W}^{n}_{2j}\!\!\!\!\sum_{m=1,m \neq j}^{K_{n+1}}\\ &\quad \mathbf{E}^{n}_{2mj}\mathbf{P}^{n}_{2m}\mathbf{x}^{n}_{2m} \end{array} $$

(9b)

$$\begin{array}{*{20}l} &+ \ldots +\mathbf{H}^{n}_{K_{n}j}\mathbf{Q}^{n}\mathbf{W}^{n}_{K_{n}j}\!\!\!\!\sum_{m=1,m \neq j}^{K_{n+1}}\mathbf{E}^{n}_{K_{n}mj}\mathbf{P}^{n}_{K_{n}m}\mathbf{x}^{n}_{K_{n}m}+\mathbf{z}^{n}_{j}, \end{array} $$

(9c)

where the formula (9a) represents the desired signals from all K _n nth layer relays. The formulas in (9b) and (9c) represents the interference signals from each transmitter. From (9b) and (9c), we can find that the interference signals from the same transmitter are aligned into the same T _int-dimensional subspace, e.g., interference signals coming from the ith transmitter ${R^{n}_{i}}$ are aligned into the subspace $\mathbf {H}^{n}_{ij}\mathbf {Q}^{n}\mathbf {W}^{n}_{ij}$.

We then substitute the set of equations in (3) into Eq. (9) and we can rewrite $\mathbf {Y}^{n+1}_{j}$ as:

$$ \begin{aligned} \mathbf{Y}^{n+1}_{j} &=\sum_{i=1}^{K_{n}}\mathbf{H}^{n}_{ij}\mathbf{Q}^{n}\mathbf{V}^{n}_{ij}\mathbf{P}^{n}_{ij}\mathbf{x}^{n}_{ij} + \mathbf{H}^{n}_{1j}\mathbf{Q}^{n}\mathbf{W}^{n}_{1j}\left(\sum_{m=1,m \neq j}^{K_{n}+1}\!\!\mathbf{E}^{n}_{1mj}\mathbf{P}^{n}_{1m}\mathbf{x}^{n}_{1m}\!\!\right.\\ &\left.+\!\!\!\!\sum_{m=1,m \neq j}^{K_{n+1}}\!\!\mathbf{E}^{n}_{2mj}\mathbf{P}^{n}_{2m}\mathbf{x}^{n}_{2m} + \ldots +\!\!\!\!\sum_{m=1,m \neq j}^{K_{n+1}}\!\!\mathbf{E}^{n}_{K_{n}mj}\mathbf{P}^{n}_{K_{n}m}\mathbf{x}^{n}_{K_{n}m}\right)+\mathbf{z}^{n}_{j}. \end{aligned} $$

(10)

The first part is the desired signals and the second part represents the interference signals. By now, for each receiver $R^{n+1}_{j}$, all the interference signals are aligned into the same T _int-dimensional subspace $\mathbf {H}^{n}_{1j}\mathbf {Q}^{n}\mathbf {W}^{n}_{1j}$.

3.1.2 Example of the scheme: K _n =K _n+1=3

In this section, we will use a 3×3 MIMO X channel as an example to illustrate how the IA scheme introduced in Section 3.1.1 works.

In the scheme of IA, we only need to consider the subspaces occupied by the received signals, as shown in Fig. 4. For each receiver, it can receive nine signals, since K _n =K _n+1=3. The formulas in black are the desired signals. The formulas in the same block are the interference signals transmitted from the same transmitter, e.g., for the first receiver $R^{n+1}_{1}$, the formulas in the red block indicate the interference signals transmitted from the first transmitter ${R^{n}_{1}}$.

Fig. 4

Example of IA scheme on the 3×3 MIMO X channel. Illustration of the two steps of IA scheme on the example of 3×3 MIMO X channel

Recall the two steps of IA scheme introduced in Section 3.1.1. Step I: for each receiver, we need to align the interference signals in the same blocks into the subspaces pointed by corresponding arrows in Fig. 4. For example, for the first receiver $R^{n+1}_{1}$, we need to align the signals in the red block into the subspace $\mathbf {H}^{n}_{11}\mathbf {Q}^{n}\mathbf {W}^{n}_{11}$, align the signals in the blue block into the subspace $\mathbf {H}^{n}_{21}\mathbf {Q}^{n}\mathbf {W}^{n}_{21}$ and align the signals in the yellow block into the subspace $\mathbf {H}^{n}_{31}\mathbf {Q}^{n}\mathbf {W}^{n}_{31}$. This step is denoted as “(i)” in Fig. 4. Step II: for each receiver, we let the interference subspaces from the first step to lie in the same subspace, e.g., for the first receiver $R^{n+1}_{1}$, we have $\mathbf {H}^{n}_{11}\mathbf {Q}^{n}\mathbf {W}^{n}_{11} = \mathbf {H}^{n}_{21}\mathbf {Q}^{n}\mathbf {W}^{n}_{21} = \mathbf {H}^{n}_{31}\mathbf {Q}^{n}\mathbf {W}^{n}_{31}$. This step is denoted as “(ii)” in Fig. 4.

To finish step I, from Fig. 4, we have:

$$\begin{array}{*{20}l} \text{span}(\mathbf{V}^{n}_{im}) \subset \text{span}(\mathbf{W}^{n}_{ij}), ~\forall j \neq m,~ j \in \{1,2,3\}, \end{array} $$

which means there exist a series of matrices E _imj such that:

$$\begin{array}{*{20}l} &\mathbf{V}^{n}_{11} = \mathbf{W}^{n}_{12}\mathbf{E}^{n}_{112} = \mathbf{W}^{n}_{13}\mathbf{E}^{n}_{113}, ~~\mathbf{V}^{n}_{21} = \mathbf{W}^{n}_{22}\mathbf{E}^{n}_{212} =\\ &\mathbf{W}^{n}_{23}\mathbf{E}^{n}_{213}, ~~\mathbf{V}^{n}_{31} = \mathbf{W}^{n}_{32}\mathbf{E}^{n}_{312} = \mathbf{W}^{n}_{33}\mathbf{E}^{n}_{313}, \end{array} $$

(11a)

$$\begin{array}{*{20}l} &\mathbf{V}^{n}_{12} = \mathbf{W}^{n}_{11}\mathbf{E}^{n}_{121} = \mathbf{W}^{n}_{13}\mathbf{E}^{n}_{123}, ~~\mathbf{V}^{n}_{22} = \mathbf{W}^{n}_{21}\mathbf{E}^{n}_{221} =\\ &\mathbf{W}^{n}_{23}\mathbf{E}^{n}_{223}, ~~\mathbf{V}^{n}_{32} = \mathbf{W}^{n}_{31}\mathbf{E}^{n}_{321} = \mathbf{W}^{n}_{33}\mathbf{E}^{n}_{323}, \end{array} $$

(11b)

$$\begin{array}{*{20}l} &\mathbf{V}^{n}_{13} = \mathbf{W}^{n}_{11}\mathbf{E}^{n}_{131} = \mathbf{W}^{n}_{12}\mathbf{E}^{n}_{132}, ~~\mathbf{V}^{n}_{23} = \mathbf{W}^{n}_{21}\mathbf{E}^{n}_{231} = \\ &\mathbf{W}^{n}_{22}\mathbf{E}^{n}_{232}, ~~\mathbf{V}^{n}_{33} = \mathbf{W}^{n}_{31}\mathbf{E}^{n}_{331} = \mathbf{W}^{n}_{32}\mathbf{E}^{n}_{332}. \end{array} $$

(11c)

For, we need to align the interference signals from different transmitters into the same subspace. Therefore, the selection of $\mathbf {W}^{n}_{ij}$ should satisfy the following conditions:

$$\begin{array}{*{20}l} &\mathbf{H}^{n}_{11}\mathbf{Q}^{n}\mathbf{W}^{n}_{11} = \mathbf{H}^{n}_{21}\mathbf{Q}^{n}\mathbf{W}^{n}_{21}=\mathbf{H}^{n}_{31}\mathbf{Q}^{n}\mathbf{W}^{n}_{31}, \end{array} $$

(12a)

$$\begin{array}{*{20}l} &\mathbf{H}^{n}_{12}\mathbf{Q}^{n}\mathbf{W}^{n}_{12} = \mathbf{H}^{n}_{22}\mathbf{Q}^{n}\mathbf{W}^{n}_{22}=\mathbf{H}^{n}_{32}\mathbf{Q}^{n}\mathbf{W}^{n}_{32}, \end{array} $$

(12b)

$$\begin{array}{*{20}l} &\mathbf{H}^{n}_{13}\mathbf{Q}^{n}\mathbf{W}^{n}_{13} = \mathbf{H}^{n}_{23}\mathbf{Q}^{n}\mathbf{W}^{n}_{23}=\mathbf{H}^{n}_{33}\mathbf{Q}^{n}\mathbf{W}^{n}_{33}. \end{array} $$

(12c)

We can finally find $\mathbf {W}^{n}_{ij}$ by solving equations in (12). Then by solving equations in (11), we can get $\mathbf {V}^{n}_{ij}$ for all i and j.

3.1.3 The selection of space confining matrix: Q ⁿ

Different Q ⁿ in the previous subsection may lead to different results in terms of the transmission rate under a certain power constraint. Here, we propose a method to design Q ⁿ in this section.

The decoding matrix used to decode the signal from the nth layer relay ${R^{n}_{i}}$ to the (n+1)th layer relay $R^{n+1}_{j}$ is denoted by $\mathbf {U}^{n}_{ij} \in \mathbb {C}^{M \times T_{\text {sig}}}$, which is a full rank matrix. Since $\mathbf {U}^{n}_{ij}$ is the matrix used to decode signal transmitted from transmitter ${R^{n}_{i}}$ to receiver $R^{n+1}_{j}$, from Eq. (10) we know that it should satisfy the conditions:

$$\begin{array}{*{20}l} &{\mathbf{U}^{n}_{ij}}^{H}\mathbf{H}^{n}_{mj}\mathbf{Q}^{n}\mathbf{V}^{n}_{mj} = 0, ~~\forall m \neq i, \end{array} $$

(13)

$$\begin{array}{*{20}l} &{\mathbf{U}^{n}_{ij}}^{H}\mathbf{H}^{n}_{1j}\mathbf{Q}^{n}\mathbf{W}^{n}_{1j} = 0. \end{array} $$

(14)

With perfect IA, the transmission rate of the K _n ×K _n+1 MIMO X channel is represented as follows:

$${} \sum_{i=1}^{K_{n}}\!\sum_{j=1}^{K_{n+1}}\!\text{log}\big(\text{det}(\mathbf{I}+{\mathbf{U}^{n}_{ij}}^{H}\mathbf{H}^{n}_{ij} \mathbf{Q}^{n}\mathbf{V}^{n}_{ij}\mathbf{P}^{n}_{ij}{\mathbf{P}^{n}_{ij}}^{H}{\mathbf{V}^{n}_{ij}}^{H}{\mathbf{Q}^{n}}^{H} {\mathbf{H}^{n}_{ij}}^{H}\mathbf{U}^{n}_{ij})\big). $$

(15)

We assume that the power is equally allocated in each layer. Then, the optimization problem corresponding to the space confining matrix Q ⁿ can be formulated as follows:

$$ \begin{aligned} &\max_{\{\mathbf{Q}^{n}\}}\ \sum_{i=1}^{K_{n}} \sum_{j=1}^{K_{n+1}} \text{log}\big(\text{det}(\mathbf{I}+{\mathbf{U}^{n}_{ij}}^{H}\mathbf{H}^{n}_{ij}\mathbf{Q}^{n}\mathbf{V}^{n}_{ij}{\mathbf{V}^{n}_{ij}}^{H}{\mathbf{Q}^{n}}^{H}{\mathbf{H}^{n}_{ij}}^{H}\mathbf{U}^{n}_{ij})\big),\\ &~\text{s.t.} \left\{ \begin{aligned} &\mathbf{H}^{n}_{ij}\mathbf{Q}^{n}\mathbf{W}^{n}_{ij} = \mathbf{H}^{n}_{1j}\mathbf{Q}^{n}\mathbf{W}^{n}_{1j},~\forall i \in \{2,\ldots,K_{n}\};\\ &j\in \{1,\ldots,K_{n+1}\}\\ &\text{span}(\mathbf{V}^{n}_{ij}) \!\subset \text{\!span}(\mathbf{W}^{n}_{im}), ~\!\forall m \neq j; ~m,j \in \{\!1,\ldots,K_{n+1}\};\\ &i \in \!\{\!1,\ldots,K_{n}\}\\ &{\mathbf{U}^{n}_{ij}}^{H}\mathbf{H}^{n}_{mj}\mathbf{Q}^{n}\mathbf{V}^{n}_{mj} = 0, ~\forall m \neq i;~m,i \in \{1,\ldots,K_{n}\};\\ &j \in \{1,\dots,K_{n+1}\}\\ &{\mathbf{U}^{n}_{ij}}^{H}\mathbf{H}^{n}_{1j}\mathbf{Q}^{n}\mathbf{W}^{n}_{1j} = 0,~\forall i \in \{1,\ldots,K_{n}\};~j \in \{1,\ldots,K_{n+1}\}.\\ \end{aligned} \right. \end{aligned} $$

(16)

Due to the non-convexity of the problem (16), we propose sub-optimal solutions. We apply genetic algorithms [24] to find a sub-optimal Q ⁿ, which provides good performance as shown in our simulation results.

We designed Algorithm 1 (on page 17) to solve our problem. In this algorithm, there are five steps: Initialization, Selection, Crossover, Mutation, and Termination.

• Initialization: we randomly generate a group of (M−T)×M matrices as samples, the size of the group is G (G is an odd number). We denote each sample as $\mathbf {F}^{n}_{g}$, where g∈{1,2,…,G}. For each g∈{1,2,…,G}, we get $\mathbf {Q}_{g}^{n}$ as the orthonormal basis for the null space of $\mathbf {F}^{n}_{g}$. Therefore, the size of $\mathbf {Q}^{n}_{g}$ is M×T.

• Selection: we use this group of $\mathbf {Q}_{g}^{n}$ to get the fitness value, which is the objective function in optimization problem (16):

  $$ \text{FIT}(g) = \sum_{i=1}^{K_{n}}\sum_{j=1}^{K_{n+1}} \text{log}\left(\text{det}\left(\mathbf{I}+{\mathbf{U}^{n}_{ij}}^{H}\mathbf{H}^{n}_{ij}\mathbf{Q}_{g}^{n}\mathbf{V}^{n}_{ij}{\mathbf{V}^{n}_{ij}}^{H}{\mathbf{Q}^{n}_{g}}^{H}{\mathbf{H}^{n}_{ij}}^{H}\mathbf{U}^{n}_{ij}\right)\right), $$

  (17)

  and select (G−1) samples by applying Roulette Wheel Selection and denote them as $\mathbf {S}^{n}_{g}, g \in \{1,2,\ldots,G-1\}$. In this way, the one that leads to a larger fitness value is more likely to be selected. In the last step of this part, we select the $\mathbf {F}^{n}_{g}$ that leads to the largest fitness value and denote it as $\mathbf {S}^{n}_{G}$.

• Crossover: since G is an odd number, for g∈{1,2,…,G−1}, we divide $\mathbf {S}^{n}_{G}$ into $\frac {G-1}{2}$ pairs. In each pair, we randomly swap some parts of them. For $\mathbf {S}^{n}_{G}$, we keep it unchanged.

• Mutation: for each g∈{1,2,…,G−1}, we randomly change part of $\mathbf {S}^{n}_{G}$. For $\mathbf {S}^{n}_{G}$, we keep it unchanged. Let $\mathbf {F}^{n}_{g} = \mathbf {S}^{n}_{g}$, for all g∈{1,2,…,G}.

• Termination: we repeatedly run the algorithm until a fixed number of generations σ is reached.

In Selection, we generate $\mathbf {S}^{n}_{G}$ with the largest fitness value and keep it unchanged in Crossover and Mutation. We do this in order to guarantee that in each generation, the largest fitness value will not decrease.

Finally, we get the space confining matrix Q ⁿ as the orthonormal basis for the null space of the $\mathbf {F}^{n}_{g}$ and hence can get $\mathbf {V}^{n}_{ij}$ by solving Eqs. (3) and (5) and $\mathbf {U}^{n}_{ij}$ by solving Eqs. (13) and (14).

Recall that our goal is to maximize the data rate of the K _n ×K _n+1 MIMO X channel, which is written as Eq. (15). To maximize it, $\mathbf {U}^{n}_{ij}$ and $\mathbf {V}^{n}_{ij}$ should be multiplied by the matrix whose columns are left singular vectors and the matrix whose columns are right singular vectors of ${\mathbf {U}^{n}_{ij}}^{H}\mathbf {H}^{n}_{ij}\mathbf {Q}^{n}\mathbf {V}^{n}_{ij}$, respectively.

3.2 Broadcast channel

For the $1 \times \bar {K}$ broadcast channel, the channel gain from the transmitter to the jth receiver is denoted as H _bj . H _bj is an M×M _t matrix, where M and M _t are the numbers of antennas at each receiver and the transmitter, respectively. U _bj is the decoding matrix at receiver j with rank T _sig, and V _bj is the encoding matrix used to encode the signal x _bj from the transmitter to receiver j. The received signal of receiver j is:

$$ \mathbf{Y}_{j}=\mathbf{H}_{bj}\sum_{m = 1}^{\bar{K}}\mathbf{V}_{bm}\mathbf{P}_{bm}\mathbf{x}_{bm}+ \mathbf{z}_{bj}, $$

where P _bm is the power allocation matrix.

Since there are M antennas at each receiver, z _bj is an M×1 zero mean, unit variance, complex additive white Gaussian noise vector. We use V _bj and U _bj to denote the precoder and decoder. To perfectly align the interference, the following condition of precoder should be satisfied:

$$\text{span}(\mathbf{V}_{bj}) = \text{null}(\mathbf{S}_{bj}), $$

where S _bj is given by

$${} \mathbf{S}_{bj} \triangleq \left[\!\mathbf{U}^{H}_{b1}\mathbf{H}_{b1};\ldots;\mathbf{U}^{H}_{b(j-1)}\mathbf{H}_{b(j-1)};\mathbf{U}^{H}_{b(j+1)}\mathbf{H}_{b(j+1)};\ldots;\mathbf{U}^{H}_{b\bar{K}}\mathbf{H}_{b\bar{K}}\!\right]. $$

Let $\tilde {\mathbf {V}}_{bj}$ be the orthonormal basis for the null space of S _bj . Then we have: $\mathbf {V}_{bj} = \tilde {\mathbf {V}}_{bj}\mathbf {T}_{bj}$, where T _bj denotes the coordinate transformation under the basis $\tilde {\mathbf {V}}_{bj}$. For each receiver, to maximize the received signal $\mathbf {U}^{H}_{bj}\mathbf {H}_{bj}\mathbf {V}_{bj}\mathbf {P}_{bj}$, which is equivalent to $\mathbf {U}^{H}_{bj}\mathbf {H}_{bj}\tilde {\mathbf {V}}_{bj}\mathbf {T}_{bj}\mathbf {P}_{bj}$, the decoder U _bj and the coordinate transformation matrix T _bj should be the left and right singular matrix of $\mathbf {H}_{bj}\tilde {\mathbf {V}}_{bj}$.

Then we can achieve the optimal solution by rewriting the iterative algorithm given in [25] as shown in Algorithm 2.

The convergence of the algorithm has been proved in [25].

The data rate received by the jth receiver of the broadcasting part is

$$ \begin{aligned} &\text{log}(\text{det}(\mathbf{I} + \mathbf{U}^{H}_{bj}\mathbf{H}_{bj}\mathbf{V}_{bj}\mathbf{P}_{bj}\mathbf{P}_{bj}^{H}\mathbf{V}_{bj}^{H}\mathbf{H}_{bj}^{H}\mathbf{U}_{bj})) = \\ &\text{log}(\text{det} (\mathbf{I} + \mathbf{\Lambda}_{bj}\mathbf{P}_{bj}\mathbf{P}_{bj}^{H}\mathbf{\Lambda}_{bj}^{H})), \end{aligned} $$

(18)

where Λ _bj is a diagonal matrix:

$$\mathbf{\Lambda}_{bj} \triangleq \mathbf{U}^{H}_{bj}\mathbf{H}_{bj}\tilde{\mathbf{V}}_{bj}\mathbf{T}_{bj}. $$

3.3 Multiple-access channel

Similar to the broadcast channel, for the $\bar {K} \times 1$ multiple-access channel, the channel gain from the ith transmitter to the receiver is denoted as H _im . H _im is an M _r ×M matrix, where M _r and M are the numbers of antennas at the receiver and each transmitter respectively. The ith transmitter transmits x _im with the precoding matrix V _im and the power allocation matrix P _im to the receiver. Then, the destination decodes the received signal Y _m by using the decoding matrix U _im , where

$$\mathbf{Y}_{m} =\sum_{i = 1}^{\bar{K}}\mathbf{H}_{im}\mathbf{V}_{im}\mathbf{P}_{im}\mathbf{x}_{im}. $$

Same as the broadcast channel, the following condition should be satisfied to perfectly align the interference:

$$\text{span}(\mathbf{U}_{im}) = \text{null}\left(\mathbf{S}^{H}_{im}\right), $$

where S _im is defined as follows:

$$\begin{aligned} \mathbf{S}_{im} \triangleq &\left[\mathbf{H}_{1m}\mathbf{V}_{1m},\ldots,\mathbf{H}_{(i-1)m}\mathbf{V}_{(i-1)m},\mathbf{H}_{(i+1)m}\mathbf{V}_{(i+1)m},\right.\\ &\left.\ldots,\mathbf{H}_{\bar{K}m}\mathbf{V}_{\bar{K}m}\right]. \end{aligned} $$

Let $\tilde {\mathbf {U}}_{im}$ be the orthonormal basis for the null space of $\mathbf {S}^{H}_{im}$. We have an equation as follows:

$$\mathbf{U}_{im} = \tilde{\mathbf{U}}_{im}\mathbf{R}_{im}, $$

where R _im denotes the span matrix of $\tilde {\mathbf {U}}_{im}$. We can find the optimal solution of U _im and V _im by using the previous algorithm. The transmission rate of the ith transmitter of this channel is

$$ \begin{aligned} &\text{log}\left(\text{det}\left(\mathbf{I}+\mathbf{U}^{H}_{im}\mathbf{H}_{im}\mathbf{V}_{im}\mathbf{P}_{im}\mathbf{P}_{im}^{H}\mathbf{V}_{im}^{H}\mathbf{H}_{im}^{H}\mathbf{U}_{im}\right)\right) =\\ &\text{log}(\text{det}(\mathbf{I}+\mathbf{\Lambda}_{im}\mathbf{P}_{im}\mathbf{P}_{im}^{H}\mathbf{\Lambda}_{im}^{H})), \end{aligned} $$

(19)

where the diagonal matrix Λ _im is

$$\mathbf{\Lambda}_{im} = \mathbf{R}^{H}_{im}\tilde{\mathbf{U}}^{H}_{im}\mathbf{H}_{im}\mathbf{V}_{im}. $$

4 Power allocation

In this section, we first find space confining matrix Q ⁿ by using Algorithm 1 in Section 3.1.3 under the condition of equal power allocation and then try to optimize the power allocation without changing Q ⁿ.

We formulate the problem of power allocation for each part of the system. As has been assumed before, the total power constraint of each layer is P. We use $J^{n}_{ij}$ to denote the data rate from the nth layer relay ${R^{n}_{i}}$ to the (n+1)th layer relay $R^{n+1}_{j}$. If we assume the transmission rate of transmitters in layer n or the receiving rate of receivers in layer (n+1) are either unequal, the optimization will be extremely difficult. In our system, we equalize transmission rate of transmitters in layer n and equalize receiving rate of receivers in layer (n+1) for all n∈{1,2,…,N}. If there are K _n relays in the nth layer and K _n+1 relays in the (n+1)th layer, where K _n ≥1 and K _n+1≥1, then we have

$$\begin{array}{*{20}l} &\sum^{K_{n+1}}_{j=1}J^{n}_{1j} = \sum^{K_{n+1}}_{j=1}J^{n}_{2j} =\ldots= \sum^{K_{n+1}}_{j=1}J^{n}_{K_{n}j}, \end{array} $$

(20)

$$\begin{array}{*{20}l} &\sum^{K_{n}}_{i=1}J^{n}_{i1} = \sum^{K_{n}}_{i=1}J^{n}_{i2} =\ldots= \sum^{K_{n}}_{i=1}J^{n}_{{iK}_{n+1}}. \end{array} $$

(21)

According to the number of relays in each layer, there are four kinds of channels in the whole system: point-to-point MIMO channel, broadcast MIMO channel, multiple-access MIMO channel, and K _n ×K _n+1 MIMO X channel. In the following, we discuss them one by one.

4.1 Point-to-point MIMO channel

In this subsection, we consider a point-to-point MIMO channel with channel matrix H. The optimal decoding matrix U and encoding matrix V should be the singular value decomposition (SVD) of H. Then, the transmission rate of this channel is

$$\text{log}(\text{det}(\mathbf{I}+\mathbf{U}^{H}\mathbf{H}\mathbf{V}\mathbf{P}\mathbf{P}^{H}\mathbf{V}^{H}\mathbf{H}^{H}\mathbf{U})), $$

where

$$\text{tr}(\mathbf{P}\mathbf{P}^{H}) \leq P. $$

Water-filling algorithm can be used to achieve the optimal power allocation.

4.2 Broadcast and multiple-access MIMO channel

Since the optimization of power allocation in multiple-access channel is similar to that in the broadcast channel, in this section, we only talk about the optimization of the $1 \times \bar {K}$ broadcast MIMO channel.

For broadcast MIMO channel, we use the method shown in Section 3.2 to select precoding matrices and decoding matrices. The data rate received by the jth receiver is

$$ \text{log}(\text{det}(\mathbf{I}+\mathbf{U}^{H}_{bj}\mathbf{H}_{bj}\mathbf{V}_{bj}\mathbf{P}_{bj}\mathbf{P}^{H}_{bj}\mathbf{V}^{H}_{bj}\mathbf{H}^{H}_{bj}\mathbf{U}_{bj})), $$

(22)

where P _bj is the power allocation matrix for signals desired by receiver j.

With the assumption of equal receiving rate of each receiver in the same layer and the objective to maximize the transmission rate of this broadcast channel, the optimization problem can be formulated as follows

$$ \begin{aligned} &\text{maximize}\ \sum_{j=1}^{\bar{K}} \text{log}(\text{det}(\mathbf{I}+\mathbf{U}^{H}_{bj}\mathbf{H}_{bj}\mathbf{V}_{bj}\mathbf{P}_{bj}\mathbf{P}^{H}_{bj}\mathbf{V}^{H}_{bj}\mathbf{H}^{H}_{bj}\mathbf{U}_{bj})),\\ &\text{subject to}\ \left\{ \begin{aligned} & \text{log}(\text{det}(\mathbf{I}+\mathbf{U}^{H}_{b1}\mathbf{H}_{b1}\mathbf{V}_{b1}\mathbf{P}_{b1}\mathbf{P}^{H}_{b1}\mathbf{V}^{H}_{b1}\mathbf{H}^{H}_{b1}\mathbf{U}_{b1})) \\ &= \text{log}(\text{det}(\mathbf{I}+\mathbf{U}^{H}_{b2}\mathbf{H}_{b2}\mathbf{V}_{b2}\mathbf{P}_{b2}\mathbf{P}^{H}_{b2}\mathbf{V}^{H}_{b2}\mathbf{H}^{H}_{b2}\mathbf{U}_{b2}))\\ & = \cdots \\ &= \text{log}(\text{det}(\mathbf{I}+\mathbf{U}^{H}_{b\bar{K}}\mathbf{H}_{b\bar{K}}\mathbf{V}_{b\bar{K}}\mathbf{P}_{b\bar{K}}\mathbf{P}^{H}_{b\bar{K}}\mathbf{V}^{H}_{b\bar{K}}\mathbf{H}^{H}_{b\bar{K}}\mathbf{U}_{b\bar{K}})),\\ & \sum_{j=1}^{\bar{K}}\text{tr}(\mathbf{P}_{bj}\mathbf{P}^{H}_{bj}) \leq P.\\ \end{aligned} \right. \end{aligned} $$

(23)

Since both $\mathbf {U}^{H}_{bj}\mathbf {H}_{bj}\!\mathbf {V}_{bj}$ and P _bj are diagonal matrices, we denote the (i,i)th entity of $\mathbf {U}^{H}_{bj}\mathbf {H}_{bj}\!\mathbf {V}_{bj}$ as α _bji and the (i,i)^th entity of P _bj as p _bji . The objective function of Eq. (23) can be rewritten as

$$\sum_{j=1}^{\bar{K}}\text{log}\left(\prod_{i=1}^{T_{\text{sig}}}\left(1+\alpha^{2}_{bji}p^{2}_{bji}\right)\right) = \sum_{j=1}^{\bar{K}}\text{log}\left(\prod_{i=1}^{T_{\text{sig}}}G_{bji}\right), $$

by letting:

$$ G_{bji} = 1+\alpha^{2}_{bji}p^{2}_{bji}. $$

(24)

Total power used in this channel is

$$\sum_{j=1}^{\bar{K}}\text{tr}\left(\mathbf{P}_{bj}\mathbf{P}^{H}_{bj}\right)=\sum_{j=1}^{\bar{K}}\sum_{i=1}^{T_{\text{sig}}}p^{2}_{bji} = \sum_{j=1}^{\bar{K}}\sum_{i=1}^{T_{\text{sig}}}\left(\frac{G_{bji}-1}{\alpha^{2}_{bji}}\right). $$

The optimization problem can be rewritten as follows

$$ \begin{aligned} &\text{maximize}\ \prod_{j=1}^{\bar{K}}\prod_{i=1}^{T_{\text{sig}}} G_{bji},\\ &\text{subject to}\ \left\{ \begin{aligned} & \prod_{i = 1}^{T_{\text{sig}}} G_{b1i} = \prod_{i=1}^{T_{\text{sig}}} G_{b2i} = \cdots = \prod_{i=1}^{T_{\text{sig}}} G_{b\bar{K}i},\\ & \sum_{j=1}^{\bar{K}}\sum_{i=1}^{T_{\text{sig}}}\left(\frac{G_{bji}-1}{\alpha^{2}_{bji}}\right) \leq P,\\ \end{aligned} \right. \end{aligned} $$

(25)

which is a geometric programming problem. The geometric programming problem can be transferred into a convex problem [26], and the optimal solution can be obtained by numerical method, e.g., the Matlab software CVX [27].

4.3 K _n ×K _n+1 MIMO X channel

In this subsection, we consider K _n ×K _n+1 MIMO X channels, and use the same notations in Section 3.1. Recall Eq. (15), we can find that both ${\mathbf {U}^{n}_{ij}}^{H}\mathbf {H}^{n}_{ij}\mathbf {Q}^{n}\mathbf {V}^{n}_{ij}$ and $\mathbf {P}^{n}_{ij}$ are diagonal matrices. We use $\alpha ^{n}_{ijm}$ to denote the entity at position (m,m) of ${\mathbf {U}^{n}_{ij}}^{H}\mathbf {H}^{n}_{ij}\mathbf {Q}^{n}_{j}\mathbf {V}^{n}_{ij}$ and use $p^{n}_{ijm}$ to denote the entity at position (m,m) of matrix $\mathbf {P}^{n}_{ij}$. Then we have

$$\begin{array}{*{20}l}{} J^{n}_{ij} &=\text{log}(\text{det}(\mathbf{I}+{\mathbf{U}^{n}_{ij}}^{H}\mathbf{H}^{n}_{ij}\mathbf{Q}^{n}\mathbf{V}^{n}_{ij}\mathbf{P}^{n}_{ij}{\mathbf{P}^{n}_{ij}}^{H}{\mathbf{V}^{n}_{ij}}^{H}{\mathbf{Q}^{n}}^{H}{\mathbf{H}^{n}_{ij}}^{H}\mathbf{U}^{n}_{ij}))\notag\\ &=\text{log}\left(\prod_{m = 1}^{T_{\text{sig}}}\left(1+{\alpha^{n}_{ijm}}^{2}{p^{n}_{ijm}}^{2}\right)\right)\notag\\ &= \text{log}\left(\prod_{m=1}^{T_{\text{sig}}}G^{n}_{ijm}\right),\notag \end{array} $$

by letting

$$ G^{n}_{ijm} = 1+{\alpha^{n}_{ijm}}^{2}{p^{n}_{ijm}}^{2}.\notag $$

Total power used in this channel is

$$ \begin{aligned} &\sum_{i=1}^{K_{n}}\sum_{j=1}^{K_{n+1}}\text{tr}\left(\mathbf{P}^{n}_{ij}{\mathbf{P}^{n}_{ij}}^{H}\right) =\sum_{i=1}^{K_{n}}\sum_{j=1}^{K_{n+1}}\sum_{m=1}^{T_{\text{sig}}}{p^{n}_{ijm}}^{2} =\\ &\sum_{i=1}^{K_{n}}\sum_{j=1}^{K_{n+1}}\sum_{m=1}^{T_{\text{sig}}}\left(\frac{G^{n}_{ijm}-1}{{\alpha^{n}_{ijm}}^{2}}\right).\notag \end{aligned} $$

From conditions (20) and (21), we have:

$$\begin{array}{*{20}l} &\prod_{j = 1}^{K_{n+1}}\prod_{m = 1}^{T_{\text{sig}}} G^{n}_{ijm} = \prod_{j = 1}^{K_{n+1}}\prod_{m = 1}^{T_{\text{sig}}} G^{n}_{1jm}, ~~ \forall i \in \{1,2,\ldots,K_{n}\},\notag\\ &\prod_{i = 1}^{K_{n}}\prod_{m = 1}^{T_{\text{sig}}} G^{n}_{ijm} = \prod_{i = 1}^{K_{n}}\prod_{m = 1}^{T_{\text{sig}}} G^{n}_{i1m}, ~~ \forall j \in \{1,2,\ldots,K_{n+1}\}.\notag \end{array} $$

The optimization problem can be formulated as follows

$$ \begin{aligned} &\text{maximize}\ \prod_{i=1}^{K_{n}}\prod_{j=1}^{K_{n+1}}\prod_{m=1}^{T_{\text{sig}}} G^{n}_{ijm},\\ &\text{subject to}\ \left\{ \begin{aligned} &\prod_{j = 1}^{K_{n+1}}\prod_{m = 1}^{T_{\text{sig}}} G^{n}_{ijm} = \prod_{j = 1}^{K_{n+1}}\prod_{m = 1}^{T_{\text{sig}}} G^{n}_{1jm},~~\forall i \in \{1,2,\ldots,K_{n}\},\\ &\prod_{i = 1}^{K_{n}}\prod_{m = 1}^{T_{\text{sig}}} G^{n}_{ijm} = \prod_{i = 1}^{K_{n}}\prod_{m = 1}^{T_{\text{sig}}} G^{n}_{i1m},~~\forall j \in \{1,2,\ldots,K_{n+1}\},\\ & \sum_{i=1}^{K_{n}}\sum_{j=1}^{K_{n+1}}\sum_{m=1}^{T_{\text{sig}}}\left(\frac{G^{n}_{ijm}-1}{{\alpha^{n}_{ijm}}^{2}}\right) \leq P,\\ \end{aligned} \right. \end{aligned} $$

(26)

which is also a geometric programming problem.

5 Performance evaluation

We divide this section into three parts. Firstly, we evaluate the performance of Algorithm 1, which is a genetic algorithm used to find proper space confining matrix Q ⁿ in the IA scheme of MIMO X channel. Secondly, we show the comparison in performance and computational complexity between MIMO X channel and MIMO interference channel in Section 5.1. Thirdly, we evaluate the performance of the whole system in terms of transmission rate and compare our system with two benchmarks: multiple layers of parallel relays system with only one relay in each layer and the system built by replacing the MIMO X channels in our proposed system with MIMO interference channels.

In the simulation, we model the channels as independent Rayleigh fading channels with unit average power gains, i.e., modeling the entries of $\mathbf {H}^{n}_{ij}$ as independent and identical (i.i.d.) complex Gaussian random variables with zero mean.

5.1 Performance of Algorithm 1

For the K _n ×K _n+1 MIMO X channel, we use Algorithm 1 to select the space confining matrix Q ⁿ. We evaluate the performance of Algorithm 1 with difference number of generations σ and different sizes of groups G.

Several MIMO X channels with different channel sizes are studied. With fixed size of group G=11, Fig. 5 shows the transmission rates versus number of generations σ. Figure 6 shows the transmission rates versus group size G with fixed number of generations σ=15. The transmission power P used in both figures are 40 dB. From the figures, we can see that (i) the transmission rate is increasing with both the number of generations and the group size and (ii) when the number of generations reaches 15 or the group size reaches 11, there is no significant change in the transmission rate. Therefore, in Sections 5.1 and 5.3, we set the number of generations σ as 15 and set the group size G as 11.

Fig. 5

Evaluation of Algorithm 1 with different number of generations σ. Results for evaluation of Algorithm 1 with different number of generations σ

Fig. 6

Evaluation of Algorithm 1 with different sizes of groups G. Results for evaluation of Algorithm 1 with different sizes of groups G

5.2 Comparison between MIMO X channel and MIMO interference channel

In this subsection, we will first compare the performance of the IA scheme for the K _n ×K _n+1 MIMO X channel with that of the IA scheme presented in [28] for the K _n ×K _n+1 MIMO interference channels in our corresponding conference paper [20]. Secondly, we analyze the computational complexity of these two IA schemes in the corresponding wireless channels.

5.2.1 Performance of MIMO X channel and MIMO interference channel

The interference channel is a kind of channel with multiple transmitter-receiver pairs, where each transmitter only wants to send message to its corresponding receiver. For the benchmark of the K _n ×K _n+1 MIMO interference channel, we assume each transmitter selects one distinct relay as its receiver. We compare the transmission rate of the K _n ×K _n+1 MIMO interference channel with our K _n ×K _n+1 MIMO X channel. Since each transmitter selects one distinct receiver on the MIMO interference channel, in our simulation, we show two ways for the transmitters to choose receivers: (i) for each transmitter, it randomly select a distinct receiver and (ii) throughout all the possible selection combinations, we select the one that gives the largest transmission rate of the channel. We call the case “(i)” as interference channel without selection and the case “(ii)” as interference channel with selection.

The overall transmission rate of the system is limited by the bottleneck, which is the layer with the minimum transmission rate. The system we investigate is a multiple-layer-multiple-relay system, in reality, it is rarely possible that the numbers of available relays in all layers are equal. Comparing with our MIMO X channel, the shortcoming of the MIMO interference channel is the transmission rate of adjacent layers with different numbers of transmission relays and receiving relays. Since each transmitter should select one distinct receiver for the MIMO interference channel, there will be an under-utilization of relays in the layer with more relays.

In Figs. 7 and 8, two MIMO channels with different sizes, 2×3 and 3×4, are studied. From both the figures, we can see that the transmission rate of the MIMO X channel is around 40 % higher than the interference channel with relay selection and around 55 % higher than the interference channel without relay selection.

Fig. 7

Comparison between 2×3 MIMO X channel and 2×3 interference channel. Results for comparison between 2×3 MIMO X channel and 2×3 interference channel

Fig. 8

Comparison between 3×4 MIMO X channel and 3×4 interference channel. Results for comparison between 3×4 MIMO X channel and 3×4 interference channel

5.2.2 Complexity analysis

On MIMO X channel, the computation of the IA scheme mainly lies in the selection of space confining matrix Q ⁿ. In the part of selection of Q ⁿ, we introduced a genetic algorithm (i.e., Algorithm 1). For each Q ⁿ in Algorithm 1, we need to find its fitness value as defined in Eq. (17). The complexity of finding the fitness value is dominated by finding the matrices $\mathbf {W}^{n}_{ij}$ and matrices $\mathbf {E}^{n}_{im1}$ ⁴, which are the null space of $\mathbf {A}^{n}_{j(-i)}$ and $\mathbf {B}^{n}_{i(-m,-1)}$, respectively.

To find the complexity of getting $\mathbf {A}^{n}_{j(-i)}$, we firstly need to find the complexity of calculating the matrix (I−(H _1j Q ⁿ)(H _1j Q ⁿ)^†)H _ij Q ⁿ, which is a sub-matrix in the matrix $\mathbf {A}^{n}_{j(-i)}$. For an m×n matrix A, an n×p matrix B and an m×m matrix C, the complexity of calculating the multiplication of A and B is $\mathcal {O}(mnp)$ [29] and the complexity of calculating the inverse of matrix C is $\mathcal {O}(m^{3})$ [30]. Thus, the complexity of getting (I−(H _1j Q ⁿ)(H _1j Q ⁿ)^†)H _ij Q ⁿ is $\mathcal {O}(5M^{2}T +M^{3})$. Since T≤M, we can write $\mathcal {O}(5M^{2}T + M^{3})$ as $\mathcal {O}(m^{3})$. The complexity of calculating the matrix $\mathbf {A}^{n}_{j(-i)}$ is therefore $\mathcal {O}(M^{3}K_{n})$. By the same analysis, the complexity of calculating the matrix $\mathbf {B}^{n}_{i(-m,-1)}$ is $\mathcal {O}(M^{3}K_{n+1})$. Hence, the complexity of getting matrices $W^{n}_{ij}$ for all i∈{1,…K _n } and j∈{1,…K _n+1} is $\mathcal {O}(M^{3}{K_{n}}^{2}K_{n+1})$ and the complexity of getting the matrices E _{i m1} for all i∈{1,…K _n } and m∈{1,…K _n+1} is $\mathcal {O}(M^{3}K_{n}{K_{n+1}}^{2})$. The complexity of finding the fitness value is then $\mathcal {O}(\text {max}\{K_{n}, K_{n+1}\}M^{3}K_{n}K_{n+1})$. The number of calculations of the fitness value is determined by the number of generations multiplied by the size of group in Algorithm 1. By setting size of the group to 11, from the simulation results in Figs. 5 and 6, we can find that the number of generations is a relatively small value, i.e., less than 20.

For the K _n ×K _n+1 MIMO interference channel, a distributed algorithm is introduced to realize the interference alignment. However, the complexity of the distributed algorithm is unknown, since as the channel size increases, the rate of convergence is still an open problem [28].

Nevertheless, we can compare the complexity of MIMO X channel and MIMO interference channel through numerical results (i.e., running time). Table 1 demonstrates the comparison of the running time for the two wireless channels.

Table 1 Comparison in running time between the MIMO X channel and interference channel

From the table, we can find that the running time of the MIMO X channel is almost in the same order of magnitude with the running time of the MIMO interference channel without selection. However, the running time of the MIMO interference channel with selection is much larger, since the procedure of relay selection requires the traversal of all the possible combinations to find the one that gives the maximum transmission rate. On a K _n ×K _n+1 MIMO interference channel, assuming K _n ≤K _n+1, there are $C^{K_{n}}_{K_{n+1}}$ (i.e., selections of K _n numbers of a set with K _n+1 numbers where order is regarded) possible choices, which leads to a multiple of $\frac {K_{n}!}{(K_{n}-K_{n+1})!}$ to the running time of the interference channel without selection.

5.3 Evaluation of the whole system

We evaluate the performance of our system in data rate with two benchmark systems: (i) multiple layers of parallel relays system with only one relay in each layer and (ii) the system built by replacing the MIMO X channels in our proposed system with MIMO interference channls and uses the IA algorithm presented in [28] to design transceivers.

In Figs. 9 and 10, we plot the transmission rates versus the transmission power of one layer. There are three layers of relays, and the numbers of relays in the layers are 3−2−3. The comparison between our system and system “(i)”, where only one relay can be used in each layer, is shown in Fig. 9. We can see that the transmission rate of our system, which uses all available relays in each layer, is around 40 % higher than that of system “(i)”. The insight is, by using multiple relays in each layer, we can increase the DoF of the middle part of system, and hence increase the transmission rate of the whole system.

Fig. 9

Comparison between our system and the system selecting one relay in each layer. Results for comparison between our system and the system selecting one relay in each layer

Fig. 10

Comparison between our system and the system with MIMO interference channel. Results for comparison between our system and the system with MIMO interference channel

In Fig. 10, we compare our system with system “(ii),” which is built by replacing the MIMO X channels in our proposed system with MIMO interference channels. We have shown the MIMO interference channel of system “(ii)” in our corresponding conference paper [20], where each first layer relay should select a distinct second layer relay. In that system, there are two cases: interference channel without selection and interference channel with selection.

From Fig. 10, we can see that due to the inequality of the relays in each layer, the transmission rate of our system with MIMO X channels is 52 and 66 % higher than the system with MIMO interference channels with and without selection, respectively.

6 Conclusions

In this paper, we studied the relay network consisting of multiple layers of relays with multiple antennas. We divided the system into three parts, consisting of broadcast channels, multiple-access channels, and MIMO X channels. For the broadcast and multiple-access channels, we optimized the precoder and decoder of each transmitter and receiver under the condition of interference alignment. For the MIMO X channels, we designed transmit and receive beamforming matrices based on IA technique. Furthermore, we generalized the power constraint from node level to layer level in system evaluation and determined the power allocation of all nodes. Finally, we evaluated the performance of our system and two benchmark systems and showed that our system could achieve around 40 % higher transmission rate.

7 Endnotes

¹In the following of this paper, we call it as a T _int-dimensional interference subspace.

² $\mathbf {H}^{n}_{ij}\mathbf {Q}^{n}\mathbf {W}^{n}_{ij}$ is the subspace of interference signals from transmitter ${R^{n}_{i}}$ received by receiver $R^{n+1}_{j}$

³The conditions (K _n −1)(M−T)<T and (K _n+1−2)(T−T _int)<T _int should be satisfied to guarantee the existences of $\mathbf {W}^{n}_{ij}$ and $\mathbf {E}^{n}_{jmn}$, since $\mathbf {W}^{n}_{ij}$ and $\mathbf {E}^{n}_{imj}$ are in the null space of $\mathbf {A}^{n}_{j(-i)}$ and $\mathbf {B}^{n}_{i(-m,-j)}$, respectively.

⁴According to Eq. (5), we can find that $\mathbf {V}^{n}_{im} = \mathbf {W}^{n}_{i1}\mathbf {E}^{n}_{im1}$. In order to get $\mathbf {V}^{n}_{im}$, we only need to calculate $\mathbf {E}^{n}_{im1}$ rather than E _imj for all j∈{1,2,…K _n+1}.