Impact of CCI on performance analysis of downlink satellite-terrestrial systems: outage probability and ergodic capacity perspective

Abstract

The evolution of non-orthogonal multiple access (NOMA) has raised many opportunities for massive connectivity with less latency in signal transmissions at great distances. We aim to integrate NOMA with a satellite communications network to evaluate system performance under the impacts of imperfect channel state information and co-channel interference from nearby systems. In our considered system, two users perform downlink communications under power-domain NOMA. We analyzed the performance of this system with two modes of shadowing effect: heavy shadowing and average shadowing. The detailed performance was analyzed in terms of the outage probability and ergodic capacity of the system. We derive closed-form expressions and performed a numerical analysis. We discover that the performance of two destinations depends on the strength of the transmit power at the satellite. However, floor outage occurs because the system depends on other parameters, such as satellite link modes, noise levels, and the number of interference sources. To verify the authenticity of the derived closed-form expressions, we also perform Monte-Carlo simulations.

Introduction

Satellite communications are especially able to provide radio access in difficult areas [1]. Because satellite networks can provide higher quality of services (QoS) for comparatively less cost, they can also attain significant improvements in the efficiency of fixed and mobile satellite services. In the evolution of fifth generation (5 G) networks, satellite communications have been viewed as a potential addition to many technologies such as the internet of things (IoT), sensor networks and relaying communications [2]. The future of satellite networks is expected to support services of massive connectivity and reduce operational costs. Therefore, they can be deployed through their integration with various geostationary and non-geo-stationary orbital satellites by applying cooperative transmission or cognitive radio networks to increase the spectrum efficiency. To date, most satellite networks have adopted the orthogonal multiple access (OMA) technique for the transmission and reception of data [3].

The major disadvantage of OMA is that it cannot meet the growing requirements of communications networks. Under OMA, efficient spectrum use and limitations on the number of users have become major challenges which diminish system performance. In the present paper, we consider non-orthogonal multiple access (NOMA) to tackle the challenges raised by OMA. Of the two NOMA categories, we applied Power-Domain NOMA since much of the research has proved this system as having promising features. In NOMA, signals are transmitted superimposed in the same resource block by varying the power level of each user according to their channel gain. To identify the required user signal at the receiver, the system applies successive interference cancellation (SIC) and thereby extracts the required signal. As mentioned, NOMA uses the same resource block for multiple users and thus increases the efficiency of spectrum use at a reasonable level of implemented complexity [4, 5]. The NOMA technique has achieved significant attention from researchers around the globe and is a promising technology with advantages which can be exploited in 5 G communications. Numerous studies have been performed to compare the performance of the NOMA system to OMA. The main finding is that NOMA is efficient [6, 7]. With an increase in spectrum efficiency, the benefits of NOMA performance can be more prevalent if NOMA is integrated with other techniques.

Various studies have introduced the NOMA technique in satellite communications [8,9,10,11,12,13,14,15,16]. In [11], the authors studied integration of the NOMA technique with multi-beam satellite networks, while in [12], the authors investigated integration of NOMA with cognitive satellite networks to increase ergodic performance of the system. The performance of NOMA-hybrid satellite relay networks (HSRN) was studied in [13, 14]. NOMA integrated cognitive HSRN has been studied to analyze outage performance [15]. The performance of a similar system studied in [13] was investigated with the effect of hardware impairments [16]. The work in [17,18,19] considered NOMA-based satellite-terrestrial networks to increase the efficiency of the spectrum by beamforming. In [20], a cooperative NOMA-HSRN was considered in which the user with better channel gain acted as a relay to the remaining users in the cluster. In [21], the authors studied the effect of imperfect channel state information (CSI) and channel impairments (CI) in a NOMA-based terrestrial mobile communications network (TMCN) which functioned with multiple relays. In [22], the authors considered NOMA-based integrated terrestrial satellite networks (ISTN) to study the effect of relaying configurations such as Amplify and Forward (AF) and Decode and Forward (DF). The authors in [29] investigated a system similar to the study in [22] and explored the effect of CI under a DF relay configuration.

In the context of NOMA-HSRN, the effect of co-channel interference (CCI) in all system models has rarely been addressed. In practice, NOMA-HSRN might experience a rich CCI situation, which is an important consideration in the deployment of NOMA and HSRN in wireless communications. It can be demonstrated that the aggressive reuse of spectrum resources leads to degraded performance because of the effect of CCI. As a result, it is more than simply an important priority consideration, as the performance of NOMA-HSRN is guaranteed only if CCI is taken into account. To the best of the authors’ knowledge, the performance of NOMA-HSRN under the impact of CCI has not been solved yet. Aiming to overcome the effect of CCI, which is unavoidable in practical scenarios, the authors in [23] reported degraded performance in a single-user hybrid satellite-terrestrial amplify-and-forward relay network (HSTAFRN) with multiple Rayleigh-faded interference sources. The performance metrics of a downlink multi-user HSTAFRN were examined using a fixed-gain relaying protocol under the effect of CCI [24]. The authors in [25] employed dual-hop relay networks which assumed interference-limited relays and noisy destinations. These may arise from cell-edge or frequency-division relaying [26], although the NOMA-HSRN still experiences worse performance under the effect of CCI. In [27], the authors analysis the performance of millimeter-wave in multi-user HSRN system and using the shadowed-Rician for the link from satellite to relay. The authors in [28] deployed a single-antenna satellite for a multi-user HSTAFRN system and evaluated its outage performance under the effect of both CCI and outdated CSI.

Satellite networks have been designed to replace terrestrial communications systems, but challenges still exist in some aspects of signal processing. The referred studies suggest that integrating NOMA with satellite systems will extend the efficiency of communication between users. Although satellite systems have numerous advantages, challenges such as fading effect and interference require solutions. The majority of research has studied the effect of various scenarios, including imperfect SIC, imperfect CSI and channel impairments, but none has mentioned the effect on system performance from CCI. Therefore, in the present paper, we studied the effect of interference by considering shadowing and interference links in dual-user communications occurring under a NOMA-assisted satellite network. We studied the shadowing effect in two modes: heavy shadowing (HS) and average shadowing (AS). We also investigated the effect of interference links on communications channels in HS mode. In contrast to similar studies listed in Table 1 and to highlight the superiority of NOMA-aided satellite systems, performance was analyzed completely in terms of outage probability (OP) and ergodic capacity (EC).

The primary contributions of the paper are manifold:

• In contrast to other studies, our study focus on a complex NOMA-based terrestrial satellite relay network with two users on the ground and a relay which encounters interference from nearby sources. We provide system performance metrics by considering a Shadowed-Rician channel between the satellite and relay, and a Nakagami-m channel between the relay and destination.

• The system encounters HS and AS effects as a result of the shadowing channel links. We analyzed and compared the system’s performance under these two shadowing modes.

• We derived the closed-form expressions for outage probability and ergodic capacity of both users in a dedicated NOMA user group. A performance gap is expected for these users depending on their demands and the portion of power allocated to each user.

• Finally, we performed a numerical analysis and Monte-Carlo simulations for the derived expressions to verify their authenticity and analyze the system’s performance. We illustrate the system’s performance in HS mode by varying the interference links and mean square error of the channels.

The paper is organized as follows. Section 2 explains the system model and types of signal received from the users. Section 3 provides a performance analysis and describes the mathematical expressions for outage probability, ergodic capacity and diversity order of the system. Section 4 provides an analysis and simulations of the expressions obtained in Sect. 3. Section 5 concludes the paper with the attained results.

Table 1 Summary of related works

System model

Table 2 Table of parameters

In this section, we assume a satellite (S), a relay (R) and two users $D_i,i\in (1,2)$ as in Fig. 1. All nodes are equipped with a single antenna, and the relay operates with the DF protocol. The relay node is also affected by N co-channel interference sources $\{I_n\}^N_{n=1}$. The link from S to $D_i$ is not available because of heavy shadowing [30]. $h_{R}$ denotes the channel coefficient from S to R and follow a Shadowed-Rician channel, $h_{i}$ denotes the channel coefficients from R to $D_i$ and follow a Nakagami-m channel, $h_{nR}$ denotes the channel coefficient of the link between the n-th interference source and relay and follows independent and non-identically distributed (i.ni.d.) Nakagami-m random variable (RVs). Under these conditions, the CSI procedure exhibits error. The estimation channel is expressed as [31]

$$\begin{aligned} h_j = {\hat{h}}_j + e_{j}, \end{aligned}$$

(1)

where $j\in \{R,1,2\}$, $e_j$ is the error term for $CN(0,\sigma _{e_j}^2)$ [32].

Fig. 1

System model

The power-domain assisted NOMA signal from the source transmits user signals superimposed in the same resource block by varying the power coefficient of each user according to their channel state information (CSI). At the receiver’s end, perfect successive interference cancellation (SIC) is assumed to extract the desired signal from the superimposed signal. Imperfect CSI should therefore be studied in practical scenario. Satellite-terrestrial networks needs relay to empower signals before forwarding them to mobile users. The main reason of design a relay is to deal with signal transmission at long distances. The satellite needs to allocate suitable power level to expected users. The first user $D_1$ is assumed to be located at far distance and such weak signal needs higher power allocation. Meantime, the near user $D_2$ just acquire lower level of transmit power. In the first phase, S transmits the signal $\sqrt{P_S A_1 } x_1$ + $\sqrt{P_SA_2 } x_2$ to R, where $P_S$ is the transmit power, $A_1$ and $A_2$ are power allocation such that $A_1 + A_2 = 1$ and $A_1>A_2$ assumed under the NOMA scheme. Then, the signal received at R is given as

$$\begin{aligned} \begin{gathered} y_R = \left( {\sqrt{A_1 } x_1 + \sqrt{A_2 } x_2 } \right) \sqrt{P_S } \left( {\hat{h}_j + e_j } \right) + \sum \limits _{n = 1}^N {\sqrt{P_{cn} } h_{nR} x_n } + n_R, \\ \end{gathered} \end{aligned}$$

(2)

where $P_{Cn}$ is the transmit power of the n-th interference source and $n_R$ is the additive white Gaussian noise (AWGN) at R for $CN(0,N_0)$. The signal to interference plus noise ratio (SINR) is then used to decode $x_1$ and given as

$$\begin{aligned} \varGamma _{R \rightarrow x_1 } = \frac{{\rho _S A_1 \left| {\hat{h}_R } \right| ^2 }}{{\rho _S A_2 \left| {\hat{h}_R } \right| ^2 + \gamma _C + \rho _S \sigma ^2_{e_R} + 1}}, \end{aligned}$$

(3)

where $\rho _S = \frac{{P_S }}{{N_0 }}$ is the transmit SNR, $\rho _{Cn} = \frac{{P_{Cn} }}{{N_0 }}$ and $\gamma _C = \sum \limits _{n = 1}^N {\rho _{Cn} \left| {h_{nR} } \right| ^2 }$. Then, the SINR decoded $x_2$ is given as

$$\begin{aligned} \varGamma _{R \rightarrow x_2 } = \frac{{\rho _S A_2 \left| {\hat{h}_R } \right| ^2 }}{{\gamma _C + \rho _S \sigma ^2_{e_R} + 1}}. \end{aligned}$$

(4)

In the second phase, relay R forwards the signals to the ground users. The signal received at $D_i$ is given as

$$\begin{aligned} \begin{gathered} y_{D_i } = \left( {\sqrt{A_1 } x_1 + \sqrt{A_2 } x_2 } \right) \sqrt{P_R } \left( {\hat{h}_i + e_i } \right) + n_{i }, \\ \end{gathered} \end{aligned}$$

(5)

where $P_R$ is the transmit power at R and $n_{D_i}$ AWGN for $CN(0,N_0)$. It is noted that the other main parameters are listed in Table 2. The SINR which decodes $x_1$ at $D_1$ is given as

$$\begin{aligned} \varGamma _{D_1 \rightarrow x_1 } = \frac{{\rho _R A_1 \left| {\hat{h}_1 } \right| ^2 }}{{\rho _R A_2 \left| {\hat{h}_1 } \right| ^2 + \rho _R \sigma ^2_{e_1} + 1}}, \end{aligned}$$

(6)

where $\rho _R = \frac{{P_R }}{{N_0 }}$, the SINR which decodes signal $x_1$ at $D_2$ is given as [32]

$$\begin{aligned} \varGamma _{D_2 \rightarrow x_1 } = \frac{{\rho _{\rm{R}}A_1 \left| {\hat{h}_2 } \right| ^2 }}{{\rho _{\rm{R}} A_2 \left| {\hat{h}_2 } \right| ^2 + \rho _{\rm{R}} \sigma ^2_{e_2} + 1}}, \end{aligned}$$

(7)

Applying SIC, the SINR which decodes its own signal $x_2$ at $D_2$ is computed according to

$$\begin{aligned} \varGamma _{D_2 \rightarrow x_2 } = \frac{{\rho _{\rm{R}} A_2 \left| {\hat{h}_2 } \right| ^2 }}{{ \sigma ^2_{e_2} + 1}}. \end{aligned}$$

(8)

For performance analysis, these SINRs provide important information which allows us to compute probabilities.

Performance analysis

In this section, we analyze the two main system metrics with the assumed channel models below.

Channel model

Following the results in [33], the probability density function (PDF) of $|\hat{h}_{\rm{R}}|^2$ is formulated by

$$\begin{aligned} f_{\left| {\hat{h}_{\rm{R}} } \right| ^2 } \left( x \right) = \alpha _{\rm{R}} e^{ - \beta _{\rm{R}} x} {_1 F_1} \left( {m_{\rm{R}} ,1,\delta _{\rm{R}} x} \right) , \end{aligned}$$

(9)

where $\alpha _{\rm{R}} = \frac{{\left( {\frac{{2b_{\rm{R}} m_{\rm{R}} }}{{2b_{\rm{R}} m_{\rm{R}} + \varOmega _{\rm{R}} }}} \right) ^{m_{\rm{R}} } }}{{2b_{\rm{R}} }}$, $\beta _{\rm{R}} = ({{2b_{\rm{R}} }})^{-1}$, $\delta _{\rm{R}} = \frac{{\varOmega _{\rm{R}} }}{{2b_{\rm{R}} \left( {2b_{\rm{R}} m_{\rm{R}} + \varOmega _{\rm{R}} } \right) }}$, $m_{\rm{R}}$ is the fading severity parameter, $2b_{\rm{R}}$ and $\varOmega _{\rm{R}}$ denote multipath components and the average power of light of sight (LOS), respectively, and ${_1 F_1}\left( {.,.,.} \right)$ is the confluent hypergeometric function of the first kind [46, Eq. 9.210.1]. Using [34], we can rewrite the PDF of $|h_{\rm{R}}|^2$ as

$$\begin{aligned} f_{\left| {\hat{h}_R } \right| ^2 } \left( x \right) = \alpha _R \sum \limits _{k = 0}^{m_R - 1} {\xi \left( k \right) x^k e^{ - \varXi _R x} }, \end{aligned}$$

(10)

where $\xi \left( k \right) = \frac{{\left( { - 1} \right) ^k \left( {1 - m_R } \right) _k \delta _R^k }}{{\left( {k!} \right) ^2 }}$, $\varXi _R = \beta _R - \delta _R$ and $(.)_x$ denotes the Pochhammer symbol [46, p. xliii]. Based on [46, Eq.3.351.2], the the cumulative distribution function (CDF) of $\left| {\hat{h}_R } \right| ^2$ can be obtained as

$$\begin{aligned} F_{\left| {\hat{h}_R } \right| ^2 } \left( x \right) = 1 - \alpha _R \sum \limits _{k = 0}^{m_R - 1} {\xi \left( k \right) } \sum \limits _{l = 0}^k {\frac{{k!x^l e^{ - \varXi _R x} }}{{l!\left( {\varXi _R } \right) ^{k - l + 1} }}}. \end{aligned}$$

(11)

The PDF and CDF of $|h_i|^2$ are then, respectively, given as [35]

$$\begin{aligned} f_{\left| {\hat{h}_i } \right| ^2 } \left( x \right) = \left( {\frac{{m_i }}{{\varOmega _i }}} \right) ^{m_i } \frac{{x^{m_i - 1} e^{ - \left( {\frac{{m_i }}{{\varOmega _i }}} \right) x} }}{{\varGamma \left( {m_i } \right) }}, \end{aligned}$$

(12)

and

$$\begin{aligned} \begin{aligned} F_{{\left| {\hat{h}_{i} } \right| ^{2} }} \left( x \right)&= 1 - \frac{1}{{\varGamma \left( {m_{i} } \right) }}\varGamma \left( {m_{i} ,\frac{{m_{i} x}}{{\varOmega _{i} }}} \right) \\&= 1 - \sum \limits _{{b_{i} = 0}}^{{m_{i} - 1}} {\frac{1}{{b_{i} !}}} \left( {\frac{{m_{i} x}}{{\varOmega _{i} }}} \right) ^{{ + b_{i} }} e^{{ - \left( {\frac{{m_{i} }}{{\varOmega _{i} }}} \right) x}} \\ \end{aligned} \end{aligned}$$

(13)

where $m_i$ and $\varOmega _i$ are the fading severity parameter and the average power, respectively, and $\varGamma (.,.)$ is the upper incomplete gamma function [46].

Moreover, the PDF of $\gamma _C$ is calculated with corresponding severity parameters $\{m_{Cn}\}^N_n$ and average powers $\{\varOmega _{Cn}\}^N_n$. Therefore, we can express the PDF of $\gamma _C$ as [36, 37] and [24]

$$\begin{aligned} f_{\gamma _C } \left( x \right) = \left( {\frac{{m_I }}{{\varOmega _I }}} \right) ^{m_I } \frac{{x^{m_I - 1} }}{{\varGamma \left( {m_I } \right) }}e^{ - \left( {\frac{{m_I }}{{\varOmega _I }}} \right) x}, \end{aligned}$$

(14)

where the parameters $m_I$ and $\varOmega _I$ are obtained from moment based estimators. For this, we define $\varTheta = \sum \nolimits _{n = 1}^I {\left| {h_{nR} } \right| ^2 }$, and without loss of generality, we assume no power control is used, i.e., $P_{Cn} = P_C$ or $\rho _{Cn} = \rho _C$. Then, we have $\varOmega _I = \rho _C \varOmega _C$, where $\varOmega _C = E\left[ \varTheta \right] = \sum \nolimits _{n = 1}^N {\varOmega _{Cn} }$ and $m_I = \frac{{\varOmega _C^2 }}{{E\left[ {\varTheta ^2 } \right] - \varOmega _C^2 }}$. From this, the exact moments of $\varTheta$ can be obtained as

$$\begin{aligned} \begin{aligned} E\left[ {\varTheta ^{n} } \right]&\approx \sum \limits _{{n_{1} = 0}}^{n} {\sum \limits _{{n_{2} = 0}}^{{n_{1} }} \cdots } \sum \limits _{{n_{{N - 1}} = 0}}^{{n_{{N - 2}} }} {\left( \begin{gathered} n \\ n_{1} \\ \end{gathered} \right) \left( \begin{gathered} n_{1} \\ n_{2} \\ \end{gathered} \right) \left( \begin{gathered} n_{{N - 2}} \\ n_{{N - 1}} \\ \end{gathered} \right) } \\&\quad \times E\left[ {\left| {h_{{1R}} } \right| ^{{2\left( {n - n_{1} } \right) }} } \right] E\left[ {\left| {h_{{1R}} } \right| ^{{2\left( {n_{1} - n_{2} } \right) }} } \right] \cdots E\left[ {\left| {h_{{1R}} } \right| ^{{2\left( {n_{{N - 1}} } \right) }} } \right] \\ \end{aligned} \end{aligned}$$

(15)

where $E\left[ {\left| {h_{iR} } \right| ^n } \right] = \frac{{\varGamma \left( {m_{Cn} + \frac{n}{2}} \right) }}{{\varGamma \left( {m_{Cn} } \right) }}\left( {\frac{{m_{Cn} }}{{\varOmega _{Cn} }}} \right) ^{ - \frac{n}{2}}$.

Outage probability of $D_1$

An outage event of $D_1$ is given when R and $D_1$ cannot detect $x_1$ correctly. Then, the outage probability of $D_1$ is given as

$$\begin{aligned} \begin{aligned} P_{D_1 }&= \Pr \left( {\min \left( {\varGamma _{R \rightarrow x_1 } ,\varGamma _{D_1 \rightarrow x_1 } } \right) < \gamma _1 } \right) \\&= 1 - \underbrace{\Pr \left( {\varGamma _{R \rightarrow x_1 }> \gamma _1 } \right) }_{B_1 }\underbrace{\Pr \left( {\varGamma _{D_1 \rightarrow x_1 } > \gamma _1 } \right) }_{B_2 } \end{aligned}, \end{aligned}$$

(16)

where $\gamma _{i } = 2^{2R_i } - 1$, and $R_i$ is the target rate.

Proposition 1

Here, the closed-form of $B_1$ is given as

$$\begin{aligned} \begin{aligned} B_1&= \alpha _R \sum \limits _{k = 0}^{m_R - 1} {\sum \limits _{l = 0}^k {\sum \limits _{p = 0}^l {\left( {\begin{array}{*{20}c} l \\ p \\ \end{array} } \right) } \frac{{\xi \left( k \right) k!\varGamma \left( {m_I + p} \right) }}{{l!\varGamma \left( {m_I } \right) \left( {\varXi _R } \right) ^{k - l + 1} }}} } e^{ - \varXi _R \phi _1 \left( {\rho _S \sigma _{e_R }^2 + 1} \right) } \\&\quad \times \left( {\phi _1 \left( {\rho _S \sigma _{e_R }^2 + 1} \right) } \right) ^l \left( {1 + \frac{{\varOmega _I \varXi _R \phi _1 }}{{m_I }}} \right) ^{ - m_I - p} \left( {\frac{{\varOmega _I }}{{m_I \left( {\rho _S \sigma _{e_R }^2 + 1} \right) }}} \right) ^p \\ \end{aligned} \end{aligned}$$

(17)

Proof

See Appendix A.

Next, using (6), $B_2$ is rewritten as

$$\begin{aligned} \begin{aligned} B_{2}&= \Pr \left( {\frac{{\rho _{R} A_{1} \left| {\hat{h}_{1} } \right| ^{2} }}{{\rho _{R} A_{2} \left| {\hat{h}_{1} } \right| ^{2} + \rho _{R} \sigma _{{e_{1} }}^{2} + 1}}> \gamma _{1} } \right) \\&= \Pr \left( {\left| {\hat{h}_{1} } \right| ^{2} > \phi _{2} } \right) \\&= 1 - F_{{\left| {\hat{h}_{1} } \right| ^{2} }} \left( {\phi _{2} } \right) \\ \end{aligned} \end{aligned}$$

(18)

where $\phi _2 = \frac{{\left( {\rho _R \sigma _{e_1 }^2 + 1} \right) \gamma _1 }}{{\left( {A_1 - A_2 \gamma _1 } \right) \rho _R }}$. Based on the CDF of $\hat{h} _i$ in (13), $B_2$ can be expressed as

$$\begin{aligned} B_2 = \sum \limits _{b_1 = 0}^{m_1 - 1} {\frac{{e^{ - \frac{{m_1 \phi _2 }}{{\varOmega _1 }}} }}{{b_1 !}}} \left( {\frac{{m_1 \phi _2 }}{{\varOmega _1 }}} \right) ^{b_1 }. \end{aligned}$$

(19)

Finally, substituting (17) and (19) into (16), $P_{D_1}$ can be obtained by

$$\begin{aligned} \begin{gathered} P_{D_1 } = 1 - \alpha _R \sum \limits _{k = 0}^{m_R - 1} {\sum \limits _{l = 0}^k {\sum \limits _{p = 0}^l {\sum \limits _{b_1 = 0}^{m_1 - 1} {\left( {\begin{array}{*{20}c} l \\ p \\ \end{array} } \right) \frac{{\xi \left( k \right) k!}{\varGamma \left( {m_I + p} \right) e^{ - \varXi _R \phi _1 \left( {\rho _S \sigma _{e_R }^2 + 1} \right) - \frac{{m_1 \phi _2 }}{{\varOmega _1 }}} }}{{b_1 !l!\varGamma \left( {m_I } \right) }{\left( {\varXi _R } \right) ^{k - l + 1} \left( {\phi _1 \left( {\rho _S \sigma _{e_R }^2 + 1} \right) } \right) ^{ - l} }}} } } } \\ \times \left( {\frac{{m_1 \phi _2 }}{{\varOmega _1 }}} \right) ^{b_1 } \left( {1 + \frac{{\varOmega _I \varXi _R \phi _1 }}{{m_I }}} \right) ^{ - m_I - p} \left( {\frac{{\varOmega _I }}{{m_I \left( {\rho _S \sigma _{e_R }^2 + 1} \right) }}} \right) ^p \\ \end{gathered} \end{aligned}$$

(20)

Outage probability of $D_2$

The outage events of $D_2$ occurs when R and $D_2$ cannot detect $x_2$ correctly. Therefore, the outage probability of $D_2$ is given as

$$\begin{aligned} \begin{aligned} P_{D_2 }&= \Pr \left( {\min \left( {\varGamma _{R \rightarrow x_2 } ,\varGamma _{D_2 \rightarrow x_2 } } \right) < \gamma _{2 } } \right) \\&= 1 - \Pr \left( {\varGamma _{R \rightarrow x_2 }> \gamma _{2 } } \right) \Pr \left( {\varGamma _{D_2 \rightarrow x_2 } > \gamma _{2} } \right) \\ \end{aligned} \end{aligned}$$

(21)

Proposition 2

The closed-form outage probability of $P_{D_2}$ is obtained as

$$\begin{aligned} \begin{gathered} P_{D_2 } = 1 - \alpha _R \sum \limits _{k = 0}^{m_R - 1} {\sum \limits _{l = 0}^k {\sum \limits _{p = 0}^l {\sum \limits _{b_2 = 0}^{m_2 - 1} {\left( {\begin{array}{*{20}c} l \\ p \\ \end{array} } \right) } \frac{{k!\xi \left( k \right) }{\varGamma \left( {m_I + p} \right) e^{ - \varXi _R \psi _1 \left( {\rho _S \sigma _{e_R }^2 + 1} \right) - \frac{{m_2 \psi _2 }}{{\varOmega _2 }}} }}{{b_2 !l!\varGamma \left( {m_I } \right) }{\left( {\varXi _R } \right) ^{k - l + 1} \left( {\psi _1 \left( {\rho _S \sigma _{e_R }^2 + 1} \right) } \right) ^{ - l} }}} } } \\ \times \left( {\frac{{m_2 \psi _2 }}{{\varOmega _2 }}} \right) ^{b_2 } \left( {1 + \frac{{\varOmega _I \varXi _R \psi _1 }}{{m_I }}} \right) ^{ - m_I - p} \left( {\frac{{\varOmega _I }}{{m_I \left( {\rho _S \sigma _{e_R }^2 + 1} \right) }}} \right) ^p \\ \end{gathered} \end{aligned}$$

(22)

Proof

See Appendix B.

Diversity order

To gain some insight, we derive under the asymptotic outage probability of $D_i$ under a high SNR $(\rho = \rho _S=\rho _R \rightarrow \infty )$. The diversity order is defined as [38]

$$\begin{aligned} d = - \mathop {\lim }\limits _{\rho \rightarrow \infty } \frac{{\log \left( {P_{D_i }^\infty } \right) }}{{\log \left( \rho \right) }}, \end{aligned}$$

(23)

where ${P_{D_i }^\infty }$ is the asymptotic outage probability of $D_i$.

Proposition 3

The asymptotic outage probability of $D_1$ is given as

$$\begin{aligned} \begin{aligned} P_{D_1 }^\infty&= 1 - \left( {1 - \frac{1}{{\varGamma \left( {m_1 + 1} \right) }}\left( {\frac{{m_1 \phi _2 }}{{\varOmega _1 }}} \right) ^{m_1 } } \right) \\&\quad \times \left( {1 - \alpha _R \phi _1 \left( {\frac{{\left( {m_I } \right) !}}{{\varGamma \left( {m_I } \right) }}\left( {\frac{{\varOmega _I }}{{m_I }}} \right) + \left( {\rho _S \sigma _{e_R }^2 + 1} \right) } \right) } \right) \\ \end{aligned} \end{aligned}$$

(24)

Proof

See Appendix C.

Similarly, the asymptotic of $D_2$ can be expressed by

$$\begin{aligned} \begin{aligned} P_{{D_{2} }}^{\infty }&= 1 - \left( {1 - \frac{1}{{\varGamma \left( {m_{2} - 1} \right) }}\left( {\frac{{m_{2} \psi _{2} }}{{\varOmega _{2} }}} \right) ^{{m_{2} }} } \right) \left( 1 \right. \\&\quad - \left. {\alpha _{R} \psi _{1} \left( {\frac{{\left( {m_{I} } \right) !}}{{\varGamma \left( {m_{I} } \right) }}\left( {\frac{{\varOmega _{I} }}{{m_{I} }}} \right) + \left( {\rho _{S} \sigma _{{e_{R} }}^{2} + 1} \right) } \right) } \right) \\ \end{aligned} \end{aligned}$$

(25)

The results in (24) and (25) refer to limits of outage performance in the region of high SNR. It can be predicted that the outage performance of two ground users encounters the lower bound even though we improve other system parameters. As discussed, the diversity is then zero.

Ergodic capacity of $D_1$

The ergodic capacity of $x_i$ is expressed as [39]

$$\begin{aligned} R_{x_1 } = \frac{1}{{2\log \left( 2 \right) }}\int \limits _0^{\frac{{A_1 }}{{A_2 }}} {\frac{{1 - F_{Q_1 } \left( x \right) }}{{1 + x}}} dx, \end{aligned}$$

(26)

where $Q_1 = \min \left( {\varGamma _{R \rightarrow x_1 },\varGamma _{D_1 \rightarrow x_1 } } \right)$.

Proposition 4

The closed-form ergodic capacity of $x_1$ is given as (27), where $\varPsi _1 = \frac{{\left( {\rho _S \sigma _{e_R }^2 + 1} \right) \varXi _R \left( {1 + \theta _p } \right) }}{{A_2 \rho _S \left( {1 - \theta _p } \right) }} + \frac{{m_1 \left( {\rho _R \sigma _{e_1 }^2 + 1} \right) \left( {1 + \theta _p } \right) }}{{A_2 \varOmega _1 \rho _R \left( {1 - \theta _p } \right) }}$.

$$\begin{aligned} \begin{aligned} R_{{x_{1} }}&\approx \frac{{\alpha _{R} }}{{2\ln \left( 2 \right) }}\sum \limits _{{k = 0}}^{{m_{R} - 1}} {\sum \limits _{{l = 0}}^{k} {\sum \limits _{{p = 0}}^{l} {\sum \limits _{{b_{1} = 0}}^{{m_{1} - 1}} {\left( {\begin{array}{*{20}c} l \\ p \\ \end{array} } \right) \frac{{\xi \left( k \right) k!}}{{b_{1} !l!\varGamma \left( {m_{I} } \right) }}} } } } \frac{{\varGamma \left( {m_{I} + p} \right) }}{{\left( {\varXi _{R} } \right) ^{{k - l + 1}} }} \\&\quad \times \left( {\frac{{\left( {\rho _{R} \sigma _{{e_{1} }}^{2} + 1} \right) m_{1} }}{{\varOmega _{1} \rho _{R} }}} \right) ^{{b_{1} }} \left( {\frac{{\varOmega _{I} }}{{m_{I} \left( {\rho _{S} \sigma _{{e_{R} }}^{2} + 1} \right) }}} \right) ^{p} \left( {\frac{{\left( {\rho _{S} \sigma _{{e_{R} }}^{2} + 1} \right) }}{{\rho _{S} }}} \right) ^{l} \\&\quad \times \frac{\pi }{P}\sum \limits _{{p = 0}}^{P} {\frac{{A_{1} \sqrt{1 - \theta _{p}^{2} } e^{{ - \varPsi _{1} }} }}{{2A_{2} + A_{1} \left( {1 + \theta _{p} } \right) }}} \left( {1 + \frac{{\varOmega _{I} \varXi _{R} \left( {1 + \theta _{p} } \right) }}{{A_{2} m_{I} \rho _{S} \left( {1 - \theta _{p} } \right) }}} \right) ^{{ - m_{I} - p}} \left( {\frac{{\left( {1 + \theta _{p} } \right) }}{{A_{2} \left( {1 - \theta _{p} } \right) }}} \right) ^{{b_{1} + l}} \\ \end{aligned} \end{aligned}$$

(27)

Proof

See Appendix D.

Ergodic capacity $D_2$

Similarly, the ergodic capacity of $x_2$ is written as

$$\begin{aligned} R_{x_2 } = \frac{1}{{2\ln \left( 2 \right) }}\int \limits _0^\infty {\frac{{1 - F_{Q_2 } \left( y \right) }}{{1 + y}}} dy, \end{aligned}$$

(28)

where $Q_2 = \min \left( {\varGamma _{R \rightarrow x_2 },\varGamma _{D_1 \rightarrow x_2 } } \right)$.

Proposition 5

The closed-form ergodic capacity of $x_1$ is given as (29), where $\varPsi _2 = \frac{{\varXi _R \left( {\rho _S \sigma _{e_R }^2 + 1} \right) }}{{\rho _R A_2 }} + \frac{{m_2 \left( {\rho _R \sigma _{e_2 }^2 + 1} \right) }}{{\varOmega _2 \rho _R A_2 }}$ and $G_{1,\left[ {1:1} \right] ,0,\left[ {1:1} \right] }^{1,1,1,1,1}[.,.]$ denotes the Meijer-G function with two variables [42].

$$\begin{aligned} \begin{aligned} R_{{x_{2} }}&= \frac{{\alpha _{R} }}{{2\ln \left( 2 \right) }}\sum \limits _{{k = 0}}^{{m_{R} - 1}} {\sum \limits _{{l = 0}}^{k} {\sum \limits _{{p = 0}}^{l} {\sum \limits _{{b_{2} = 0}}^{{m_{2} - 1}} {\left( {\begin{array}{*{20}c} l \\ p \\ \end{array} } \right) } \frac{{k!\xi \left( k \right) \left( {\varXi _{R} } \right) ^{{ - k + l - 1}} }}{{b_{2} !l!\varGamma \left( {m_{I} } \right) }}} } } \\&\quad \times \left( {\frac{{m_{2} \left( {\rho _{R} \sigma _{{e_{2} }}^{2} + 1} \right) }}{{\varOmega _{2} \rho _{R} A_{2} }}} \right) ^{{b_{2} }} \left( {\frac{{\left( {\rho _{S} \sigma _{{e_{R} }}^{2} + 1} \right) }}{{\rho _{R} A_{2} }}} \right) ^{l} \left( {\frac{{\varOmega _{I} }}{{m_{I} \left( {\rho _{S} \sigma _{{e_{R} }}^{2} + 1} \right) }}} \right) ^{p} \\&\quad \times G_{{1,\left[ {1:1} \right] ,0,\left[ {1:1} \right] }}^{{1,1,1,1,1}} \left[ {\begin{array}{*{20}c} {\frac{{\varOmega _{I} \varXi _{R} }}{{m_{I} \rho _{R} A_{2} \varPsi _{2} }}} \\ {\frac{1}{{\varPsi _{2} }}} \\ \end{array} \left| {\begin{array}{*{20}c} {1 + l + b_{2} } \\ {1 - m_{I} - p} \\ - \\ {0,0} \\ \end{array} } \right. } \right] \\ \end{aligned} \end{aligned}$$

(29)

Proof

See Appendix E.

Consideration on case of multiple antenna relay

In this section, we consider how a multiple antennas relay makes influences to performance of two users $D_i$. In particular, a DF relay can be equipped with $K_R$ received antennas and $K_T$ transmit antenna. To represent mathematical expressions from now on, ${\mathbf {h}}_{R} = [h^1_{R},h^2_{R},...,h^{K_R}_{R}]^T$ is denoted the $K_R \times 1$ channel vector between and R, ${\mathbf {h}}_{i} = [h^1_{i},h^2_{i},...,h^{K_T}_{i}]^T$ denotes the $K_T \times 1$ channel vector between R and $D_i$. In this first phase, the signal received at R with help (1) is given by

$$\begin{aligned} \begin{aligned} y_R^{II}&= \left( {\sqrt{A_1 } x_1 + \sqrt{A_2 } x_2 } \right) \sqrt{P_S } {\mathbf {w}}_{R}^H \hat{{\mathbf {h}}}_{R} \\&\quad + \left( {\sqrt{A_1 } x_1 + \sqrt{A_2 } x_2 } \right) \sqrt{P_S } {\mathbf {w}}_{R}^H \mathbf {e}_{R} \\&\quad + \sum \limits _{n = 1}^N {\sqrt{P_{cn} } h_{nR} x_n } + {\mathbf {w}}_{R}^H \mathbf {n}_R \\ \end{aligned} \end{aligned}$$

(30)

where $\mathbf {n}_R$ denote the vector of zero mean AWGN with variance $N_0$ and ${\mathbf {w}}_{R}=\frac{\widehat{{\mathbf {h}}}_{R}}{\left\| {\hat{{\mathbf {h}}}_{R}} \right\| _F}$. Then, the SINR is then used to decode $x_1$ and given as

$$\begin{aligned} \varGamma ^{II} _{R \rightarrow x_1 } = \frac{{A_1 \eta _R }}{{A_2 \eta _R + \rho _S \sigma _{e_R }^2 + \gamma _C + 1}}, \end{aligned}$$

(31)

where $\eta _R = \rho _S \left\| {\hat{{\mathbf {h}}}_R } \right\| _F^2$. Then, the SINR decoded $x_2$ is given as

$$\begin{aligned} \varGamma ^{II} _{R \rightarrow x_2 } = \frac{{A_2 \eta _R }}{{\rho _S \sigma _{e_R }^2 + \gamma _C + 1}} \end{aligned}$$

(32)

In the second phase, the received at $D_i$ is expressed as

$$\begin{aligned} \begin{aligned} y^{II}_{D_i }&= \left( {\sqrt{A_1 } x_1 + \sqrt{A_2 } x_2 } \right) \sqrt{P_R } {{{\hat{{\mathbf {h}}} }_i}^H} {\mathbf {w}}_i \\&\quad + \left( {\sqrt{A_1 } x_1 + \sqrt{A_2 } x_2 } \right) \sqrt{P_R } \mathbf {e}_i + n_{D_i } \\ \end{aligned} \end{aligned}$$

(33)

where ${\mathbf {w}}_i = \frac{\hat{{\mathbf {h}}}_i}{\left\| {\hat{{\mathbf {h}}}}_i \right\| _F}$. The SINR which decodes $x_1$ at $D_1$ is given as

$$\begin{aligned} \varGamma ^{II} _{D_1 \rightarrow x_1 } = \frac{{\eta _1 A_1 }}{{\eta _1 A_2 + \rho _R \sigma _{e_1 }^2 + 1}}, \end{aligned}$$

(34)

where $\eta _i = \rho _R \left\| {\hat{{\mathbf {h}}}_i } \right\| _F^2$, the SINR which decodes signal $x_1$ at $D_2$ is given as

$$\begin{aligned} \varGamma ^{II} _{D_2 \rightarrow x_1 } = \frac{{\eta _2 A_1 }}{{\eta _2 A_2 + \rho _R \sigma _{e_2 }^2 + 1}} \end{aligned}$$

(35)

Similarly, the SINR which decodes its own signal $x_2$ at $D_2$ is computed according to

$$\begin{aligned} \varGamma ^{II} _{D_2 \rightarrow x_2 } = \frac{{\eta _2 A_2 }}{{\rho _R \sigma _{e_2 }^2 + 1}} \end{aligned}$$

(36)

Statistical characterization

In this section, we consider ${\hat{{\mathbf {h}}}_R }$ and ${\hat{{\mathbf {h}}}_i }$ have independent and identically distributed (i.i.d.) entries as [43]. In addition, the PDF of $\eta _R$ can be expressed by [44]

$$\begin{aligned} f_{\eta _R } \left( x \right) = \frac{{\alpha _R^{K_R } }}{{\varGamma \left( {K_R } \right) }}e^{ - \frac{{\beta _R }}{{\rho _S }}x} \sum \limits _{a = 0}^\infty {\frac{{\left( {m_R K_R } \right) _a \delta _R^a }}{{a!\left( {K_R } \right) _a \left( {\rho _S } \right) ^{K_R + a} }}} x^{K_R + a - 1} . \end{aligned}$$

(37)

With the help of [46, Eq. 3.351.2], we have the CDF of $\eta _R$ as

$$\begin{aligned} F_{\eta _R } \left( x \right) = 1 - \frac{{\alpha _R^{K_R } }}{{\varGamma \left( {K_R } \right) }}\sum \limits _{a = 0}^\infty {\sum \limits _{b = 0}^{K_R + a - 1} {\frac{{\varGamma \left( {K_R + a} \right) \left( {m_R K_R } \right) _a \delta _R^a }}{{a!b!\left( {K_R } \right) _a \left( {\rho _S } \right) ^b \left( {\beta _R } \right) ^{K_R + a - b} }}} } x^b e^{ - \frac{{\beta _R }}{{\rho _S }}x} . \end{aligned}$$

(38)

In addition, the CDF of $\eta _I$ is given by [44]

$$\begin{aligned} F_{\eta _i } \left( x \right) = \frac{{\gamma \left( {m_i K_T ,\frac{{m_i }}{{\rho _R \varOmega _i }}x} \right) }}{{\varGamma \left( {m_i K_T } \right) }} \end{aligned}$$

(39)

Outage probability

Proposition 6

The outage probability of $D_1$ is expressed as

$$\begin{aligned} \begin{gathered} P_{D_1 }^{II} = 1 - \frac{{\alpha _R^{K_R } }}{{\varGamma \left( {K_R } \right) \varGamma \left( {m_I } \right) }}\sum \limits _{a = 0}^\infty {\sum \limits _{b = 0}^{K_R + a - 1} {\sum \limits _{c = 0}^b {\left( {\begin{array}{c}b\\ c\end{array}}\right) \frac{{\varGamma \left( {K_R + a} \right) \left( {m_R K_R } \right) _a }}{{a!b!\left( {K_R } \right) _a }}} } } \\ \times \left( {\frac{{m_I }}{{\varOmega _I }}} \right) ^{m_I } \frac{{\varGamma \left( {m_I + c} \right) \delta _R^a \left( {\bar{\phi }_1 } \right) ^b e^{ - \frac{{\beta _R \bar{\phi }_1 \left( {\rho _S \sigma _{e_R }^2 + 1} \right) }}{{\rho _S }}} }}{{\left( {\rho _S } \right) ^b \left( {\beta _R } \right) ^{K_R + a - b} \left( {\rho _S \sigma _{e_R }^2 + 1} \right) ^{c - b} \varGamma \left( {m_1 K_T } \right) }} \\ \times \left( {\frac{{m_I }}{{\varOmega _I }} + \frac{{\beta _R \phi _1 }}{{\rho _S }}} \right) ^{ - m_I - c} \varGamma \left( {m_1 K_T ,\frac{{m_1 \bar{\phi }_1 \left( {\rho _R \sigma _{e_1 }^2 + 1} \right) }}{{\rho _R \varOmega _1 }}} \right) . \\ \end{gathered} \end{aligned}$$

(40)

Proof

See Appendix F.

Similarly, the outage probability of $D_2$ can be expressed by

$$\begin{aligned} \begin{gathered} P_{D_2 }^{II} = 1 - \frac{{\alpha _R^{K_R } }}{{\varGamma \left( {K_R } \right) \varGamma \left( {m_I } \right) }}\sum \limits _{a = 0}^\infty {\sum \limits _{b = 0}^{K_R + a - 1} {\sum \limits _{c = 0}^b {\left( {\begin{array}{c}b\\ c\end{array}}\right) \frac{{\varGamma \left( {K_R + a} \right) \left( {m_R K_R } \right) _a }}{{a!b!\left( {K_R } \right) _a }}} } } \\ \times \left( {\frac{{m_I }}{{\varOmega _I }}} \right) ^{m_I } \frac{{\varGamma \left( {m_I + c} \right) \delta _R^a \left( {\bar{\phi }_2 } \right) ^b e^{ - \frac{{\beta _R \bar{\phi }_2 \left( {\rho _S \sigma _{e_R }^2 + 1} \right) }}{{\rho _S }}} }}{{\left( {\rho _S } \right) ^b \left( {\beta _R } \right) ^{K_R + a - b} \left( {\rho _S \sigma _{e_R }^2 + 1} \right) ^{c - b} \varGamma \left( {m_2 K_T } \right) }} \\ \times \left( {\frac{{m_I }}{{\varOmega _I }} + \frac{{\beta _R \bar{\phi }_2 }}{{\rho _S }}} \right) ^{ - m_I - c} \varGamma \left( {m_2 K_T ,\frac{{m_2 \bar{\phi }_2 \left( {\rho _R \sigma _{e_2 }^2 + 1} \right) }}{{\rho _R \varOmega _2 }}} \right) \\ \end{gathered} \end{aligned}$$

(41)

where $\bar{\phi }_2 = \frac{\gamma _2}{A_2}$

Simulation results and discussion

Table 3 System parameters

In this section, we set $\rho _C= 1$ dB, $\rho = \rho _S = \rho _R$ and the main parameters given in Table 3. The Shadowed-Rician fading parameters for the satellite link are taken from [40] and shown in Table 4. Additionally, the interference channels parameters were set and calculated according to the respective analytical curves in [24] and are shown in Table 5. Moreover, we using MATLAB for Monte Carlo simulations.

Table 4 Satellite link parameters

Table 5 Interference link parameters

Figure 2 shows the outage performance versus the $\rho$ (dB) for different shadowing satellite links. We can observe that the performance of the system under AS is superior to the system under HS. That means satellite channel conditions contribute significantly to system performance at the ground users. We can also observe the difference in performance of the NOMA and OMA systems. In the OMA system, the gap between the two curves shows that with increased SNR, system performance increases similarly to the NOMA system. The authenticity of the derived expressions is also evident from the strict match of the Monte Carlo simulations with the analytical simulations.

Figure 3 represents how multiple antennas at relay contribute to improve the system performance at ground users. Once we design the relay with $K_T=K_R=3$, the big gap outage behavior can be observed compared with the case of the relay with $K_T=K_R=2$. The reason is that higher diversity from multiple antennas design could be strengthen signal received at ground users and hence outage performance could be improved thoroughly.

Figure 4 indicates the impact of imperfect CSI on outage probability when we change value of $\rho$ for case of HS. We can see how performance could be affected by such CSI error by varying $\sigma ^2$. An increase in the value of $\sigma ^2$ shows a reduction in the performance of the user, and for the lowest value of $\sigma ^2$, both users demonstrate better performance. As the SNR increases, the performance of both users continues to increase, while in similar conditions in Fig. 5, we varied the number of interfering links for both users by keeping $\sigma ^2$ constant. Considering impact of CCI concern, the simulation shows that with a greater number of interference links, the performance of both users decreases. However, in all the links, the curves for each user meet at a saturated point at high SNR. We conclude that in the high SNR region, interference links do not have a great effect on user performance.

Figure 6 shows the simulation for outage performance versus $\rho$ (dB) with the different satellite links as in Fig. 2. The ergodic capacity rates of the message at $D_1$ are almost similar in both HS and AS modes, but for messaging at $D_2$, the gap between the curves of ergodic capacity in both modes is comparatively very high. With a simultaneous increase in the SNR, the gap increase is unlike $D_1$.

Figures 7 and 8 indicate the several curves of ergodic capacity versus $\rho$ (dB) under HS. We can see the impact of CSI error levels of $\sigma ^2$ in Fig. 7 while Fig. 8 confirms how the number of CCI sources leads to degradation of performance in term of ergodic capacity. In Fig. 7, as we increase the value of $\sigma ^2$, the gap between the curves increases simultaneously at high SNR values. Figure 8 shows the ergodic capacity versus $\rho$ (dB) varying N with satellite link under HS. Although gaps between the curves are evident at medium SNR values, the curves for both the users meet at a single point at higher SNRs, suggesting that at higher SNRs, a greater number of interference links does not show much differential effect on ergodic capacity rates.

Fig. 2

Outage performance vs $\rho$ (dB) with different satellite links

Fig. 3

Outage performance vs $\rho$ (dB) varying $K_T = K_R$ with the satellite link under HS

Fig. 4

Outage performance vs $\rho$ (dB) varying $\sigma ^2$ with the satellite link under HS

Fig. 5

Outage performance vs $\rho$ (dB) varying N with the satellite link under HS

Fig. 6

Ergodic capacity vs $\rho$ (dB) with different satellite links

Fig. 7

Ergodic capacity vs $\rho$ (dB) varying $\sigma ^2$ with the satellite link under HS

Fig. 8

Ergodic capacity vs $\rho$ (dB) varying N with the satellite link under HS

Conclusion

We described the use of the NOMA technique for communication between a satellite to a relay and the relay to users. We investigated performance of the system in terms of outage probability and ergodic capacity under the effect of CCI at the relay. The performance gap between two destinations can be explained by the differences in power level allocated to the destinations. We also derived the closed-form expressions for outage probability and ergodic capacity at both users. We observed that with an increase in the channel error, both the outage probability and ergodic capacity of the users were significantly affected. The effect of CCI on both outage probability and ergodic capacity is more prominent when the SNR falls in range of 20–30 dB, whereas a greater number of interference links shows little effect in the high SNR region. More specifically, if, for example, the number of interference sources is 5, the outage performance of the system experiences a decrease of approximately 40% at a signal-to-noise ratio (SNR) of 30 dB at the satellite. Outage probability and ergodic capacity became saturated at SNRs of 50 dB and 45 dB, respectively. We also simulated and compared system performance under AS and HS modes for both outage probability and ergodic capacity.

Appendices

Appendix A

Using (3), $B_1$ is written as

$$\begin{aligned} \begin{aligned} B_1&= \Pr \left( {\frac{{\rho _S A_1 \left| {\hat{h}_R } \right| ^2 }}{{\rho _S A_2 \left| {\hat{h}_R } \right| ^2 + \gamma _C + \rho _S \sigma _{e_R }^2 + 1}}> \gamma _1 } \right) \\&= \Pr \left( {\left| {\hat{h}_R } \right| ^2 > \left( {\gamma _C + \rho _S \sigma _R^2 + 1} \right) \phi _1 } \right) , \\ \end{aligned} \end{aligned}$$

(42)

where $\phi _1 = \frac{{\gamma _{1 } }}{{\left( {A_1 - A_2 \gamma _{1 } } \right) \rho }}$. $B_1$ can therefore be expressed as

$$\begin{aligned} B_1 = \int \limits _0^\infty {f_{\gamma _C } } \left( x \right) \left( {1 - F_{\left| {\hat{h} _R } \right| ^2 } \left( {\left( {x + \rho _S \sigma _R^2 + 1} \right) \phi _1 } \right) } \right) dx. \end{aligned}$$

(43)

Substituting (11) and (12) into (43), $B_1$ is rewritten as

$$\begin{aligned} \begin{aligned} B_1&= \alpha _R \sum \limits _{k = 0}^{m_R - 1} {\sum \limits _{l = 0}^k {\frac{{\xi \left( k \right) k!\left( {\phi _1 } \right) ^l e^{ - \varXi _R \phi _1 \left( {\rho _S \sigma _R^2 + 1} \right) } }}{{l!\varGamma \left( {m_I } \right) \left( {\varXi _R } \right) ^{k - l + 1} }}} } \left( {\frac{{m_I }}{{\varOmega _I }}} \right) ^{m_I } \\&\quad \times \int \limits _0^\infty {x^{m_I - 1} } \left( {x + \rho _S \sigma _R^2 + 1} \right) ^l e^{ - \left( {\frac{{m_I + \varOmega _I \varXi _R \phi _1 }}{{\varOmega _I }}} \right) x} dx. \\ \end{aligned} \end{aligned}$$

(44)

Based on [46, Eq. 1.111] and [46, Eq. 3.381.4], the closed-form of $B_1$ is obtained as

$$\begin{aligned} \begin{aligned} B_1&= \alpha _R \sum \limits _{k = 0}^{m_R - 1} {\sum \limits _{l = 0}^k {\sum \limits _{p = 0}^l {\left( {\begin{array}{*{20}c} l \\ p \\ \end{array} } \right) } \frac{{\xi \left( k \right) k!\varGamma \left( {m_I + p} \right) }}{{l!\varGamma \left( {m_I } \right) \left( {\varXi _R } \right) ^{k - l + 1} }}} } e^{ - \varXi _R \phi _1 \left( {\rho _S \sigma _R^2 + 1} \right) } \\&\quad \times \left( {\phi _1 \left( {\rho _S \sigma _R^2 + 1} \right) } \right) ^l \left( {1 + \frac{{\varOmega _I \varXi _R \phi _1 }}{{m_I }}} \right) ^{ - m_I - p} \left( {\frac{{\varOmega _I }}{{m_I \left( {\rho _S \sigma _R^2 + 1} \right) }}} \right) ^p. \\ \end{aligned} \end{aligned}$$

(45)

This completes the proof.

Appendix B

Let us denote the first and second terms of (21) as $C_1$ and $C_2$, respectively. Using (4), $C_1$ can then be written as

$$\begin{aligned} \begin{aligned} C_1&= \Pr \left( {\frac{{\rho _S A_2 \left| {\hat{h}_R } \right| ^2 }}{{\gamma _C + \rho _S \sigma _{e_R }^2 + 1}}> \gamma _2 } \right) \\&= \Pr \left( {\left| {\hat{h}_R } \right| ^2 > \psi _1 \left( {\gamma _C + \rho _S \sigma _{e_R }^2 + 1} \right) } \right) \\&= \int \limits _0^\infty {f_{\gamma _C } } \left( x \right) F_{\left| {\hat{h}_R } \right| ^2 } \left( {\left( {x + \rho _S \sigma _{e_R }^2 + 1} \right) \psi _1 } \right) dx, \\ \end{aligned} \end{aligned}$$

(46)

where $\psi _1 = \frac{{\gamma _2 }}{{\rho _S A_2 }}$. As in Proposition 1, we obtain $C_1$ as

$$\begin{aligned} \begin{aligned} C_1&= \alpha _R \sum \limits _{k = 0}^{m_R - 1} {\sum \limits _{l = 0}^k {\sum \limits _{p = 0}^l {\left( {\begin{array}{*{20}c} l \\ p \\ \end{array} } \right) \frac{{k!\xi \left( k \right) \varGamma \left( {m_I + p} \right) }}{{l!\varGamma \left( {m_I } \right) \left( {\varXi _R } \right) ^{k - l + 1} }}} } } e^{ - \varXi _R \psi _1 \left( {\rho _S \sigma _{e_R }^2 + 1} \right) } \\&\quad \times \left( {\psi _1 \left( {\rho _S \sigma _{e_R }^2 + 1} \right) } \right) ^l \left( {1 + \frac{{\varOmega _I \varXi _R \psi _1 }}{{m_I }}} \right) ^{ - m_I - p} \left( {\frac{{\varOmega _I }}{{m_I \left( {\rho _S \sigma _{e_R }^2 + 1} \right) }}} \right) ^p. \\ \end{aligned} \end{aligned}$$

(47)

Next, $C_2$ is calculated as

$$\begin{aligned} \begin{aligned} B_2&= \Pr \left( {\frac{{\rho _R A_2 \left| {\hat{h}_2 } \right| ^2 }}{{\rho _R \sigma {e_2 }^2 + 1}}> \gamma _2 } \right) \\&= \Pr \left( {\left| {\hat{h}_2 } \right| ^2 > \psi _2 } \right) \\&= \sum \limits _{b_2 = 0}^{m_2 - 1} {\frac{{e^{ - \frac{{m_2 \psi _2 }}{{\varOmega _2 }}} }}{{b_2 !}}} \left( {\frac{{m_2 \psi _2 }}{{\varOmega _2 }}} \right) ^{b_2 }, \\ \end{aligned} \end{aligned}$$

(48)

where $\psi _2 = \frac{{\left( {{\rho _R \sigma _{e_2 }^2 + 1}} \right) \gamma _2 }}{{\rho _R A_2 }}$.

Using (47) and (48), the closed-form outage probability of $D_2$ is obtained as (22).

This completes the proof.

Appendix C

In the high SNR region, the CDF of $|\hat{h}_R|^2$ and $|\hat{h}_i|^2$ are, respectively, given s

$$\begin{aligned} F_{\left| {h_R } \right| ^2 }^\infty \left( x \right) = \alpha _R x \end{aligned}$$

(49)

and

$$\begin{aligned} F_{\left| {h_i } \right| ^2 } \left( x \right) = \frac{1}{{\varGamma \left( {m_i + 1} \right) }}\left( {\frac{{m_i }}{{\varOmega _i }}x} \right) ^{m_i }. \end{aligned}$$

(50)

Next, the asymptotic outage probability of $D_1$ is calculated as

$$\begin{aligned} P_{D_1 }^\infty = 1 - B_1^\infty \times B_2^\infty \end{aligned}$$

(51)

Then, $B_1^\infty$ is expressed by

$$\begin{aligned} \begin{aligned} B_1^\infty&= 1 - \Pr \left( {\left| {\hat{h}_R } \right| ^2 < \left( {\gamma _C + \rho _S \sigma _{e_R }^2 + 1} \right) \phi _1 } \right) \\&= 1 - \int \limits _0^\infty {f_{\gamma _C } \left( x \right) } F_{\left| {\hat{h}_R } \right| ^2 }^\infty \left( {\left( {x + \rho _S \sigma _{e_R }^2 + 1} \right) \phi _1 } \right) . \\ \end{aligned} \end{aligned}$$

(52)

Using (49) and (14), we can rewrite $B_1^\infty$ as

$$\begin{aligned} \begin{aligned} B_1^\infty&= 1 - \left( {\frac{{m_I }}{{\varOmega _I }}} \right) ^{m_I } \frac{{\alpha _R \phi _1 }}{{\varGamma \left( {m_I } \right) }}\int \limits _0^\infty {\frac{{x^{m_I - 1} e^{ - \left( {\frac{{m_I }}{{\varOmega _I }}} \right) x} }}{{\left( {x + \rho _S \sigma _{e_R }^2 + 1} \right) ^{ - 1} }}} \\&= 1 - \left( {\frac{{m_I }}{{\varOmega _I }}} \right) ^{m_I } \frac{{\alpha _R \varphi _1 }}{{\varGamma \left( {m_I } \right) }}\left( {\int \limits _0^\infty {x^{m_I } e^{ - \left( {\frac{{m_I }}{{\varOmega _I }}} \right) x} dx} } \right. \\&\quad + \left. {\left( {\rho _S \sigma _{e_R }^2 + 1} \right) \int \limits _0^\infty {x^{m_I - 1} } e^{ - \left( {\frac{{m_I }}{{\varOmega _I }}} \right) x} dx} \right) . \\ \end{aligned} \end{aligned}$$

(53)

$B_1^\infty$ is thus obtained by

$$\begin{aligned} B_1^\infty = 1 - \alpha _R \phi _1 \left( {\frac{{\left( {m_I } \right) !}}{{\varGamma \left( {m_I } \right) }}\left( {\frac{{\varOmega _I }}{{m_I }}} \right) + \left( {\rho _S \sigma _{e_R }^2 + 1} \right) } \right) . \end{aligned}$$

(54)

Then, the term $B_2^\infty$ can be calculated by

$$\begin{aligned} \begin{aligned} B_2^\infty&= \Pr \left( {\left| {\hat{h}_1 } \right| ^2 > \phi _2 } \right) \\&= 1 - F_{\left| {\hat{h}_1 } \right| ^2 }^\infty \left( {\phi _2 } \right) \\&= 1 - \frac{1}{{\varGamma \left( {m_1 + 1} \right) }}\left( {\frac{{m_1 \phi _2 }}{{\varOmega _1 }}} \right) ^{m_1 }. \\ \end{aligned} \end{aligned}$$

(55)

Substituting (54) and (55) into (51), we obtain (24).

This completes the proof.

Appendix D

First, the CDF of $Q_1$ is expressed as

$$\begin{aligned} \begin{aligned} F_{Q_1 } \left( x \right)&= \alpha _R \sum \limits _{k = 0}^{m_R - 1} {\sum \limits _{l = 0}^k {\sum \limits _{p = 0}^l {\sum \limits _{b_1 = 0}^{m_1 - 1} {\left( {\begin{array}{*{20}c} l \\ p \\ \end{array} } \right) \frac{{\xi \left( k \right) k!}{\varGamma \left( {m_I + p} \right) }}{{b_1 !l!\varGamma \left( {m_I } \right) }{\left( {\varXi _R } \right) ^{k - l + 1} }}} } } } \\&\quad \times \left( {\frac{{\left( {\rho _R \sigma _{e_1 }^2 + 1} \right) m_1 x}}{{\left( {A_1 - A_2 x} \right) \varOmega _1 \rho _R }}} \right) ^{b_1 } \left( {1 + \frac{{\varOmega _I \varXi _R x}}{{\left( {A_1 - A_2 x} \right) m_I \rho _S }}} \right) ^{ - m_I - p} \left( {\frac{{\varOmega _I }}{{m_I \left( {\rho _S \sigma _{e_R }^2 + 1} \right) }}} \right) ^p \\&\quad \times e^{ - \frac{{x\left( {\rho _S \sigma _{e_R }^2 + 1} \right) \varXi _R }}{{\left( {A_1 - A_2 x} \right) \rho _S }} - \frac{{\left( {\rho _R \sigma _{e_1 }^2 + 1} \right) m_1 x}}{{\left( {A_1 - A_2 x} \right) \varOmega _1 \rho _R }}} \left( {\frac{{x\left( {\rho _S \sigma _{e_R }^2 + 1} \right) }}{{\left( {A_1 - A_2 x} \right) \rho _S }}} \right) ^l. \\ \end{aligned} \end{aligned}$$

(56)

Substituting (56) into (26), we obtain $R_{x_1}$ as

$$\begin{aligned} \begin{aligned} R_{x_1 }&= \frac{{\alpha _R }}{{2\ln \left( 2 \right) }}\sum \limits _{k = 0}^{m_R - 1} {\sum \limits _{l = 0}^k {\sum \limits _{p = 0}^l {\sum \limits _{b_1 = 0}^{m_1 - 1} {\left( {\begin{array}{*{20}c} l \\ p \\ \end{array} } \right) \frac{{\xi \left( k \right) k!}{\varGamma \left( {m_I + p} \right) }}{{b_1 !l!\varGamma \left( {m_I } \right) }{\left( {\varXi _R } \right) ^{k - l + 1} }}} } } } \\&\quad \times \left( {\frac{{\left( {\rho _R \sigma _{e_1 }^2 + 1} \right) m_1 }}{{\varOmega _1 \rho _R }}} \right) ^{b_1 } \left( {\frac{{\varOmega _I }}{{m_I \left( {\rho _S \sigma _{e_R }^2 + 1} \right) }}} \right) ^p \left( {\frac{{\left( {\rho _S \sigma _{e_R }^2 + 1} \right) }}{{\rho _S }}} \right) ^l \\&\quad \times \int \limits _0^{A_1 /A_2 } {\frac{1}{{1 + x}}} \left( {1 + \frac{{\varOmega _I \varXi _R x}}{{\left( {A_1 - A_2 x} \right) m_I \rho _S }}} \right) ^{ - m_I - p} \left( {\frac{x}{{\left( {A_1 - A_2 x} \right) }}} \right) ^{b_1 + l} \\&\quad \times e^{ - \frac{{\left( {\rho _S \sigma _{e_R }^2 + 1} \right) \varXi _R x}}{{\left( {A_1 - A_2 x} \right) \rho _S }} - \frac{{\left( {\rho _R \sigma _{e_1 }^2 + 1} \right) m_1 x}}{{\left( {A_1 - A_2 x} \right) \varOmega _1 \rho _R }}}. \\ \end{aligned} \end{aligned}$$

(57)

Using the Gaussian–Chebyshev property and denoting $\theta _p = \cos \left( {\frac{{2p - 1}}{{2P}}\pi } \right)$, the closed-form ergodic capacity of $x_1$ (27) can be obtained.

This completes the proof.

Appendix E

Similarly, the CDF of $Q_2$ is expressed as

$$\begin{aligned} \begin{aligned} F_{Q_2 } \left( y \right)&= 1 - \alpha _R \sum \limits _{k = 0}^{m_R - 1} {\sum \limits _{l = 0}^k {\sum \limits _{p = 0}^l {\sum \limits _{b_2 = 0}^{m_2 - 1} {\left( {\begin{array}{*{20}c} l \\ p \\ \end{array} } \right) } \frac{{k!\xi \left( k \right) }{\varGamma \left( {m_I + p} \right) }}{{b_2 !l!\varGamma \left( {m_I } \right) }{\left( {\varXi _R } \right) ^{k - l + 1} }}} } } \\&\quad \times \left( {\frac{{m_2 \left( {\rho _R \sigma _{e_2 }^2 + 1} \right) }}{{\varOmega _2 \rho _R A_2 }}} \right) ^{b_2 } \left( {\frac{{\left( {\rho _S \sigma _{e_R }^2 + 1} \right) }}{{\rho _R A_2 }}} \right) ^l \left( {\frac{{\varOmega _I }}{{m_I \left( {\rho _S \sigma _{e_R }^2 + 1} \right) }}} \right) ^p \\&\quad \times \frac{{y^{l + b_2 } }}{{\left( {1 + \frac{{\varOmega _I \varXi _R }}{{m_I \rho _R A_2 }}y} \right) ^{m_I + p} }}e^{ - \varPsi _2 y}, \\ \end{aligned} \end{aligned}$$

(58)

where $\varPsi _2 = \frac{{\varXi _R \left( {\rho _S \sigma _{e_R }^2 + 1} \right) }}{{\rho _R A_2 }} + \frac{{m_2 \left( {\rho _R \sigma _{e_2 }^2 + 1} \right) }}{{\varOmega _2 \rho _R A_2 }}$. Next, we can calculate $R_{x_2}$ as

$$\begin{aligned} \begin{aligned} R_{x_2 }&= \frac{{\alpha _R }}{{2\ln \left( 2 \right) }}\sum \limits _{k = 0}^{m_R - 1} {\sum \limits _{l = 0}^k {\sum \limits _{p = 0}^l {\sum \limits _{b_2 = 0}^{m_2 - 1} {\left( {\begin{array}{*{20}c} l \\ p \\ \end{array} } \right) } \frac{{k!\xi \left( k \right) }{\varGamma \left( {m_I + p} \right) }}{{b_2 !l!\varGamma \left( {m_I } \right) }{\left( {\varXi _R } \right) ^{k - l + 1} }}} } } \\&\quad \times \left( {\frac{{m_2 \left( {\rho _R \sigma _{e_2 }^2 + 1} \right) }}{{\varOmega _2 \rho _R A_2 }}} \right) ^{b_2 } \left( {\frac{{\left( {\rho _S \sigma _{e_R }^2 + 1} \right) }}{{\rho _R A_2 }}} \right) ^l \left( {\frac{{\varOmega _I }}{{m_I \left( {\rho _S \sigma _{e_R }^2 + 1} \right) }}} \right) ^p \\&\quad \times \int \limits _0^\infty {\frac{{y^{l + b_2 } e^{ - \varPsi _2 y} }}{{\left( {1 + y} \right) \left( {1 + \frac{{\varOmega _I \varXi _R }}{{m_I \rho _R A_2 }}y} \right) ^{m_I + p} }}}. \\ \end{aligned} \end{aligned}$$

(59)

Using [41], we have

$$\begin{aligned} \left( {1 + ax} \right) ^{ - b} = \frac{1}{{\varGamma \left( b \right) }}G_{1,1}^{1,1} \left[ {ax\left| {\begin{array}{*{20}c} {1 - b} \\ 0 \\ \end{array} } \right. } \right] \end{aligned}$$

(60)

where $G_{1,1}^{1,1} [.,.]$ is the Meijer G-function [46]. Substituting (60) into (59), $R_{x_2 }$ is rewritten as

$$\begin{aligned} \begin{aligned} R_{x_2 }&= \frac{{\alpha _R }}{{2\ln \left( 2 \right) }}\sum \limits _{k = 0}^{m_R - 1} {\sum \limits _{l = 0}^k {\sum \limits _{p = 0}^l {\sum \limits _{b_2 = 0}^{m_2 - 1} {\left( {\begin{array}{*{20}c} l \\ p \\ \end{array} } \right) } \frac{{k!\xi \left( k \right) \left( {\varXi _R } \right) ^{l - k - 1} }}{{b_2 !l!\varGamma \left( {m_I } \right) }}} } } \\&\quad \times \left( {\frac{{m_2 \left( {\rho _R \sigma _{e_2 }^2 + 1} \right) }}{{\varOmega _2 \rho _R A_2 }}} \right) ^{b_2 } \left( {\frac{{\left( {\rho _S \sigma _{e_R }^2 + 1} \right) }}{{\rho _R A_2 }}} \right) ^l \left( {\frac{{\varOmega _I }}{{m_I \left( {\rho _S \sigma _{e_R }^2 + 1} \right) }}} \right) ^p \\&\quad \times \int \limits _0^\infty {y^{l + b_2 } e^{ - \varPsi _2 y} } G_{1,1}^{1,1} \left[ {x\left| {\begin{array}{*{20}c} 0 \\ 0 \\ \end{array} } \right. } \right] G_{1,1}^{1,1} \left[ {\frac{{\varOmega _I \varXi _R }}{{m_I \rho _R A_2 }}y\left| {\begin{array}{*{20}c} {1 - m_I - p} \\ 0 \\ \end{array} } \right. } \right] \\ \end{aligned} \end{aligned}$$

(61)

Based on [45, 2.6.2], (10) is obtained.

This completes the proof.

Appendix F

The outage probability of $D_1$ can be expressed by

$$\begin{aligned} P_{D_1 }^{II} = 1 - \underbrace{\Pr \left( {\varGamma _{R \rightarrow x_1 }^{II}> \gamma _i } \right) }_{F_1 }\underbrace{\Pr \left( {\varGamma _{D_1 \rightarrow x_1 }^{II} > \gamma _i } \right) }_{F_2 } \end{aligned}$$

(62)

With the help of (31), the first term $F_1$ is obtained by

$$\begin{aligned} \begin{aligned} F_1&= \Pr \left( {\frac{{A_1 \eta _R }}{{A_2 \eta _R + {\rho _S } \sigma _{e_R }^2 + \gamma _C + 1}}> \gamma _1 } \right) \\&= \Pr \left( {\eta _R > \bar{\phi }_1 \left( {{\rho _S } \sigma _{e_R }^2 + \gamma _C + 1} \right) } \right) \\&= \int \limits _0^\infty {f_{\gamma _C } } \left( x \right) \left( {1 - F_{\eta _R } \left( {\bar{\phi }_1 \left( {\rho \sigma _{e_R }^2 + x + 1} \right) } \right) } \right) dx, \\ \end{aligned} \end{aligned}$$

(63)

where $\bar{\phi }_1 = \frac{{\gamma _1 }}{{\left( {A_1 - A_2 \gamma _1 } \right) }}$. Based on (14) and (37), $F_1$ can be calculated as

$$\begin{aligned} \begin{aligned} F_1&= \frac{{\alpha _R^{K_R } }}{{\varGamma \left( {K_R } \right) \varGamma \left( {m_I } \right) }}\left( {\frac{{m_I }}{{\varOmega _I }}} \right) ^{m_I } \sum \limits _{a = 0}^\infty {\sum \limits _{b = 0}^{K_R + a - 1} {\frac{{\varGamma \left( {K_R + a} \right) \left( {m_R K_R } \right) _a }}{{a!b!\left( {K_R } \right) _a }}} } \\&\quad \times \frac{{\delta _R^a \left( {\phi _1 } \right) ^b e^{ - \frac{{\beta _R \bar{\phi }_1 \left( {\rho _S \sigma _{e_R }^2 + 1} \right) }}{{\rho _S }}} }}{{\left( {\rho _S } \right) ^b \left( {\beta _R } \right) ^{K_R + a - b} }}\int \limits _0^\infty {x^{m_I - 1} } \left( {\rho _S \sigma _{e_R }^2 + x + 1} \right) ^b e^{ - \left( {\frac{{m_I }}{{\varOmega _I }} + \frac{{\beta _R \bar{\phi }_1 }}{{\rho _S }}} \right) x} \\ \end{aligned} \end{aligned}$$

(64)

Then, the second term of (63) can be calculated by

$$\begin{aligned} \begin{aligned} F_2&= \Pr \left( {\eta _1 > \left( {\bar{\phi }_1 \left( {\rho _R \sigma _{e_1 }^2 + 1} \right) } \right) } \right) \\&= 1 - F_{\eta _1 } \left( {\bar{\phi }_1 \left( {\rho _R \sigma _{e_1 }^2 + 1} \right) } \right) \\&= \frac{1}{{\varGamma \left( {m_1 K_T } \right) }}\varGamma \left( {m_1 K_T ,\frac{{m_1 \bar{\phi }_1 \left( {\rho _R \sigma _{e_1 }^2 + 1} \right) }}{{\rho _R \varOmega _1 }}} \right) \\ \end{aligned} \end{aligned}$$

(65)

Putting (64) and (14) into (62), we have expected result. The proof is completed.