The Hybrid Interference Cancellation (HIC) is composed of two parts: The first one (Sect. 3.1) proceeds to decode each user signal from overlapped signal. Due to interference, each decoded signal has errors. To reduce BER, LDPC decoder (Sect. 3.2) is used for each user signal.
The Sect. 3.1 presents the HIC decoding either with hard decoding (Sect. 3.1.1) or SIC decoding (Sect. 3.1.2). In Sect. 3.1.3, we propose a complexity model.
After that, the LDPC decoder (Sect. 3.2) receives and estimates HIC signal for user j and decodes it to estimate the best probability errorless transmitted signal \(d^j_{m_o}  m_o=1,\ldots ,m\).
The decoder scheme shows the complete process, where LDPC decoding is done for each user at the output of LogMPA decoder as presented of Fig. 4. Concerning the complexity, Hard decoding is less complex compared to SIC decoding but the BER of it is greater which yields to more iterations for LDPC decoder.
HIC decoding complexity
Hybrid Interference Cancellation model is a combination of hard and soft decoding. If SINR is greater than a threshold, then the strongest signal is first hard decoding and the remaining signals, (\(d_{J1}\) users) are decoded with SIC decoder. If SINR is lower than the threshold, then only SIC decoder is used to decode \(d_J\) users per frequency.
Firstly, whatever is the decoder (Soft or Hard), the euclidean distances \(\Gamma (k,m)\) between the received signal at layer frequency k and all possible expected received codewords (from 1 to m) is estimated (Fig. 5).
$$\begin{aligned} \Psi _k(y,m,h)=\frac{{(y_k\sqrt{p_{k,j}}h_{k,j}x_{k,m,j})}^2}{2 \times \sigma _{\mathrm{eq}}} \end{aligned}$$
(21)
where k is the kth frequency layer, j is the jth user in the frequency layer which is considered to be the strongest received signal, m the length of SCMA codeword and \(\sigma _{\mathrm{eq}}=\sum _{i=1}^{J1}\sqrt{p_i}h_i^2+N_0\) with \(N_0\) is the power of white additive noise.
If channel is perfectly known, hard decoding process is easier and
$$\begin{aligned} \Gamma _{k,m,h}=(y_k\sqrt{p_{k,j}}h_{k,j}x_{k,m,j})^2 \end{aligned}$$
(22)
Hard decoding
Hard decoding is based on Gaussian Approximation (GA) which consider users interference as a Gaussian noise. We suppose decreasing user channel index as Eq. (3).
To decode user J, Eq. (2) is considered as:
$$\begin{aligned} y=\sqrt{p_J}{h_J}{x_J}+\sum _{j=1 }^{J1}\sqrt{p_j}{h_j}{x_j}+{n} \end{aligned}$$
(23)
The above equation can be written as \(y=\sqrt{p_J}{h_J}{x_J}+I\) where I is the interference from other users and noise.
On the basis of Eq. (21), LLR values are estimated only for the best received signal per frequency layer.
The complexity of hard decoding for the stronger signals (log domain) is:
$$\begin{aligned} \begin{aligned} M*K*3 \end{aligned} \end{aligned}$$
(24)
When hard decoder is used, signal superposition of \(d_J1\) users in each frequency layer is left. The values of LLR for remaining \(d_J1\) users are sent to SIC decoder.
Successive interference cancellation complexity model
SIC algorithm runs in 3 steps:
Initialization At each frequency layer, initialization aims at estimating error probabilities of the received signal for all possible messages \(x_{k,j}\) of each interference users (\(d_J\) interference users).
Assuming that both noise power and equalization are estimated, then for one layer frequency, one user and for one Muple message decoding, the log complexity is \(d_J+1\) additions and \(d_J+2\) multiplications.
As a result, initialization complexity of SIC is estimated to:
\(M^{d_J}*d_J*K*\left( 2d_J+3\right)\)
Iteration The Marginal probability is computed for each layer k, for each message \(m_i\) and for each user i. As a result, the log Marginal probability complexity of SIC system is \(d_J*M^{d_J+1}*K+J*M*\log _2(M)*(2*M+1)\) at each iteration.
Extrinsic estimation
The complexity of the extrinsic estimation is \(\left( (2+d_J)M+\log _2(M)\right) J\).
As a result, the SIC complexity per symbol is:
$$\begin{aligned} \begin{aligned} M^{d_J}*d_J*K*\left( 2d_J+3\right) +\left[ (2+d_J)M+\log _2(M)\right] J+\\ N_{\mathrm{iter}}\left[ d_J*M^{d_J+1}*K+J*M*\log _2(M)*(2*M+1)\right] \end{aligned} \end{aligned}$$
(25)
where \(N_{\mathrm{iter}}\) is the number of iterations of MPA message.
After that, the threshold stage is used to define if hard decoding can be used for the strongest signal.
Threshold estimation
Based on these values of LLR, soft symbol \({\hat{x}}_J\) is calculated for stronger user in each frequency layer. The complexity is estimated according to Eq. (24).
Once the best signal is decoded, the complexity of HICSCMA is similar to SIC decoder (Eq. 25) with \(d_J1\) users instead of \(d_J\) users, which results in a complexity reduction:
$$\begin{aligned}&M*K*3+M^{d_J1}*(d_J1)*K*\left( 2d_J+1\right) +\left[ (1+d_J)M+\log _2(M)\right] J\\&\quad +N_{\mathrm{iter}}\left[ (d_J1)*M^{d_J}*K+J*M*\log _2(M)*(2*M+1)\right] \end{aligned}$$
(26)
Let:
$$\begin{aligned} \Upsilon _{k,m}=\sqrt{p_{k,m,J}}h_{k,m,j}x_{k,m,J} \;\forall \ k=1,\ldots ,K \& \; m=1,\ldots ,M \end{aligned}$$
(27)
where J is the best received signal user.
According to Eqs. (21) and (22) , only SINR measurement is available to define a rule between soft detection and hard detection. To find it out, we first compute euclidean distance between the estimated soft symbol and all possible outputs computed in Eq. (28):
$$\begin{aligned} \Gamma _{k,m}=\Upsilon _{k,m}{\hat{x}}_{k,j}^2 \;\forall \ k=1,\ldots ,K \& \; m=1,\ldots ,M \end{aligned}$$
(28)
Equation (28) defines the Euclidean distance between the received signal y and all possible transmitted SCMA code (m = 1 to M) for only one user (the strongest received signal). Among M possibilities, we suppose that the minimum \(\Gamma _{k,m}\) is the SINR of the reconstructed signal: \({\text {SINR}}_{d_j}=\min (\Gamma _{k,m})\)
For a target BER and a LDPC decoder, we define a threshold value \(\Gamma\) for which if the following inequalities are satisfied, then Hard decoding can be used for the strongest signal:
$$\begin{aligned} \Gamma _{k,m} \geqslant \Gamma \end{aligned}$$
(29)
After interference cancellation, the estimated signals are transmitted to LDPC decoder. Equation (20) gives a lower bound of iteration loop \(N_{\epsilon }\) for which the LDPC decoder converges to the target probability error.
For a regular LDPC(3,6), whose efficiency is 1/2, Fig. 6 shows BER versus SNR when several iterative processes are done. For example, to reach a target BER \(Pe=10{10}\) for SNR = 4 dB then at least 6 iterations are needed.
If LPDC does not converge after \(N_{\epsilon }\) iterations, then the collapsed signal cannot be estimated and BER increases with decreasing channel user index.
Let \(D_{K,M}\) the decision matrix. \(D_{K,M}\) is initialized as zero matrix and Value of \(D_{k,m}\) is set to 1 for the points where Eq. (28) is satisfied. Values from decision matrix are used to decide between hard estimation or soft estimation as presented in the following algorithm 1
Interference cancellation (IC) is performed at each time interval instead of waiting for the whole input of user to be decoded for performing IC.
LDPC complexity
Concerning complexity, for a LDPC(m, n) the number of VN is n and the number of CN is \(nm\). As a result, using Sum Product Algorithm (SPA), the complexity can be approximated by [29]:

Initialization 3n operations (multiplication and division)

CN Update: \((2d_c1+n.d_v).(nm)\) operations

VN Update: \(d_v.n\) operations

Hard comparison: n operations
As a result, the LDPC complexity to decode one LDPC stream with target \(P_e\) is estimated by
$$\begin{aligned} 4n+N_\epsilon .\left[ 2\left( d_c1+n.d_v\right) .\left( nm\right) +d_v.n\right] \end{aligned}$$
(30)
with \(N_\epsilon\) defined in Eq. (20).
Complete receiver complexity or Joint HICLDPC complexity
On the receiver side, either SIC or HICMPA is first executed to separate each user. After SCMA decoding, LDPC is used for each user stream to recover the transmitted signal.
Equation (25) gives the algorithm complexity for SIC decoding per symbol. Equation (26) gives the algorithm complexity for HICMPA decoding per symbol. As a result, the difference of complexity between SIC and HICMPA per symbol is given by the following equation:
$$\begin{aligned}&\Delta _{\mathrm{HICMPA}}=M*J3*M*K\\&\quad +M^{d_J1}*(2d_J+1)*(d_J1)*K* [M*\frac{d_J}{d_J1}*\frac{2d_J+3}{2d_J+1}1]\\&\quad +N_{\mathrm{iter}}*(d_J1)*M^{d_J}*K*[\frac{d_J}{d_J1}*M1]\end{aligned}$$
(31)
Let \(N_{\mathrm{iter}}\) the number of iterations of SIC iterative decoding, M the codebook size, \(d_J\) the degree of signal superposition in each frequency layer, K the number of frequency layers.
Even if complexity of HICMPA is lower than SIC, the LDPC decoder needs more iterations to recover errorless signal (6). In the sequels, we aim at finding conditions for which joint decoder is less complex than SIC decoding. The condition is based on SNR and SINR power levels.
In the sequels, SNR refers to the global power of received overlapped SCMA codebook over noise and SINR refers to the power of one SCMA block over noise.
Let:

\(N_{\epsilon }\) the number of iterations estimated at LDPC decoder to reach a target BER according to Eq. (20),

\({\text {SNR}}^k=\frac{P_T^k}{N_0}^k\) where \(P_T^k\) is the cumulative power of each user at frequency layer k on the receiver side and \(N_0^k\) is the noise power at frequency layer k.
We note that \(P^k_j=\alpha ^k_j*P_T^k\), where \(\alpha _j^k\) is the fraction of power for user j of the receiver at frequency layer k.
For the sake of simplicity, the frequency layer reference k is not used in the sequel. In case of \(d_J\) interference signals, without loss of generality, we suppose that \(\alpha _{d_J} \ge \alpha _{d_J1} \ge \cdots \ge \alpha _{1}\). The SIC algorithm aims at recovering each user information \(d_j\) by successive iterative cancellation when received signal is considered to be \(y=\sum _{j=1}^{d_J}d_j+n\) with n as a Gaussian noise and \({\text {SNR}}=\frac{P_T}{N_0}\) defines the power of cumulative \(d_J\) received signals over the Gaussian noise power.
Let \({\text {SINR}}_{d_Jj}\) the SINR of resulting signal after \(d_J(j+1)\) cancellation : \(y(j)=d_{Jj}+\sum _{l=1}^{d_J(j+1)}d_l+n\) .
$$\begin{aligned} {\text {SINR}}_{d_Jj}=\frac{P_{d_Jj}}{N_0+\sum _{l=1}^{d_J(j+1)}.P_l} \end{aligned}$$
(32)
With the following notation, \(P_j=\alpha _j*P_T\), we can write
$$\begin{aligned} {\text {SINR}}_{d_Jj}=\frac{\alpha _{d_Jj}}{1/SNR+\sum _{l=1}^{d_J(j+1)}\alpha _l} \end{aligned}$$
(33)
For \(j=0\) (strongest received signal decoding first):
$$\begin{aligned} {\text {SINR}}_{d_J}=\frac{\alpha _{d_J}}{\frac{1}{{\text {SNR}}}+\sum _{l=1}^{d_J1}.\alpha _l} \end{aligned}$$
(34)
Since \(\sum _{l=1}^{d_J} \alpha _j=1\) then the last equation can be written:
$$\begin{aligned} {\text {SINR}}_{d_J}=\frac{\alpha _{d_J}}{\frac{1}{{\text {SNR}}}+1\alpha _{d_J}} \end{aligned}$$
(35)
For a target BER \(P_e\), the number of iterations \(N_\epsilon\) of LDPC is reached when \(\mu ^{\mathrm{VN}}(l)=2.erfc^{1}{(2.P_e)}^2\).
Our goal is to find a threshold value \(\Gamma\) for which if \({\text {SINR}}_{d_J} \ge \Gamma\) then HICMPA decoding is used, otherwise SIC decoding is processed. According to Eqs. (20) and (30):
Since SINR depends on SNR (35), for a given target BER, i.e. \(P_e\), for a given SNR and for a choosen LDPC(n,m) decoder, the difference of LDPC complexity depends on the value of \(\alpha _{d_J}\):
$$\begin{aligned} \begin{aligned} \Delta _{\mathrm{LDPC}}= \left[ 2\left( d_c1+n.d_v\right) .\left( nm\right) +d_v.n\right] .\\ \left( N_{\epsilon }({\text {SINR}}_{d_J})N_{\epsilon }({\text {SNR}}) \right) \end{aligned} \end{aligned}$$
(38)
With the target BER, we can write:
$$\begin{aligned}&\Delta _{\mathrm{LDPC}}=\\&\quad \left[ 2\left( d_c1+n.d_v\right) .\left( nm\right) +d_v.n\right] .ln \left( x_l2.erfc^{1}{(2.P_e)}^2 \right) \\&\quad \left[ \frac{1}{\ln \left( \left( \frac{{\mathrm{d}}f}{{\mathrm{d}}x}\right) {{x=SINR}_{d_J}}\right) }  \frac{1}{\ln \left( \left( \frac{{\mathrm{d}}f}{{\mathrm{d}}x}\right) {{x=SNR}}\right) }\right] \end{aligned}$$
(39)
Since LDPC complexity is greater for lower SNR, the SINR threshold gives a value for which complexity of HICMPA and LDPC decoder is lower than complexity of MUD and LDPC decoder.
Let \(y=\sum _{k=1}^{d_J}x_k+n\) be the received signal on receiver side.

According to HICMPA algorithm, the first step proceeds to estimate the strongest signal \(x_{d_J}\) from interference :\(y=x_{d_J}.+I\) where \(I=\sum _{k=1}^{d_J1}.x_k+n\).

According to MUD, the successive cancellation leads to find each useful signal \(x_k\) in presence of noise n.
The global complexity is given by Eqs. (25), (26), (30). Equation (31) gives the difference of complexity between MUD and HICMPA without LDPC. As a result, the complexity of decoder HICMPA is lower than that of MUD if
$$\begin{aligned} \Delta _{{\mathrm{LDPC}}(d_J,{\mathrm{SNR}},n,m)} < \frac{n}{\log _2(M)} \Delta _{\mathrm{HICMPA}} \end{aligned}$$
(40)
with n as the length of the LDPC codeword.
Either MUD or HIClog MPA is executed according to SNR level compared to \(\Gamma\) threshold. According to Eq. (29):

if \({\text {SINR}}_{d_J} < \Gamma\), classical MUD detection is done. The complexity of MUD is defined by Eq. (25);

if \({\text {SINR}}_{d_J} \ge \Gamma\), then Gaussian detection is done for user \(d_J\) and HICMPA is computed for other users. The complexity of MUD is defined by Eq. (26).
Equation (40) gives the complexity comparison between MUD and HICMPA and the limit of 40 allows to find the value of \(\Gamma\) for which MUD becomes more attractive compare to HICMPA.