We consider the downlink transmission (from the satellite to the user) of a geosynchronous orbit HTS that deploys full frequency reuse to provide services for fixed users in the satellite coverage area. The multibeam antenna on the satellite adopts the single feed per beam (SFB) structure and creates *N* beams corresponding to *N* feeds with the set \({\mathcal {N}} = \left \{ {1,2, \cdots,N} \right \}\), as shown in Fig. 1. In beam \(n\left ({n \in {\mathcal {N}}} \right), M_{n}\) single-antenna users are distributed in the beam area with the user set \({{\mathcal {M}}_{n}} = \left \{ {1,2, \ldots,{M_{n}}} \right \}\). The data of the users in each beam are embedded in frames. This leads to the multicast transmission. We assume that the transmission of frames in different beams are synchronized and the number of users involved in the transmission of each frame is lower than a fixed number *R*. Obviously, \(R \ll \left | {{{\mathcal {M}}_{n}}} \right |\). User scheduling should sequentially determine the users participating in the transmission of each frame among all the users in these beams. In this paper, user scheduling is decoupled into intra-beam and inter-beam scheduling. The intra-beam scheduling, which is performed in each beam, divides the *M*_{n} users into *K*_{n}*frame groups* with the group set \({{\mathcal {K}}_{n}} = \left \{ {1,2, \ldots,{K_{n}}} \right \}\) and the corresponding user set \(\left \{ {{\mathcal {U}}_{1}^{\left (n \right)},{\mathcal {U}}_{2}^{\left (n \right)}, \ldots,{\mathcal {U}}_{{K_{n}}}^{\left (n \right)}} \right \}\) that satisfies \({\sum \nolimits }_{k = 1}^{{K_{n}}} {\left | {{\mathcal {U}}_{k}^{\left (n \right)}} \right |} = {M_{n}}\). Without loss of generality, we assume that the number of users in each beam is equal, i.e., \({M_{n}} = M,\forall n \in {\mathcal {N}}\). This assumption implies that each beam has the same number of frame groups, i.e., \({K_{n}} = K,\forall n \in {\mathcal {N}}\). The inter-beam scheduling partitions the frame groups of different beams into *K**multiplexed groups*\(\left \{ {{{\mathcal {G}}_{1}},{{\mathcal {G}}_{2}}, \ldots,{{\mathcal {G}}_{K}}} \right \}\) with the group set \({\mathcal {K}} = \left \{ {1,2, \ldots,K} \right \}\). Each multiplexed group \({{\mathcal {G}}_{k}}\left ({k \in {\mathcal {K}}} \right)\) consists of *N* frame groups, i.e., \(\left | {{{\mathcal {G}}_{k}}} \right | = N,\forall k \in {\mathcal {K}}\), respectively, corresponding to frame group \({l_{k,n}} \in {{\mathcal {K}}_{n}}\) in beam *n*. After the precoding processing, the *N* frames groups in a multiplexed group are simultaneously transmitted with the same frequency band. It takes *K* successive frames to complete the transmission for all the users.

The diagram of multicast transmission with user scheduling for *N*=2 and *K*=3 is presented as an example in Fig. 2. The output of the intra-beam scheduling is the three frame groups in each beam. For each frame group, the data of the users are embedded in the same frame. Then, three multiplexed groups are obtained by the inter-beam scheduling. Each multiplexed group consists of two frames groups respectively from the two beams, and the data of the users in the two frames groups are transmitted with the same frequency band after precoding.

### Channel model

We assume that the channel is flat-fading and the influence of inter-symbol interference is ignored. The channel coefficients between each user and the feeds can be expressed as a complex vector, and the channel vector of user \(j \in {{\mathcal {M}}_{n}}\) in beam *n* is indicated as \(\mathbf {h}_{j}^{\left (n \right)} \in {\mathbb {C}^{1 \times N}}\). The *i*th element stands for the channel coefficient between user *j* and feed \(i \in {\mathcal {N}}\), which can be expressed as

$$ h_{j,i}^{\left(n \right)} = G_{j,i}^{\left(n \right)}{e^{j\varphi_{j,i}^{\left(n \right)}}}. $$

(1)

\(G_{j,i}^{\left (n \right)}\) is the free space path loss of user *j* from feed *i*, which can be expressed as [16]

$$ G_{j,i}^{\left(n \right)} = \frac{{{G_{R}}b_{j,i}^{\left(n \right)}}}{{4\pi \frac{{d_{j,i}^{\left(n \right)}}}{\lambda }\sqrt {{K_{B}}{T_{R}}{B_{W}}} }}, $$

(2)

where \(d_{j,i}^{\left (n \right)}\) is the distance between user *j* and feed *i*, *λ* is the wavelength, and *K*_{B}*T*_{R}*B*_{W} is the noise power, in which *K*_{B},*T*_{R}, and *B*_{W} are the Boltzmann constant, the noise temperature, and the user link bandwidth, respectively. \(G_{R}^{2}\) is the receiver antenna gain of the user, and \({\left ({b_{j,i}^{\left (n \right)}} \right)^{2}}\) is the multibeam antenna beam gain between user *j* and feed *i*. For the SPF antenna, the beam gain can be approximated by [25]

$$ {\left({b_{j,i}^{\left(n \right)}} \right)^{2}} = {b^{2}}\left({\theta_{j,i}^{\left(n \right)}} \right) = {G_{\max }}{\left({\frac{{{J_{1}}\left(u \right)}}{{2u}} + 36\frac{{{J_{3}}\left(u \right)}}{{{u^{3}}}}} \right)^{2}}, $$

(3)

where *J*_{1} and *J*_{3} are respectively the first-kind Bessel function of order 1 and 3. \(u = 2.07123~{\text {sin}}~\theta _{j,i}^{(n)} \big / {\text {sin}{\theta _{3dB}}}\), where *θ*_{3dB} is the 3 dB angle of each beam and \(\theta _{j,i}^{(n)}\) is the angle between the location of user *j* and the *i*th beam center.

The phase of the *i*th channel coefficient of user *j* is assumed as [8, 26]

$$ \varphi_{j,i}^{\left(n \right)} = \theta_{j}^{\left(n \right)} + {\delta_{i}}, $$

(4)

where \(\theta _{j}^{\left (n \right)} \sim U\left [ {0,2\pi } \right)\) is the same for the phases of all the channel coefficients of user *j*. It is the phase due to radio frequency (RF) signal propagation. Since the distance between a user and any feed is much longer than that between any two feeds, the phases caused by the RF signal propagation are almost the same for the same user. *δ*_{i}∼*N*(0,*σ*^{2}) is the phase caused by the payload oscillator of feed *i* [16]. *δ*_{i} is the same for the *i*th channel coefficient of each user in the coverage area.

The phases of the channel vectors have a great effect on the system performance [27], and the phase variation *δ*_{i} cannot be obtained by channel estimation [28]. Thus, we make a reasonable assumption that the CSI available for the scheduling, i.e., *available CSI*, is donated as

$$ {\hat{\mathbf{h}}}_{j}^{\left(n \right)} = \left({\hat h_{j,1}^{\left(n \right)},\hat h_{j,2}^{\left(n \right)}, \ldots,\hat h_{j,N}^{\left(n \right)}} \right) = {e^{j\theta_{j}^{\left(n \right)}}} \cdot \left({\begin{array}{*{20}{c}} {G_{j,1}^{\left(n \right)}}&{G_{j,2}^{\left(n \right)}}& \cdots &{G_{j,N}^{\left(n \right)}} \end{array}} \right), $$

(5)

which takes the imperfection estimation of phases into consideration.

### Signal model

In the frame transmission of multiplexed group *k*, the channel vector of user *q* in beam *n* is donated as \(\mathbf {h}_{q,{l_{k,n}}}^{\left (n \right)} \in {\mathbb {C}^{1 \times N}}\left ({q \in {\mathcal {U}}_{{l_{k,n}}}^{\left (n \right)},{l_{k,n}} \in {{\mathcal {K}}_{n}}} \right)\), and the received signal of the user is written as

$$ y_{q,{l_{k,n}}}^{\left(n \right)}{\mathrm{ = }}\mathbf{h}_{q,{l_{k,n}}}^{\left(n \right)}{\mathbf{P}_{k}}{\mathbf{s}_{k}} + n_{q,{l_{k,n}}}^{\left(n \right)}, $$

(6)

where \(n_{q,{l_{k,n}}}^{\left (n \right)} \sim CN\left ({0,{N_{0}}} \right)\) is the received noise and \({\mathbf {s}_{k}} \in {\mathbb {C}^{N \times 1}}\) is the transmitted signal from the *N* feeds to the corresponding beams satisfying that \(E\left [ {{{\left | {{s_{k,n}}} \right |}^{2}}} \right ] = 1,\forall n \in {\mathcal {N}}\). \({\mathbf {P}_{k}} \in {\mathbb {R}^{N \times N}}\) is the diagonal power factor matrix, and [*t**r**a**c**e*(**P**_{k}**P**_{k})]_{nn} is the transmitted power of feed *n*. Multicast precoding is adopted to reduce the interference between adjacent beams. For the signal *s*_{k,n} of beam *n*, with the precoding vector \({\mathbf {w}_{k,n}} \in {\mathbb {C}^{N}}\), the received signal can be expressed as

$$ y_{q,{l_{k,n}}}^{\left(n \right)}{\mathrm{ = }}\mathbf{h}_{q,{l_{k,n}}}^{\left(n \right)}{\mathbf{w}_{k,n}}{s_{k,n}} + \sum\limits_{p \in {\mathcal{N}}\backslash \left\{ n \right\}} {\mathbf{h}_{q,{l_{k,n}}}^{\left(n \right)}{\mathbf{w}_{k,p}}{s_{k,p}}} + n_{q,{l_{k,n}}}^{\left(n \right)}. $$

(7)

The actual signal sent by the feeds is \({\mathbf {x}_{k}} = {\sum \nolimits }_{n \in {\mathcal {K}}} {{s_{k,n}}{\mathbf {w}_{k,n}}} \). Assume that \(\left \{ {{s_{k,n}}} \right \}_{n = 1}^{N}\) are mutually uncorrelated during the transmission. The transmitted power of all the feeds is \({\sum \nolimits }_{n \in {\mathcal {N}}} {\mathbf {w}_{k,n}^{H}{\mathbf {w}_{k,n}}}\), and the transmitted power of feed *n* is \({\left ({{\sum \nolimits }_{n' \in {\mathcal {N}}} {{\mathbf {w}_{k,n'}}\mathbf {w}_{k,n'}^{H}}} \right)_{{nn}}}\).

The signal to interference plus noise ratio (SINR) of the user after precoding is

$$ SINR_{q,{l_{k,n}}}^{\left(n \right)}{\mathrm{ = }}\frac{{{{\left| {\mathbf{h}_{q,{l_{k,n}}}^{\left(n \right)}{\mathbf{w}_{k,n}}} \right|}^{2}}}}{{\sum\limits_{p \in {\mathcal{N}}\backslash \left\{ n \right\}} {{{\left| {\mathbf{h}_{q,{l_{k,n}}}^{\left(n \right)}{\mathbf{w}_{k,p}}} \right|}^{2}}} + {N_{0}}}}. $$

(8)

The actual data rate of the users in each frame group depends on the lowest SINR of the frame group members because of the multicast fashion. Thus, based on Shannon formulation, the spectral efficiency of frame group *l*_{k,n} in beam *n* can be expressed as

$$ C_{{l_{k,n}}}^{\left(n \right)} = {\log_{2}}\left({1 + \mathop {\min }\limits_{q \in {\mathcal{U}}_{{l_{k,n}}}^{\left(n \right)}} SINR_{q,{l_{k,n}}}^{\left(n \right)}} \right). $$

(9)

For multiplexed group *k* which consists of frame groups \(\left \{ {{\mathcal {U}}_{{l_{k,n}}}^{\left (n \right)},\forall n \in {\mathcal {N}}} \right \}\), the average spectral efficiency per beam can be expressed as

$$ {C_{k}} = \frac{{\sum\limits_{n \in {\mathcal{N}}} {C_{{l_{k,n}}}^{\left(n \right)}} }}{N}. $$

(10)

### Multicast precoding

The precoding design influences the system performance. For multicast transmission, the precoding problem is NP-hard for optimization objectives such as minimizing the transmitted power, maximizing the fairness among users [29, 30]. These algorithms are complex and involve iterative interior point methods during the calculation of precoding vectors. A low-complex precoding algorithm, which was a one-shot design, was proposed to improve the transmission efficiency by limiting the interference between adjacent beams in HTS systems [16]. The aforementioned algorithms, however, all have higher complexity than the average minimum mean squared error (MMSE) scheme, which was first proposed for multicast transmission in HTS systems [31]. Considering that this scheme can achieve satisfactory performance with low complexity, we adopt it as the multicast precoding scheme in this paper. As the available CSI shown in (5) has the same phase for each feed link, we leave out the phases of the channel coefficients. Thus, in multiplexed group *k*, the average available CSI of frame group *l*_{k,n} used for multicast precoding is

$$ {\hat{\mathbf{h}}}_{{l_{k,n}}}^{\left(n \right)} = \frac{{\sum\limits_{q \in {\mathcal{U}}_{{l_{k,n}}}^{\left(n \right)}} {\left| {{\hat{\mathbf{h}}}_{q,{l_{k,n}}}^{\left(n \right)}} \right|} }}{{\left| {{\mathcal{U}}_{{l_{k,n}}}^{\left(n \right)}} \right|}}. $$

(11)

The channel matrix used for precoding is \({\mathbf {H}_{k}} = {\left ({{\hat {\mathbf {h}}}{{{~}_{{l_{k,1}}}^{\left (1 \right)}}^{T}},{\hat {\mathbf {h}}}{{{~}_{{l_{k,2}}}^{\left (2 \right)}}^{T}}, \ldots,{\hat {\mathbf {h}}}{{{~}_{{l_{k,N}}}^{\left (N \right)}}^{T}}} \right)^{T}}\). For the sum transmitted power *P*_{T}, the precoding matrix **W**_{k}=(**w**_{k,1},**w**_{k,2},…,**w**_{k,N}) is

$$ {\mathbf{W}_{k}} = \gamma \mathbf{H}_{k}^{H}{\left({{\mathbf{H}_{k}}\mathbf{H}_{k}^{H} + \frac{N}{{{P_{T}}}}\mathbf{I}} \right)^{- 1}}, $$

(12)

where *γ* is the power factor, fulfilling \(trace\left ({\mathbf {W}_{k}{\mathbf {W}_{k}^{H}}} \right) \le {P_{T}}\) for the sum power constraint or \({\left [ {\mathbf {W}_{k}{\mathbf {W}_{k}^{H}}} \right ]_{nn}} \le {{P_{T}}} \bigg / {N},\forall n \in {\mathcal {N}}\) for per-antenna power constraints.