A two-hop relay system is considered where there are antennas at the source, antennas at the destination, and a single antenna at the relay, as shown in Figure 1. To make the relay as simple as possible, an AF relaying protocol is employed. It is also assumed that a direct communication link between source and destination is not available, as is reasonable in the case where the communication link between source and destination is in a deep fading state and/or the separation distance between them is large. In addition, half-duplex transmission is assumed; that is, the relay cannot transmit and receive simultaneously in the same time slot or frequency band.

OSTBC transmission containing symbols and block length of is utilized at the source to achieve space diversity. During the first time slot, the received vector signal at the relay can be written as

where denotes a OSTBC transmission matrix, denotes the Rayleigh fading complex channel gain vector from the source to the relay, and is the independent and identically distributed (i.i.d) complex Gaussian noise vector at the relay. During the second time slot, the received signal at the destination can be written as

where denotes the Rayleigh fading channel gain vector from the relay to the destination, is the i.i.d noise matrix at the destination where denotes the noise at the th receive antenna during the th symbol period, and denotes the signal vector sent by the relay where is a relay gain. As categorized in the literature, relay gains may be *fixed* or *variable*. In this paper, variable relay gain is considered. Relay gains can be further classified as *exact* or *ideal*. We denote as the exact relay gain, where and are average power constraints at the source and relay, respectively. If we ignore the noise at the relay, which is denoted as the ideal relay gain [5] and is amenable to mathematical manipulation. As in most of the literature, the ideal relay gain is used to compute exact average SERs of the proposed system in this paper. Later, it will be observed from simulations that the ideal relay gain provides a tight lower bound on outage probability and average SER in the case of medium-to-high SNR. Substituting into (2) leads to the received signal at the destination given by

Using maximum likelihood (ML) detection of OSTBCs for the case of spatially colored noise given in [11], the received SNR is obtained as follows.

Theorem 1.

Using the exact relay gain, the received SNR of two-hop AF OSTBC transmission is given by

When the ideal relay gain is utilized, the received SNR is given by

where , , , , and denotes the OSTBC code rate.

Proof.

It can be observed from (3) that the noise at the destination is temporally white but spatially colored; that is, the columns of noise matrix are independent and Gaussian with covariance matrix . Also, note that system (3) under consideration is equivalent to a conventional MIMO system with an effective channel gain matrix . We first whiten the colored noise and then employ the ML method given in [11, 12] to decode each symbol in the OSTBC. The noise-whitening process is given by

where , and . After ML detection, the MIMO system in (6) can be transformed into the following parallel and independent single-input and single-output (SISO) systems:

where and denotes the Frobenius norm of . Following similar arguments to [11], the received SNR for each symbol can be written as

where denotes trace of a matrix, is inverse of matrix , and denotes average power of a symbol where is assumed for OSTBCs. By substituting for and defined above, (8) can be written as

Using the Matrix Inversion Lemma [13], (9) can be written as

Using , (10) can be simplified to

Using the fact that and in OSTBCs, substituting the exact relay gain into (11) yields (4), and substituting the ideal relay gain into (11) yields (5).

Remark 2.

Using maximal ratio combining (MRC) at the destination, Theorem 1 can be generalized to the case of relay systems with a direct link by including a term corresponding to the SNR of the direct link that increases the SNR by , where is the Rayleigh fading complex channel gain matrix from source to destination and denotes the received noise variance during the source-to-destination time slot. It is assumed that relay-to-destination communication occurs in a separate second time slot, that is, half-duplex communication.