Fig. 12

Convergence performance of DSPSA: J(ϕ ⁽ⁿ⁾), the value of the objective function at the nth iteration of DSPSA (left); \(\phi _{1}^{(n)}(1,6,4)\), the estimations of the optimal threshold to queue 1 when b ₂=1, g ₁=6 and g ₂=4 (right). The channels are both Rayleigh fading with \(\overline {\gamma }_{1}=\overline {\gamma }_{2}=0\text {dB}\) and modeled by 8-state FSMCs. The system parameters are set as p ₁=0.1, p ₂=0.2, λ=0.05, τ=1, η=2, ξ _o =4 and β=0.97 in the first 500 iterations. Then p ₁ and p ₂ are changed to 0.8 and 0.9, respectively, in the second 500 iterations. The optimal value of queue threshold vector ϕ ^∗ is determined via (22) by using the optimal policy θ ^∗ found by DP