Table 1 Q-learning algorithm for the proposed scheme

while required packet exists

if time index > sensing window size

initialize the sensing results and obtained results

else

time index = time index +1

while time index < n _sensing n _reply

Determine sensing tool (with IA or without IA)

from the k out of N rule with N = n _sensing n _reply sensing results

end

Obtain P(D₀), P(D₁) from the sensing results

Calculate P(H₀), P(H₁) from the simultaneous equations of P(D₀),P(D₁)

Calculate R _I ,R _L from the conditional probabilities and determine the state

Select an action a _t based on the optimal policy from the current state s _t

Obtain the immediate payoff R from action a _t

Observe the next state s_t + 1 by R _I ,R _L

Update \( \mathcal{Q}\left({s}_t,{a}_t\right) \) based upon this experience as

\( \mathcal{Q}\left({s}_t,{a}_t\right)\leftarrow \left(1-\alpha \right)\mathcal{Q}\left({s}_t,{a}_t\right)+\alpha \left\{{r}_t+\gamma \underset{a_{t+1}}{\max}\mathcal{Q}\left({s}_{t+1},{a}_{t+1}\right)\right\} \)

end

end