Efficient authentication and re-authentication protocols for 4G/5G heterogeneous networks

Abstract

In the next-generation heterogeneous wireless networks, designing authentication protocols that meet the demand of mobile users/applications is a challenge. This paper proposes authentication and re-authentication protocols for 4G wireless networks, in particular, LTE-Advanced (LTE-A), WLAN, and WiMAX-Advanced (WiMAX-A) interworking architecture. The proposed protocols are applicable to 5G networks. With the consideration of the existing standard authentication protocols, a new set of authentication and re-authentication protocols has been reinvented to provide fast and secure handovers (HO) in the current 4G and the next 5G networks. The proposed authentication protocols can be invoked when the users perform a vertical HO (between different networks) for the first time, whereas the re-authentication protocols can be invoked when the users perform a horizontal HO (within the same network domain). These protocols provide an efficient method to protect user identity and reduce the burden on the authentication server (AS) during the sequential handovers. The results of the analytical model show that the proposed protocols achieve better performance than standard and other protocols. The reduction of handover cost, handover delay, and energy consumption in the proposed protocols reaches up to 22%, 44%, and 17%, respectively. In addition, the verification tools show that the proposed protocols are secure, dependable, and prevent all types of authentication and secrecy attacks.

1 Introduction

The 3GPP standards support the interworking between the advanced long-term evolution (LTE-A) networks and other wireless networks to provide better services in coverage, cost, and performance. These heterogeneous wireless networks will serve a huge number of users and applications that demand higher data rates, lower latency, and energy consumption. For the sake of providing seamless and fast handovers in the heterogeneous networks, the delay and cost caused by authentication protocols should be reduced. In addition, the authentication protocols should be secured against authentication attacks. Therefore, the authentication process has increasingly become more important, especially in the new 5G heterogeneous networks.

The basics of current 4G Authentication and Key Agreement (AKA) protocols will be utilized in the new 5G networks; thus, the 4G authentication protocols should be improved to meet the demand of this new technology. For instance, the users in the LTE-WLAN-WiMAX interworking architecture must be authenticated by the LTE Home Subscriber Server (HSS) in the home network, which adds delay and overhead on these servers each time the user connects or moves in the interworking architecture. It also makes this server a subject of single point of failure. From the performance aspect, the delay caused by the authentication process adds overhead to the seamless and fast handover process. The effect of delay could be more severe in the case of 5G application that are delay-sensitive applications. From the security point of view, the user identity disclosure attack can be launched in the first connection, when the International Mobile Subscriber Identities (IMSIs) are sent by users to HSS without protection in a clear text.

In WLAN, LTE, and WiMAX networks, the fast re-authentication protocols have been proposed by standards to reduce the authentication delay and cost of full authentication protocols such as Improved Extensible Authentication Protocol-AKA ^′ (EAP-AKA ^′), Evolved Packet System-AKA (EPS-AKA), and Initial Network Entry Authentication (INEA), respectively [1] and [2]. Despite of fast re-authentication protocol’s efficiency in reducing the authentication delay and cost, they still suffer from User Identity Disclosure (UID) attack, Lack of Perfect Forward Secrecy (PFS), and Man-In-The-Middle (MITM) attack. In addition, the fast re-authentication protocols are invoked regardless of the handover type (i.e., inter and intra), which inherit high delay; therefore, it is considered as insufficient solution for 5G networks.

In this paper, the standard AKA protocols that are used in LTE, WLAN, and WiMAX networks have been reinvented to present new re-authentication protocols for each network domain. The new re-authentication protocols enhance the security aspects and the performance in terms of delay, cost, and energy consumption. The proposed protocols are aimed to provide fast and secure different handover types, and these features allow the proposed protocols to effectively work in the 5G heterogeneous wireless networks.

In the case of applying the proposed protocols in the 5G networks, the names of the entities that are involved in the authentication process will be changed, for example, the functions of eNB will be handled by next-generation Evolved Node-B (ng-eNB), the functions of authentication server (AS) will be handled by Authentication Server Function (AUSF), and the function of storing long-term keys in HSS will be handled by the Unified Data Management (UDM).

The contributions of this paper can be summarized as follows:

• A new method is proposed to prevent UID attack and reduce the handover delay, cost, and the overhead on AS, which contributes significantly in reducing delay and cost during different handover types in heterogeneous networks.

• Three standard full authentication protocols, EAP-AKA ^′, INEA, and EPS-AKA protocols, are enhanced to provide full authentication process between the user and WLAN, WiMAX, and LTE networks, respectively, when the user connects to one of these networks for the first time.

• A set of new re-authentication protocols is proposed to be performed after the enhanced standard authentication protocols. These protocols provide fast inter and intra re-authentication processes in LTE-WiMAX-WLAN interworking architecture during inter and intra handovers, respectively. In addition, in the case of any failure occurs in the HSS server, the local servers have the ability to complete the authentication process, which results in avoiding the single point of failure.

• A new unified key hierarchy is proposed to be suitable for the module of the networks involved in this work.

The remainder of this paper is organized as follows: Section 3 presents an overview of the interworking architecture between LTE-WLAN-WiMAX networks; it also presents the standard full and fast authentication protocols. A brief of the related works is also presented in this section. Section 4 describes the proposed authentication and re-authentication protocols. Section 5 provides a security analysis of the proposed protocols while Section 6 evaluates the performance of the proposed protocols compared to the standard and other protocols. Section 7 concludes the paper.

2 Methods

In this paper, the proposed protocols and methods are presented in Section 4, where the standard AKA protocols that are used in LTE, WLAN, and WiMAX networks are enhanced to present new re-authentication protocols for each network domain. These enhancements make the proposed protocols applicable for 5G networks. To prevent the UID attack, a Kerberos-based method is proposed. A new unified key hierarchy is used to be applied in each network type in the interworking architecture. The new re-authentication protocols are locally performed during inter and intra handovers to provide secure and fast handovers. In Section 5, the security aspects of the proposed protocols are analyzed and verified using well-known verification tools. Section 6 presents the evaluation results and discussion of the proposed protocols, where a scenario of user movements and an analytical model is proposed for evaluation and comparisons of the proposed protocols with standard and other methods in terms of delay, cost, storage, and energy consumption.

3 Overview and related works

3.1 Heterogeneous wireless networks

Heterogeneity is one of the most features of the current 4G and the next 5G networks. This section presents an overview of the heterogeneous wireless networks such as LTE, WLAN, and WiMAX networks.

The security was not completely specified in the earlier versions of WLAN. It was specified in the IEEE 802.11i amendment. The key management and authentication are also included in this standard. The Remote Authentication Dial in User Service (RADIUS) protocol supports EAP-AKA authentication protocol [3].

The security of WiMAX has been specified by the IEEE 802.16 standard as a security sub-layer in the Medium Access Control layer. The INEA authentication protocol is a part of the Privacy Key Management (PKM). The PKM is a security protocol that has been adopted in the WiMAX security sub-layer to provide authorization, authentication, key exchange, and key distribution between base stations (BSs) and mobile stations (MSs) [4]. The last standard of WiMAX is the WiMAX-Advanced which has many features such as supporting mobile internet and MIMO [5].

The long-term evolution of the Universal Mobile Telephone System (UMTS) is one of the Third-Generation Partnership Projects (3GPPs) which was defined by 3GPP in November 2004. The recent projects are LTE and its enhanced version, which is the LTE-Advanced (LTE-A).

In LTE-A, a wider bandwidth is provided, and antenna technology is improved and used in both uplink and downlink directions. These technologies are called 4th Generation (4G) networks that rapidly spread over the world. This leads to more needs of higher bit rate and lesser delay to serve a large number of users.

The AKA protocol is continuously evaluated and developed by 3GPP. The development of this protocol has been started from 2G-AKA [6], 3G-AKA [7], or UMTS-AKA until 4G networks version EPS-AKA [8]. The development is ongoing to use AKA in 5G networks [9].

The future 5G networks will have extraordinary improvements in data rate, system capacity, energy consumption, and massive device connectivity. Mostly, the AKA security protocol that is used in the 4G networks will be used in the future 5G networks with some improvements.

The integration or interworking between the aforementioned different wireless networks is established via connecting those networks to the HSS server in LTE home network. The HSS server must authenticate any user connecting to the interworking architecture. Figure 1 shows a simplified LTE-WLAN-WiMAX interworking architecture where the 3GPP Authentication, Authorization, and Accounting server (3AAAS); Proxy AAA server (PAAAS); and Wireless AAA server (WAAAS) play a role of the bridge node between LTE, WiMAX, and WLAN domains. Those servers are connected via STa or Wa interfaces to perform authentication and re-authentication procedures.

Fig. 1

LTE-WLAN-WiMAX interworking

A simplified domain of LTE network includes the radio part, which is identified as the evolved UMTS Terrestrial Radio Access Network (eUTRAN). This part includes one or more Evolved Node-B (eNBs). The other part is the packet core, which is identified as the Evolved Packet Core (EPC). It consists of HSS, Mobility Management Entity (MME), and Serving Gateway (S-GW). The total system is identified as the EPS. This structure makes LTE simple, scalable to be interoperable with legacy networks such as UMTS and other wireless networks. In addition, it makes LTE efficient to widely use the internet protocols and applications [10].

A WLAN domain includes the WAAAS, which may control one or several Access Points (APs), whereas WiMAX network domain includes the Access Service Network (ASN), which is the access network of WiMAX. It contains one or several BSs and ASN Gateways (ASN-GWs). The ASN is an interface between the Connectivity Service Network (CSN) and MSs or the Subscriber Stations (SSs).

3.2 Standard Full EAP-AKA ^′ Authentication Protocol (AKAP)

The full authentication protocol is invoked whenever the user connects to a new network domain for the first time. Unlike previous studies, this work employs the improved EAP-AKA ^′ protocol rather than the EAP-AKA protocol [2]. This is due to its advantages in the security aspects. In the EAP-AKA ^′ protocol, the Access Network Identity (ANID) [11] is used to derive Integrity Key (IK) and Ciphering Key (CK), to be named as IK^′ and CK^′, respectively. The usage of ANID leads to specify the generated keys for a particular network that is provided in ANID; therefore, it ensures that the same ANID is only used by the parties involved in the authentication procedure. Moreover, it adds an additional protection against compromised node by limiting the attacker choices and allowing identifying the compromised networks. The additional keys are derived from IK^′ and CK^′ rather than from IK and CK. The manner of deriving Master Session Key (MSK), Extended MSK (EMSK), Transient Session Key (TSK), Authentication Key (K_auth), and Encryption Key (K_enc) is considerably different as compared to the manner of derivation in the EAP-AKA protocol. The EAP-AKA ^′ protocol uses SHA-256 (256-bit hash) [12], which is stronger and more popular than SHA-1 (160 bits hash) [13] that is used in EAP-AKA.

More details about this protocol can be found in our previous work in [14].

3.3 Standard Fast EAP-AKA ^′ Re-authentication Protocol (FAKAP)

The 3GPP has specified fast re-authentication to reduce authentication delay and communication overhead between the HSS and other nodes [15]. To achieve this, the full EAP-AKA ^′ credentials are not used in fast re-authentication, and HSS is not involved. The protocol mechanism is started by sending a user identity request message from the AP to the UE. The UE replies with a response message that contains a re-authentication ID, which was derived in the previous full EAP-AKA ^′. For more details about this protocol, refer to our previous work in [14].

3.4 Related works

Recent studies of authentication and re-authentication protocols in the heterogeneous networks [16, 17], and [18] are designed based on the EAP-AKA protocol [19]. However, they still suffer from UID and LNAS attacks. In literature, many studies such as [20] and [21] have been proposed to solve the limitations of EAP-AKA and improve the handover process. However, they are based on asymmetric key methods that might require additional processing capability in the UE. Moreover, other methods require modification in the architecture or adding new entities. For LTE networks, the enhanced protocol in [22] has solved the problem of UID attack, but it has been designed for non-trusted wireless networks. In addition, it has not been designed for handover process. The work in [4] can be used with minor modification to perform the authentication process with untrusted networks, since it has a tunnel phase that can protect the rest of the protocol procedure. However, it has not been designed for handover process. This work presents authentication and re-authentication protocols that overcome the aforementioned limitations. The details of the proposed protocols are in the next section. The work in [23] has proposed authentication and re-authentication protocols to reduce the delay and cost; however, handovers to LTE network have not been considered. In [24], another authentication method called EAP-CRA has been proposed for different wireless networks such as WLAN, WiMAX, and LTE networks. A single set of credentials is used with any network, which reduces the time of authentication and reduces the messages exchanged between entities. However, many channels between nodes are assumed to be secure; thus, this method could be vulnerable to many attacks when it is implemented. In addition, it needs modifying the network infrastructure, which makes it difficult to be implemented. In [25] and [26], efficient group-based authentication and re-authentication protocols for 5G networks and Wireless Mesh Networks (WMN) have been proposed. However, they are limited to LTE-WLAN interworking and WMN, respectively.

4 Proposed authentication protocols

The proposed protocols have many features that make them more secure and faster than others. These features makes them good substitute for the 4G and the next 5G generation. In the proposed protocols, the signaling between UE and the AS in the home network is reduced. This is achieved by employing the delegation concept in the re-authentication processes [27]. The AS performs a full authentication protocol with UE during the first connection. In the next authentication process, when the UE requires reconnection to the same network, the AS delegates the re-authentication and key distribution process to the local server in the serving network.

The HSS server maintains the database of users and other network entities such as APs/BSs/eNBs, GWs, and AAA servers. A part of this database is a table that contains the IDs of UEs, the corresponding keys, and the IMSI. This mechanism takes some concepts from the Kerberos method [28].

In the beginning of the authentication process, the UE is not required to send its IMSI or to perform encryption operations to protect its identity. Instead, the user sends its ID and ANID in a clear text. According to the received ID, the HSS retrieves the key and IMSI. ANID is used by HSS to derive the appropriate keys for the specified network. A unified key hierarchy for the LTE-WLAN-WiMAX architecture is proposed in the next section.

4.1 Unified key hierarchy

A unified key hierarchy is proposed to be applied in each network type in the interworking architecture. For instance, in WLAN networks, the MSK and EMSK are required to be sent to UE and AS; in WiMAX networks, the Pairwise Master Key (PMK) and Authorization Key (AK) are required to be sent to BS and derived in MS; in LTE networks, the K_eNBs key is required to be sent to eNB and derived in UE. In addition, the Master Key (MK) is corresponding to K_ASME, which is used in EPS-AKA. The K_ASME key is derived using IK, CK while MK is derived using IK^′, CK^′. The unified key hierarchy includes all keys that are required for each network type. Figure 2 illustrates the unified key hierarchy.

Fig. 2

The unified key hierarchy

Two-level keys are proposed, one for re-authentication, which its name ends with “r” and the other for handover, which its name ends with “h.” Separation between re-authentication and handover keys is useful to provide a higher level of control on different security values [23]. This concept is applied as part of the new key hierarchy that is proposed in this work with modification to be suitable for EAP-AKA ^′ protocol. The key hierarchy of EAP-AKA ^′ authentication protocol [1] is adapted to be applied on key derivation in LTE and WiMAX authentication protocols. The keys in the proposed protocols are named based on the type of network and handover. For example, in the name of key NLK_r, “N” denotes to WLAN network, “L” denotes that it is used locally, and “r” denotes that it is used for the re-authentication process.

The Message Authentication Code (MAC) messages are also named based on the network and the protocol type. For example, in the following name of MAC message ANMAC_WU, “A” denotes to intra handover, “N” denotes to WLAN networks, and “WU” meaning that the message originated in the WAAAS is intended to UE. In addition, the challenge messages are also named in the same way. For instance, in the challenge message “RX3PC,” “R” denotes to inter handover, “X” denotes to WiMAX networks, and “3P” meaning that the challenge message originated in the 3AAAS is intended to PAAAS.

Unlike the standard protocols, EMSK key is used to derive two-level handover keys for WLAN networks, handover WLAN Network level Key (NNK_h), and handover WLAN Local level Key (NLK_h). The same two-level keys are derived from EMSK in the case of WiMAX or LTE networks. The keys, MSK, K_auth, K_enc, K_re, and EMSK are derived from MK as in the standard key hierarchy. The re-authentication key K_re is modified to be re-authentication WLAN Network level Key NNK_r and re-authentication WLAN Local level Key NLK_r. The same two-level keys in the case of WiMAX or LTE networks. For space reasons, the keys K_auth and K_enc are named K_a and K_e, respectively, in the rest of this paper. In the following sections, the proposed set of authentication and re-authentication protocols is presented.

4.2 Protocols for handover to WLAN networks

This section presents the enhanced EAP-AKA ^′ protocol and the new re-authentication protocols such as the inter and intra WLAN re-authentication protocols.

4.2.1 Enhanced EAP-AKA ^′ Protocol (EAKAP)

The EAKAP protocol is invoked upon the following cases, if the UE connects to the WLAN for first time, the timer is expired, or if the number of re-authentications exceeds, the number of allowed re-authentication processes (n_R).

Additional keys are generated in this protocol to be used in the inter and intra WLAN re-authentication protocols. This section presents the modifications that have been applied to this protocol. These modifications aim to reduce the communication between UE and HSS and to improve the security aspects using an efficient key exchange method.

To prevent the UID attack, the IMSI is not sent by UE, instead a temporary ID, which is called user ID (UE_ID) is sent by the UE to the HSS. The HSS maintains a table that contains UE_ID for each UE, corresponding IMSI, the pre-shared key K, and an extra field to store nonces that are received from UE. In the first authentication procedure, when the HSS receives the message that contains the UE_ID and nonce of UE, it finds the corresponding K and IMSI. The key K is used to generate the Authentication Vector (AV). At the end of EAKAP protocol, a new re-authentication id is generated to be used in the next re-authentication process.

In addition, a new key hierarchy for the enhanced EAP-AKA ^′ protocol is proposed as part of the unified key hierarchy to handle the different types of HO protocols. Unlike EAP-AKA, a key for re-authentication K_re is derived in EAP-AKA ^′ from MK key. In the proposed key hierarchy, K_re is considered as a re-authentication key for network level. It is named WLAN Network level Key for re-authentication (NNK_r), and the handover keys are derived from the EMSK key. The details of key derivation are provided in the protocol steps. Figure 3 shows the enhanced EAP-AKA ^′ protocol. For space reasons, the only proposed mechanism is clarified in the following steps:

• The UE sends a user response identity message to the WAAAS. This message contains its ID UE_ID, ANID, and its nonce for the current authentication i ($\phantom {\dot {i}\!}{R}_{{U}_{{i}}}$), then, the WAAAS attaches its nonce ${R}_{{W}_{i}}\phantom {\dot {i}\!}$ and forwards the message to HSS through the 3AAAS.

  Fig. 3

  

  Enhanced EAP-AKA ^′

• The HSS uses the UE_ID to retrieve the IMSI and the keys of the UE and WAAAS.

• After that, HSS generates the AV and sends it to the 3AAAS, which generates the keys K_e, K_a, K_re, MSK, and EMSK, then it attaches those keys in a challenge message to WAAAS (N3WC) using Eq. (1). The challenge message indicates that the 3AAAS delegates and provides the WAAAS with the key materials to perform the rest of the authentication process. The main part of the message contains Random number (RAND), Authentication Token (AUTN), MAC, and a challenge from the 3AAAS to UE (N3UC) as follows:

  $$\begin{array}{*{20}l} \textit{N3UC} &= \lbrace \textit{R}_{\textit{W}_{i}},\textit{n}_{AR},\textit{n}_{R}\rbrace_{\textit{K}_{e}} \\ \textit{N3WC} &= \lbrace \textit{K}_{e},\textit{K}_{a},\textit{NNK}_{r},\textit{NNK}_{h},\textit{MC}_{R},\textit{n}_{AR} \rbrace_{\textit{K}_{\textit{3W}}}, \end{array} $$

  (1)

  where MC_R is the maximum value of the counter of re-authentication process (C_R), n_AR is the number of allowed intra re-authentication processes, and K_e, K_a, and NNK_r are derived from MK. Those keys are substrings of bits [0..127], [128..383], and [384..639], respectively. The key NNK_h is derived using Eq. (2). In this paper, NNK_r and NNK_h are the network level keys for re-authentication and handover, respectively.

  $$\begin{array}{*{20}l} {NNK}_{{h}} = {F} ({EMSK}, |{R}_{{W}_{i}}|{W}_{{ID}}|{MSM}|{}^{\prime\prime}{NNKh}^{\prime\prime},{256}), \end{array} $$

  (2)

  where F is a key derivation function [19], MSM is the UE address in MAC layer, “NNKh” is the key label, W_ID is the ID of WAAAS, and 256 is the key length in bits.

• The WAAAS forwards RAND, AUTN, MAC, and N3UC to UE and stores N3WC.

• When UE receives the message, it checks MAC. If the checking process is successful, it derives the required keys. After that, the UE computes the RES and MAC and sends them in the EAP response challenge message to WAAAS.

• Upon receiving the message, WAAAS checks MAC and compares the Expected Response (XRES) with the RES value that has been sent from UE. If the checking process is successful, it generates NLK_r using Eq. (3), then it sends it along with the successful message to AP.

  $$\begin{array}{*{20}l} {NLK}_{{r}} = {F} ({NNK}_{{r}}, {C}_{{AR}}|{AP}_{{ID}}|{MSM}|{}^{\prime\prime}{NLKr}^{\prime\prime}, {512}) \end{array} $$

  (3)

  where C_AR is the counter of intra re-authentication process and AP_ID is the ID of AP.

• AP stores the NLK_r key and forwards the message to UE, which starts to derive the NLK_r key.

• UE and WAAAS derive a WLAN Re-authentication Identity (NRID_i), which is used for the next re-authentication process. NRID_i is derived as follows:

  $$\begin{array}{*{20}l} {NRID}_{{i}} = {SH} ({NNK}_{{r}}, {NNK}_{{h}}|{R}_{{U}_{i}}), \end{array} $$

  (4)

  where SH is a secure hash function.

4.2.2 Inter WLAN Re-authentication Protocol (RNRP)

To reduce the overhead on the HSS server, the 3AAAS performs the RNRP protocol on behalf of the HSS server using the key materials that have been sent from HSS server during the previous EAKAP protocol. The RNRP is performed when the user needs an HO to a new WLAN network domain in the interworking environment. After executing the RNRP protocol in a WLAN domain, RNRP gives permission and prepares the key materials to UE and WAAAS for future Intra handover within this domain. Figure 4 and the following steps describe this protocol.

• The UE sends the NRID_i−1 to WAAAS. NRID_i−1 has been derived using Eq. (4) in the previous EAKAP protocol.

  Fig. 4

  

  Inter WLAN re-authentication protocol

• Upon receiving the message, WAAAS validates NRID_i−1 to check whether the UE has previously visited this domain or not. Then, it forwards NRID_i−1 along with its nonce $\phantom {\dot {i}\!}{R}_{{W}_{i}}$ to the 3AAAS. ${R}_{{W}_{i}}\phantom {\dot {i}\!}$ will be sent to UE, which will send it again to WAAAS in the future re-authentication process. Thus, the WAAAS can verify the UE.

• The 3AAAS computes its MAC, which is intended to UE RNMAC_3U and a challenge (RN3UC) to UE as shown in Eq. (5).

  $$\begin{array}{*{20}l} {{RNMAC}}_{{3U}}&= {SH} ({K}_{{a}}, {NRID}_{{i}}|{R}_{{U}_{{i-1}}}|{R}_{{W}_{i}}) \\ {{RN3UC}}&= \lbrace {NRID}_{{i}},{n}_{{AR}},{R}_{{W}_{i}}\rbrace_{{K}_{{e}}} \end{array} $$

  (5)

  Note that the value ${R}_{{U}_{{i-1}}}\phantom {\dot {i}\!}$ is the nonce of UE that has been sent by UE to the 3AAAS in the previous EAKAP protocol. When UE receives this nonce, it can verify the 3AAAS. Each time the C_R exceeds n_AR, the full authentication EAKAP is invoked.

• After that, it prepares a delegation message, which is intended to WAAAS containing MC_R, n_AR, NNK_r, and NNK_h. This message will be used in the future ANRP protocol, and it is encrypted by the shared key between WAAAS and 3AAAS K_3w using Eq. (6). Then, it sends it along with RNMAC_3U and RN3UC to WAAAS.

  $$ {{RN3WC}}= \lbrace {K}_{{e}},{K}_{{a}},{NNK}_{{r}}, {NNK}_{{h}}, {MC}_{{R}}, {n}_{{AR}}\rbrace_{{K}_{{3W}}} $$

  (6)

• The WAAAS stores RN3WC and forwards RNMAC_3U and RN3UC to UE via AP.

• When UE receives RNMAC_3U, it checks the values and computes its MAC, which is named as RNMAC_UW and a challenge to WAAAS RNUWC as shown in Eq. (7).

  $$\begin{array}{*{20}l} {{RNMAC}}_{{UW}}&= {SH} ({K}_{{a}}, {R}_{{W}_{{i-1}}}|{R}_{{U}_{i}}|{C}_{{R}}) \\ {{RNUWC}}&= \lbrace {R}_{{U}_{i}},{C}_{{R}}\rbrace_{{K}_{{e}}} \end{array} $$

  (7)

• Then, the UE sends RNMAC_UW along with RNUWC to WAAAS.

• When WAAAS receives the message, it verifies RNMAC_UW by checking the value of ${R}_{{W}_{{i-1}}}\phantom {\dot {i}\!}$ that has been sent from 3AAAS to UE in the previous EAKAP protocol. In case of successful verification, it derives NLK_r key using Eq. (8) and sends it in the EAP success message to AP.

  $$\begin{array}{*{20}l} {NLK}_{{h}} = {F} ({NNK}_{{h}}, {C}_{{AR}}|{AP}_{{ID}}|{MSM}|{}^{\prime\prime}{NLKh}^{\prime\prime},512) \end{array} $$

  (8)

• Finally, the AP stores the NLK_h key and forwards the message to UE, which starts to derive NLK_h that is used to protect messages between UE and AP.

4.2.3 Intra WLAN Re-authentication Protocol (ANRP)

Instead of using standard fast re-authentication protocol, the ANRP is performed when the user moves within the same WLAN domain or to another domain in the interworking environment that was previously visited. Figure 5 and the following steps describe this protocol.

• The UE sends the previous NRID_i−1 to WAAAS.

  Fig. 5

  

  Intra WLAN re-authentication protocol

• WAAAS receives NRID_i−1, then it checks the value of counter C_AR to ensure that it does not exceed the n_AR. If the checking is successful, it generates ANMAC_WU and a challenge ANWUC as shown in Eq. (9), then it sends it along in the challenge message to UE.

  $$\begin{array}{*{20}l} {ANMAC}_{{WU}} &= {SH}({K}_{{a}}, {R}_{{W}_{i}}| {C}_{{AR}}) \\ {{ANWUC}}&= \lbrace {C}_{{AR}},{R}_{{W}_{i}}\rbrace_{{K}_{{e}}} \end{array} $$

  (9)

• When UE receives the message from WAAAS, it verifies the challenge, and it matches the value of counter C_AR with the stored value. If it matches, it generates ANMAC_UW and a challenge ANUWC as shown Eq. (10), then it sends it along in the EAP response challenge message.

  $$\begin{array}{*{20}l} {ANMAC}_{{UW}} &= {SH} ({K}_{{a}}, {R}_{{W}_{i}}|{C}_{{AR}}|{C}_{{R}}) \\ {{ANUWC}}&= \lbrace {C}_{{AR}}, {C}_{{R}}, {R}_{{W}_{i}} \rbrace_{{K}_{{e}}} \end{array} $$

  (10)

• The WAAAS checks whether the received C_AR matches the C_AR sent in the previous message or not. If it matches, it verifies the received MAC. If the verification is successful, it increments the counter C_AR and derives NLK_h key and sends it with EAP success message to AP.

• Upon successful authentication, the counters C_AR and C_R in UE are increased by 1. The UE starts deriving the NLK_h key.

4.3 Protocols for handover to WiMAX networks

The INEA protocol has been specified by WiMAX Forum in [29] and [30] to provide mutual authentication between mobile station (MS) and 3AAAS in the 3G-WiMAX interworking architecture. This section presents the enhanced INEA protocol, the inter, and intra WiMAX re-authentication protocols.

4.3.1 Enhanced INEA Protocol (EINEAP)

The EINEAP is performed if the UE connects to the WiMAX for first time. It is also performed if the timer is expired or if the number of re-authentication processes exceeds the n_R. Otherwise, the re-authentication protocols of WiMAX are invoked. The INEA protocol starts when the user receives the request identity message from ASN-GW. The rest of the protocol steps are illustrated in Fig. 6 and summarized as follows:

Fig. 6

Enhanced INEA protocol

• The UE sends UE_ID, ${R}_{{U}_{{i}}}\phantom {\dot {i}\!}$, and ANID to ASN-GW via BS. When ASN-GW receives EAP response identity message from UE, it attaches its nonce R_Gi and forwards the message to HSS through PAAAS and 3AAAS.

• When the 3AAAS receives the message, it stores the nonces and forwards the other values to HSS in AV request message.

• The HSS uses the received IDs to retrieve the shared keys and IMSI of the user. It uses the user key K to generate the AV and sends it along with the retrieved keys to the 3AAAS.

• The 3AAAS generates the network re-authentication level key from the MK key. In this protocol, this key is called re-authentication WiMAX Network level Key (XNK_r). In addition, the 3AAAS generates the handover key XNK_h as illustrated in Eq. (11). It delegates the rest of the authentication to PAAAS by sending the derived keys to PAAAS in a challenge message, which is encrypted by the shared key between 3AAAS and PAAAS as shown in Eq. (12).

  $$ {XNK}_{{h}}= {F} ({EMSK}, {R}_{{G}_{i}}|{G}_{{ID}}|{MSM}|{}^{\prime\prime}{XNKh}^{\prime\prime},{256}) $$

  (11)

  where G_ID is the ID of GW.

  $$\begin{array}{*{20}l} {X3PC} = \lbrace {K}_{{3G}},{K}_{{e}},{K}_{{a}}, {n}_{{AR}}, {XNK}_{{r}},{XNK}_{{h}},{MC}_{{R}}\rbrace_{{K}_{{3P}}} \end{array} $$

  (12)

  It also generates a challenge message to UE as shown in Eq. (13).

  $$\begin{array}{*{20}l} {X3UC} = \lbrace {R}_{{G}_{i}}, {n}_{{AR}},{n}_{{R}}\rbrace_{{K}_{{e}}} \end{array} $$

  (13)

• The 3AAAS sends RAND, AUTN, MAC, X3UC, and X3PC to the PAAAS in the request AKA challenge message. The PAAAS then forwards RAND, AUTN, MAC, and X3UC challenge to UE and stores X3PC challenge.

• When UE receives the message, it verifies MAC. In the case of successful verification, it computes the RES and its MAC. Then, the UE sends those values in the response AKA challenge message to the PAAAS.

• When the PAAAS receives the message, it verifies the received values, then it generates WiMAX local re-authentication keys XLK_r and XLK_h using Eq. (14) and uses them with MC_R and n_AR to generate a challenge message that is intended to ASN-GW using Eq. (15). This challenge message is called XPGC, and it is encrypted using K_3G that is retrieved from X3PC, then it sends it to ASN-GW in the EAP success message. It sets the value of C_AR according to the received n_AR value.

  $$\begin{array}{*{20}l} {XLK}_{{r}}&= {F}({XNK}_{{r}}, {C}_{{AR}}|{BS}_{{ID}}|{MSM}|{}^{\prime\prime}{XLKr}^{\prime\prime},{512})\\ {XLK}_{{h}}&= {F}({XNK}_{{h}}, {C}_{{AR}}|{BS}_{{ID}}|{MSM}|{}^{\prime\prime}{XLKh}^{\prime\prime},{512}) \end{array} $$

  (14)

  where BS_ID is the ID of BS.

  $$\begin{array}{*{20}l} {XPGC} = \lbrace {XLK}_{{r}},{XLK}_{{h}},{MC}_{{R}},{n}_{{AR}},{K}_{{e}},{K}_{{a}}\rbrace_{{K}_{{3G}}} \end{array} $$

  (15)

• Then, the ASN-GW uses the received local level keys to compute PMK_r and AK_r using Eqs. (16) and (17), and it sets the counter C_AR according to the n_AR value.

  $$ {PMK}_{{r}}= {TF}({XLK}_{{r}}, 160), $$

  (16)

  where TF is the truncate function used in [31].

  $$ {AK}_{{r}}= {F} ({PMK}_{{r}}, {C}_{{AR}}|{BS}_{{ID}}|{MSM}|{}^{\prime\prime}{AKr}^{\prime\prime},{160}) $$

  (17)

• After that, the ASN-GW sends the AK_r key with the success message to BS, which forwards the message to UE and stores the AK_r key.

• The UE and ASN-GW derive a WiMAX Re-authentication Identity (XRID_i), which is used for the next re-authentication process as follows:

  $$\begin{array}{*{20}l} {XRID}_{{i}} = {SH} ({XNK}_{{r}}, {XNK}_{{h}}|{R}_{{U}_{i}}) \end{array} $$

  (18)

4.3.2 Inter WiMAX Re-authentication Protocol (RXRP)

The RXRP is performed when the user needs to perform an HO from WLAN or LTE domain to a BS in WiMAX domain for the first time. The RXRP protocol has the same mechanism of RNRP except that the challenges and keys generated are dedicated for WiMAX networks.

4.3.3 Intra ASN WiMAX Re-authentication Protocol (AXRP)

AXRP is performed when the user is moving from a BS to another one within the same domain or moving to a previously visited ASN-GW. The ASG-GW will authenticate the UE on behalf of the 3AAAS using the keys and values that have been received in the previous EINEAP. The AXRP protocol has the same mechanism of ANRP except that the challenges and keys generated are dedicated for WiMAX networks.

4.4 Protocols for handover to LTE networks

This section presents the enhanced EPS protocol and the inter and intra LTE re-authentication protocols.

4.4.1 Enhanced EPS Authentication Protocol (EEPSP)

In this work, the EPS-AKA protocol is enhanced to be used for trusted wireless networks. The networks involved in this work are considered as trusted networks, since WiMAX uses licensed radio spectrum and WLAN uses 802.1x−based authentication, which requires encryption and uses EAP-based authentication [32]. The EEPSP is designed to be appropriate for the subsequent types of handovers. It is invoked when the user connects to LTE for the first time. It is also invoked when the timer is expired or the number of re-authentication processes using the same key materials exceeds the n_R. The entities involved in this protocol are MME, 3AAAS, and HSS servers. As mentioned in the previous section, the unified key hierarchy is also adapted to be suitable for LTE networks. The protocol mechanism is illustrated in Fig. 7 and in the following steps:

Fig. 7

Enhanced EPS-AKA protocol

• The UE sends UE_ID, ${R}_{{U}_{{i}}}\phantom {\dot {i}\!}$, and ANID to HSS via MME.

• When MME receives the EAP response identity message from UE, it attaches its nonce ${R}_{{M}_{i}}\phantom {\dot {i}\!}$ and forwards the message to HSS.

• The HSS uses UE_ID to retrieve IMSI and the pre-shared key K. It uses K to generate the AV and sends it to the 3AAAS.

• Then, the 3AAAS generates MSK, EMSK, and K_eNB. The K_eNB key is derived from MK key using Eq. (20). It is considered as network level re-authentication key, and it is named LTE Network level Key for re-authentication (ENK_r) in this protocol. The 3AAAS also generates LTE Network level Key for handover (ENK_h) using Eq. (20).

• After that, the 3AAAS delegates the rest of the authentication process to MME by sending the derived keys to MME. The keys K_NASint and K_NASenc are derived from EMSK key [33] using Eq. (19). In this protocol, the ENK_r key and C_AR counter correspond to the NH key and NCC counter used in the standard protocol.

  $$ {K}_{{NASint}}|{K}_{{NASenc}}= {PRF}({EMSK}), $$

  (19)

  where PRF is a pseudo-random function. These keys are used to protect the data in the Non-Access Stratum (NAS) layer.

  $$\begin{array}{*{20}l} {ENK}_{{r}} &= {MK}[384.. 639] \\ {{ENK}_{{h}}}&= {F}({EMSK}, {R}_{{M}_{i}}|{M}_{{ID}}|{MSM}|{}^{\prime\prime}{ENKh}^{\prime\prime}, 256) \end{array} $$

  (20)

  In addition, the 3AAAS generates challenges to MME and UE using Eq. (21) as follows:

  $$\begin{array}{*{20}l} {E3UC} & = \lbrace {R}_{{M}_{i}}| {n}_{{AR}}|{n}_{{R}}\rbrace_{{K}_{{e}}} \\ {EEMC} & = \lbrace {K}_{{eNB}}|{ENK}_{{h}}|{MC}_{{R}}|{n}_{{AR}}\rbrace_{{K}_{{EM}}} \end{array} $$

  (21)

• The 3AAAS also extracts RAND and AUTN from the received AV and sends them along with E3UC and EEMC to MME in the request AKA challenge message. The MME then, forwards RAND, AUTN, MAC, and E3UC challenge to UE and stores EEMC challenge.

• The UE verifies MAC. In the case of successful verification, it computes the RES and its MAC.

• Then, the UE sends the computed values in the response AKA challenge message to MME.

• When the MME receives the message, it verifies the received MAC and RES. In case of successful verification, it generates LTE Local level keys for re-authentication and handover ELK_r using Eq. (22), then it sends ELK_r along with the successful message to eNB.

  $$\begin{array}{*{20}l} {ELK}_{{r}} &= {F}({ENK}_{{r}}, {C}_{{AR}}|{eNB}_{{ID}}|{MSM}|{}^{\prime\prime}{ELKr}^{\prime\prime}, {512}) \end{array} $$

  (22)

• When eNB receives the message, it stores the ELK_r key and forwards the message to UE, which starts to derive the keys.

• Finally, the UE and MME derive LTE re-authentication identity (ERID_i), which is used for the next re-authentication process as follows:

  $$\begin{array}{*{20}l} {ERID}_{{i}} = {SH} ({ENK}_{{r}}, {ENK}_{{h}}|{R}_{{U}_{i}}) \end{array} $$

  (23)

4.4.2 Inter LTE Re-authentication Protocol (RERP)

The RERP is performed when the user moves to another MME within the same domain. It is also performed when the user performs an HO from WiMAX or WLAN networks to LTE network. The mutual authentication between the 3AAAS and UE is done without the need to communicate with HSS. In this protocol, the 3AAAS delegates the MME to perform the future re-authentication process. The RERP protocol has the same mechanism of ANRP except that the challenges and keys generated are dedicated for LTE networks.

4.4.3 Intra LTE Re-authentication Protocol (AERP)

The AERP is invoked when the user is roaming from eNB1 to eNB2 within the same LTE domain or performing an HO from other networks to a previously visited LTE domain. In AERP, the mutual authentication is done between UE and MME without the need to communicate with 3AAAS or HSS servers. MME uses the key materials that have been received when RERP was invoked. The AERP protocol has the same mechanism of ANRP except that the challenges and keys generated are dedicated for LTE networks.

5 Security analysis

5.1 Security features and robustness

In this section, the security properties of the proposed protocols are analyzed to demonstrate that the proposed protocols can satisfy the security requirements [34] and [35].

5.1.1 Mutual authentication

Full/fast EAP-AKA, INEA, and EPS-AKA authentication/re-authentication protocols achieve mutual authentication between UE and the 3AAAS server. The proposed authentication protocols provide a secure mutual authentication to prevent several attacks such as MITM, impersonation, and rogue AP/BS attacks. In the local re-authentication protocols, the 3AAAS server delegates the authentication operation to the local servers WAAA, PAAA, and MME. The local servers and UE mutually authenticate each other by checking their MACs and proving that they have the correct nonces and counters. The legitimacy of local servers WAAA, PAAA, and MME is verified by checking their counters C_R and C_AR. The UE matches the received counters with the counters stored in its database. Successful matching indicates that the keys K_e and K_a that are used to encrypt counters and MAC are valid. In the same way, the local servers authenticate UE by checking the counters and MACs, since only the legitimate UEs can generate the keys K_e and K_a.

5.1.2 Protection of message integrity

MAC is appended with the authentication challenges/response messages that are exchanged between UE and AS in the interworking environment. It protects the integrity of those messages. In addition, it provides authentication for the sender. The sender of MAC attaches the previous nonce of the receiver in its MAC, then the receiver can authenticate the sender by checking that nonce. For example, in RERP protocol, UE receives the nonce of MME during the previous authentication process, which was called $\phantom {\dot {i}\!}{R}_{{M}_{{i}}}$. In the current authentication process, UE includes that nonce of MME, which is called $\phantom {\dot {i}\!}{R}_{{M}_{{i-1}}}$ in its MAC (REMAC_UM); thus, MME authenticates UE by checking this nonce.

5.1.3 Identity protection

Concealing the UE’s identity helps to prevent UID attack. Most recent solutions use temporary identities to conceal the IMSI; however, IMSI is still sent in the full authentication process, which makes them vulnerable to UID attack. In the proposed protocols, the IMSI is not sent, instead, a temporary user identity UE_ID is sent in the full authentication process and re-authentication identity RID is computed and sent in the next re-authentication process. When the HSS receives UE_ID, it uses it as a pointer to retrieve the IMSI and the pre-shared key. In the full authentication and the subsequent re-authentication processes, UE and local servers must derive new ID using Eqs. (4), (18), and (23), since those IDs are only used for a single authentication process.

In the standard full authentication protocols, the 3AAAS server must generate a re-authentication ID that will be used for future fast re-authentication process. Whereas in the proposed protocols, the 3AAAS server is exonerated from computing and saving the re-authentication IDs. In addition, the re-authentication IDs are not encrypted in the proposed protocols, instead, those IDs are sent in clear text since they will not be used in the future re-authentication process.

5.1.4 Forward and backward secrecy

In the proposed protocols, the shared keys are desired to be different in each communication session to achieve one of the security requirements, which is the key freshness. After each handover process, fresh keys are computed using fresh nonces. This property also helps in limiting the number of cipher texts that can be used by some attackers. In addition, the proposed protocols provide a perfect forward and backward secrecy and prevent domino effect attacks [36]. The UE and AP/BS/eNB compute and share fresh keys K_i, which can only be used in the ith authentication process. Those keys cannot be used to decrypt messages in the previous authentication process ith-1, which provide backward secrecy. In addition, any messages exchanged in the ith+1 authentication process cannot be decrypted by those keys, which provide forward secrecy. The UE and local servers derive network level keys NNK, XNK, and ENK. Those keys are computed using fresh nonces R_w, R_G, and R_M in Eq. (2), (11), and (20), respectively. This guarantees the freshness of all keys that will be computed from NNK, XNK, and ENK keys. In the intra re-authentication protocols, local keys NLK, XLK, and ELK are derived from NNK, XNK, and ENK, respectively. These local keys are different from the local keys in the previous intra re-authentication process. They are fresh as well since they are derived using new counter C_AR.

5.2 Verifying the proposed protocols

The proposed protocols are validated using one of the well known analytical tools for checking the secrecy of authentication protocols, the Automated Validation of Internet Security Protocols and Applications (AVISPA) tools [37, 38]. The AVISPA tools consider all types of attacks that target the network security protocols. It uses the High Level Protocol Specification Language (HLPSL) to allow specifying security protocols to find possible attacks. The protocol’s behavior is analyzed and certain goals are checked using several back-ends. Generally, there are several back-ends defined by AVISPA, and they can be freely chosen to execute the HLPSL code after it is translated by HLPSL2IF into an Intermediate Format (IF).

In this work, each description of the proposed protocols is written using HLPLS language. Then, the On-the-fly Model-Checker (OFMC) and Constraint-Logic based Attack Searcher (CL-ATSE) back-ends are used to verify those protocols. This is mainly due to their interesting features such as supporting various security protocols, checking whether the verified protocol is able to provide strong authentication, and secrecy. The most important feature is proving the lack of security in the protocol rather than proving its security. The complete protocol description contains several parts. First, the function role A, which describes the behavior of the party A during the protocol session. The function role session is composed of each party’s role. The function role environment describes the protocol execution under attack. Finally, the authentication requirements are defined in the part of goals.

All proposed authentication and re-authentication protocols are separately coded and verified by AVSPA tools. As an example, the code, simulation, goals, and results of RNRP protocol are presented in this section. The role of the UE and the goals that need to be verified are excerpted from the complete code and illustrated in Fig. 8. The goals that need to be verified are illustrated in Fig. 9. The run simulation of RNRP protocol is shown in Fig. 10. In this work, both authentication and secrecy goals are verified. The authentication goal is checked using the command authentication_on rw1. This indicates that when the UE sends its fresh nonce rw1 to 3AAAS, it requires that the UE and the 3AAAS should agree on that nonce and exist in the current state. The secrecy goal is checked using the command secrecy_of nnk1, which indicates that the nonce nnk1 should be secreted and the intruder cannot learn such value. If the protocol is insecure or vulnerable to any authentication or secrecy attack, the result indicates that it is unsafe; otherwise, it is safe. Figure 11 shows the result of running the RNRP protocol using OFMC and ATSE tools, it shows that the protocol is safe.

Fig. 8

The UE rule in RNRP protocol

Fig. 9

The goals in RNRP protocol

Fig. 10

The simulation of RNRP protocol

Fig. 11

Results of ATSE and OFMCE for RNRP protocol

6 Performance evaluation results and discussion

In this section, the performance of the proposed protocols is evaluated and analyzed in terms of handover delay, key size, communication overhead, average handover cost, and energy consumption. These terms are important factors during handover process in the 4G and the next 5G networks. To do that, four different algorithms for the user movements in the heterogeneous network environment are considered. Algorithm 1 (A1) represents the standard authentication protocols EAP-AKA ^′, INEA, and EPS-AKA [2]. Those protocols are invoked whenever the UE connects to a new network domain. Algorithm 2 (A2) represents the use of fast re-authentication protocols [15], which are invoked whenever the user connects to a previously visited network domain. In addition, algorithm 3 (A3) represents Coordinated Robust Authentication and re-authentication protocols (CRA) [24]. Whereas the proposed authentication and re-authentication protocols are invoked in algorithm 4 (A4) of the UE movements (handovers) model.

The UE moves in a fixed path as illustrated in Fig. 12, it initially establishes a connection to the LTE network and then moves to WLAN1 domain that controlled by WAAAS1. Next, the UE performs a vertical HO to WiMAX network. After that, the UE reconnects to the previously visited WLAN1 domain. Then, it moves to the previously visited LTE domain. In the next movement, the UE reconnects to the previously visited WiMAX domain. Finally, the UE moves to WLAN2 domain that controlled by WAAAS2.

Fig. 12

User movements

In A1, seven standard authentication protocols are invoked to perform the authentication process during UE handovers. Four standard authentication protocols and three fast re-authentication protocols are invoked in A2. Four full coordinated robust authentication (FCRA) protocols and three coordinated robust re-authentication (CRR) protocols are invoked in A3. In A4, the UE establishes a connection to the LTE domain and performs either EAKAP or EEPSP authentication protocol via eNB1. When the UE moves to WLAN1, an Inter WLAN re-authentication protocol is invoked with the AP1 residing in WLAN1. Next, the UE moves to WiMAX domain and performs an Inter WiMAX re-authentication protocol. The next movement is to the previously visited WLAN1 domain, where an Intra WLAN re-authentication protocol is invoked. After that, the UE moves to the previously visited LTE domain and performs an Intra LTE re-authentication protocol. Subsequently, the UE moves to the previously visited WiMAX domain and performs an Intra WiMAX re-authentication protocol. Finally, the UE moves to WLAN2 and performs an Inter WLAN re-authentication protocol.

6.1 Handover delay

This section provides an analytical model for LTE-WLAN-WiMAX interworking to evaluate the proposed and other algorithms in terms of handover delay where the user is performing a sequential vertical and horizontal handovers between those networks. In this model, the network of Fig. 1 is modeled in Fig. 13. The variable TD_BG is corresponding to TD_eG in LTE network; TD_PB is corresponding to TD_eM and TD_WA in LTE and WLAN networks, respectively; TD_PG is corresponding to TD_MG; and TD_PL is corresponding to TD_ML and TD_WL. The random variables of WiMAX network TD_BU, TD_BG, TD_PG, and the corresponding variables in LTE and WLAN are exponentially distributed with mean 1/TD_BU, 1/TD_BG, and 1/TD_PG. The variable TD_PL and other corresponding variables are Erlang distributed Er(H, TD_PL).

Fig. 13

Handover model

The four algorithms that will be studied in this model are represented by A, where A includes A1, A2, A3, and A4 algorithms. The handover delay for each method in A is modeled using the parameter HD^A. Equation (24) illustrates the probability distribution function (PDF) for HD^A.

$$ {f}_{{HD}^{A}}(t) = \sum_{m \in M^{A}} P_{m}.f_{{{HD}}_{m}^{A}}(t) $$

(24)

In this model, m is the authentication or re-authentication protocol that can be invoked in each algorithm, P_m is the ratio of invoking the method m, and M^A represents the protocols in each algorithm. For example, if A represents A1, then M^A = {AKAP, INEAP, EPSP } and M^A = {AKAP, FAKAP, INEAP, FINEAP, EPSP, FEPSP } when A represents A2, where FEPSP and FINEAP are the fast EPS and fast INEAP protocols, respectively. In the case of A represents A3, M^A = {FCRA, CRR }, while M^A = {EAKAP, EINEAP, EEPSP, RNRP, ANRP, RXRP, AXRP, RERP, AERP } for the proposed algorithm A4. According to [39], Eq. (24) can be expressed using Laplace transform in Eq. (25) as follows:

$$ {f}_{{HD}_{m}^{A}}^{*}(s) = \sum_{m \in M^{A}} P_{m}.{f}_{{HD}_{m}^{A}}^{*}(s) $$

(25)

The HD^A contains different delay values; thus, it can be represented in a generic form using the convolution operator ⊗ as expressed in Eq. (26).

$$ {f}_{{HD}_{m}^{A}}(t)= f_{\sum_{i \in \Phi^{A}_{m}} {HD}_{i}(t)} = \left(\otimes_{i \in \Phi^{A}_{m}} f_{{HD}_{i}} \right) $$

(26)

Under the condition m, the $\Phi ^{A}_{m}$ is a set that represents the components of delay in algorithm A. The Laplace transform for HD^A can be written as follows:

$$ {f}_{{HD}_{m}^{A}}^{*}(s)= f_{\sum_{i \in \Phi^{A}_{m}} {HD}_{i}(s)}^{*} = \left(\Pi_{i \in \Phi^{A}_{m}} f_{{HD}_{i}}^{*} \right) $$

(27)

Authentication delay is one of the delay components HD_i, and the other component is the transmission time. Equation (28) demonstrates the methods (protocols) that can be invoked during user movements in algorithms A1, A2, A3, and A4.

$$\begin{array}{*{20}l} {A1} &= 2.{EPSP} {+} 3.{AKAP} {+} 2.{INEAP} \\ {A2} &={EPSP} {+} {FEPSP} {+} 2.{AKAP} + {FAKAP}{+}{INEAP}{+}{FINEAP} \\ {A3} &= 4.{FCRA} {+} 3.{CRR} \\ {A4} &= {EAKAP} {+} 2.{RNRP} {+} {RXRP} {+} {ANRP} {+}{AERP} {+} {AXRP} \end{array} $$

(28)

When A represents A1, the Laplace transform of HD^A1 can be written as follows:

$$\begin{array}{*{20}l} {f}_{{HD}^{{A1}}}^{*}(s) =2.{f}_{{HD}_{{EPSP}}^{{A1}}}^{*}(s) +3.{f}_{{HD}_{{AKAP}}^{{A1}}}^{*}(s) +2.{f}_{{HD}_{{INEAP}}^{{A1}}}^{*}(s) \end{array} $$

(29)

The components of Eq. (29) are computed as follows:

$$\begin{array}{*{20}l} {f}_{{HD}_{{EPSP}}^{{A1}}}^{*}(s)&=(P_{{FP}}. (f_{{TD}_{{H3}}}^{*}(s))^{{2}} (f_{{TD}_{{eU}}}^{*}(s))^{{10}}. (f_{{TD}_{{eM}}}^{*}(s))^{{4}} \\ &.(f_{{TD}_{{M3}}}^{*}(s))^{{2H}}.(f_{{FP}}^{*}(s))) \end{array} $$

(30)

$$\begin{array}{*{20}l} {f}_{{HD}_{{AKAP}}^{{A1}}}^{*}(s)&=(P_{{FP}}. (f_{{TD}_{{H3}}}^{*}(s))^{{2}} (f_{{TD}_{{AU}}}^{*}(s))^{{10}}. (f_{{TD}_{{WA}}}^{*}(s))^{{4}} \\ &.(f_{{TD}_{{W3}}}^{*}(s))^{{4H}}.(f_{{FP}}^{*}(s))) \end{array} $$

(31)

$$\begin{array}{*{20}l} {f}_{{HD}_{{INEAP}}^{{A1}}}^{*}(s)&=(P_{{FP}}. (f_{{TD}_{{H3}}}^{*}(s))^{{2}} (f_{{TD}_{{BU}}}^{*}(s))^{{10}}. (f_{{TD}_{{BG}}}^{*}(s))^{{5}} \\ &. (f_{{TD}_{{PG}}}^{*}(s))^{{4}}.(f_{{TD}_{{P3}}}^{*}(s))^{{4H}}.(f_{{FP}}^{*}(s))) \end{array} $$

(32)

Accordingly, Eq. (29) can be written as follows:

$$\begin{array}{*{20}l} {f}_{{HD}^{{A1}}}^{*}(s)& = (f_{{TD}_{{H3}}}^{*}(s))^{{6}}({2}. (P_{{FP}}. (f_{{TD}_{{eU}}}^{*}(s))^{{10}}. (f_{{TD}_{{eM}}}^{*}(s))^{{4}} \\&.(f_{{TD}_{{M3}}}^{*}(s))^{{2H}}.(f_{{FP}}^{*}(s))) {+} {3}.(P_{{FP}}. (f_{{TD}_{{AU}}}^{*}(s))^{{10}} \\ &.(f_{{TD}_{{WA}}}^{*}(s))^{4}.(f_{{TD}_{{W3}}}^{*}(s))^{{4H}}.(f_{{FP}}^{*}(s))) {+} \\ & {2}.(P_{{FP}}. (f_{{TD}_{{BU}}}^{*}(s))^{{10}}. (f_{{TD}_{{BG}}}^{*}(s))^{{5}}. (f_{{TD}_{{PG}}}^{*}(s))^{{4}} \\&. (f_{{TD}_{{P3}}}^{*}(s))^{{4H}}.(f_{{FP}}^{*}(s)))) \end{array} $$

(33)

According to [40], the Laplace transform notation for HD^A can be obtained from the following equation.

$$\begin{array}{*{20}l} E({HD}^{A}) &=\left. \int_{0}^{\infty} f_{{HD}^{A}} (t) dt = \int_{0}^{\infty} f_{{HD}^{A}} (t) e^{-ts} dt \right\lvert \end{array} $$

(34)

Then, the mean of the density function is:

$$\begin{array}{*{20}l} = -\left.\frac{d}{ds} f_{\textit{HD}^{A}}^{*}(s) \right\rvert_{s=0} \end{array} $$

(35)

In the case of the standard protocols Algorithm A1, Eq. (35) can be written as:

$$\begin{array}{*{20}l} E({HD}^{{A1}}) = -\left.\frac{d}{ds} f_{{HD}^{{A1}}}^{*}(s) \right\rvert_{s=0} \end{array} $$

(36)

According to [30] and [39], Pr represents the values of P_FP and P_RP, and it varies between 0 and 1. The processing time of full authentication protocols (FP) and fast re-authentication protocols (RP) are 1240 and 600 ms, respectively. The hop count between GW and the AS is set to 4.

Figure 14 shows the performance of the proposed and other algorithms in terms of handover delay when the value of Pr increases from 0 to 1. It demonstrates that, when Pr is 0.1, the proposed and other algorithms experience convergent values of handover delay. This is because of performing full authentication protocols in the proposed and other algorithms during the first connection. When Pr increases the handover delay in A1, A2, A3, and A4 algorithms is gradually increased until it reaches to 10560 ms, 8640 ms, 8340 ms, and 5900 ms, respectively.

Fig. 14

Handover delay with Pr

Figure 15 shows how is the handover delay in each algorithm affected by increasing the hop count between the local servers and the 3AAAS. The value of handover delay in A4 is slightly affected by increasing the hop count compared to other algorithms. This is due to performing the intra authentication protocols which are not affected by the number of hop count since there is no communication between the local servers and 3AAAS. The local re-authentication protocols ANRP, AXRP, and AERP are not affected when the number of hop count between the WAAAS / PAAAS / MME and the 3AAAS is increased. The slight increasing in handover delay with H is due to invoking RNRP, RXRP, and RERP protocols, which requires contacting the 3AAAS. Figure 16 shows the performance of the proposed algorithm compared to the standard and other algorithms in terms of handover delay and the number of users. The handover delay increases by increasing the number of users in each algorithm. However, the proposed algorithm A4 achieves less handover delay compared to A1, A2, and A3 algorithms. In general, the proposed algorithm A4 reduces the handover delay by 44%, 34%, and 29% compared to A1, A2, and A3 algorithms, respectively.

Fig. 15

Handover delay with hop count

Fig. 16

Handover delay with no. of users

A part of user movements is used to compare the proposed protocols with standard protocols and the WiMAX-WLAN authentication protocols of Shaidhani in [23]. In this part, authentication and re-authentication protocols are invoked during handovers from WiMAX to WLAN networks. Figure 17 shows the performance of the proposed protocols compared to standard and other protocols when the hop count increases from 1 to 11. Obviously, it is shown that the handover delay is not affected significantly by increasing the hop count number. This is due to the reduction of communication between the serving network and the AS during the handover processes in the proposed protocols. In Fig. 18, the handover delay for all protocols are almost the same when Pr is 0. This is because at this point, the re-authentication protocols are not invoked and all methods are starting with full authentication protocols; thus, the difference is starting to increase gradually until it reaches to 34%.

Fig. 17

Handover delay with hop count (WiMAX-WLAN)

Fig. 18

Handover delay with Pr (WiMAX-WLAN)

6.2 Key size

This section presents the key size for all keys that are generated during performing a particular protocol. The size of the exchanged and generated keys has a significant impact on the storage cost because the other factors such as IDs, and nonce values used here have the same impact compared to the other methods. The sum of key size is calculated for A1, A2, A3, and A4 algorithms. Figure 19 gives a clear indication that algorithm A4 offers lesser key size compared to A1, A2, and A3 algorithms. In the first movement M1 of the user, the modified authentication protocols are invoked and the key size is larger than the other protocols. However, it is significantly reduced during the subsequent re-authentication processes.

Fig. 19

The key size during user movements

During the movements M4, M5, and M6, the user moves to previously visited wireless domains; thus, no additional keys are generated in the 3GPP AAA server and intra re-authentication protocols are invoked with key size (1024 bits). Whereas in the movement M7, the value of key size jumps to (2304 bits), since the user moves to a new WLAN domain, which requires additional keys are generated in the 3AAAS server to perform an inter re-authentication protocol. In general, the storage that is required to save the generated keys during the user movements in the interworking architecture is reduced using the proposed algorithm A4.

6.3 Communication overhead for authentication process

The communication overhead is the time required to perform the authentication process during the HO process. In this section, we compare the communication overhead in the proposed protocols with others as illustrated in Table 1, where the TD_UA is the transmission delay between UE and AP/BS, TD_AG is the transmission delay between the AP/BS and the GW, and TD_GP is the transmission delay between the GW and the AAA server.

Table 1 Communication overhead

6.4 Average handover cost

To evaluate the performance of the proposed protocols in terms of handover cost, the hexagonal wireless network model [41] is adapted as a network model and fluid flow (FF) model [42] as mobility model. It is assumed that the sizes of each subnet are equal and take a hexagonal shape. It is also assumed that a hexagonal network model is an LTE-WLAN-WiMAX interworking domain and a cell is a subnet of one of the networks in the interworking architecture. The average handover rate (λ_j) is given by Eq. (37).

$$ \lambda_{{j}} = ({v}~.~{S}({i})) / (\pi~.~{G}({i})) $$

(37)

where j is a user group indicator and v is the average velocity of UE in the interworking environment. The perimeter S(i) of the given network domain can be computed as follows:

$$ {S}({i}) = (12{i} + 6). {R} $$

(38)

where i is the number of cells and R is the subnet radius. The coverage area G(i) can be computed as follows:

$$ {G}({i}) = (2.6. {R}^{{2}}). (3{i}. ({i} + 1) + 1) $$

(39)

The average handover cost in time unit can be represented by,

$$ {AHC}_{{m}} = \lambda_{{j}}~.~{C}_{{m}}, $$

(40)

The cost of each method C_m is expressed as follows:

$$ {C}_{{m}} = {C}_{{m},{s}} + {C}_{{m},{p}}, $$

(41)

C_m,s is the signaling cost and C_m,p is the processing cost of the method m. The AC_m,s for each algorithm can be computed as follows:

$$\begin{array}{*{20}l} {A1}{C}_{{m,s}} &= 35 {C}_{{ws}} + 50 {C}_{{wd}} + 24{H} \\ {A2}{C}_{{m,s}} &= 35 {C}_{{ws}} + 44 {C}_{{wd}}+ 22{H} \\ {A3}{C}_{{m,s}} &=35 {C}_{{ws}} + 40 {C}_{{wd}}+ 14{H} \\ {A4}{C}_{{m,s}} &=35 {C}_{{ws}} + 35 {C}_{{wd}}+ 8{H} \end{array} $$

(42)

where C_ws and C_wd are the transmission cost on wireless and wired links, respectively. The processing cost for each method C_m,p is composed of the processing cost of each node C_n,p. For instance, the C_m,p for the EPSP protocol can be written as C_EPSP,p=C_UE,p+C_MME,p, where

$$\begin{array}{*{20}l} {C}_{{UE,p}} & = {C}_{{key}} + {C}_{{enc}} + {C}_{{dec}} + {C}_{{ver}} + {C}_{{hash}} \\ {C}_{{MME,p}} & = {C}_{{key}} + {C}_{{dec}} + {C}_{{ver}}, {P}_{{3}} = {C}_{{key}} + {C}_{{hash}} + {C}_{{enc}} \end{array} $$

(43)

where C_key, C_enc, C_dec, C_ver, and C_hash are the costs of key generation, encryption, decryption, verification, and hash function, respectively. Thus, the C_m,p for A1, A2, A3, and A4 algorithms is given as follows:

$$\begin{array}{*{20}l} {A1} {C}_{{m,p}} = & 23 {C}_{{key}}+14 {C}_{{enc}}+14 {C}_{{dec}}+14 {C}_{{ver}}+ 14 {C}_{{hash}} \\ {A2} {C}_{{m,p}} = & 20 {C}_{{key}}+14 {C}_{{enc}}+14 {C}_{{dec}}+14 {C}_{{ver}}+ 14 {C}_{{hash}} \\ {A3} {C}_{{m,p}} = & 19 {C}_{{key}}+14 {C}_{{enc}}+14 {C}_{{dec}}+14 {C}_{{ver}}+ 14 {C}_{{hash}} \\ {A4} {C}_{{m,p}} = & 14 {C}_{{key}}+14 {C}_{{enc}}+14 {C}_{{dec}}+14 {C}_{{ver}}+ 14 {C}_{{hash}} \end{array} $$

(44)

The value of i is set to 10, C_wd is set to 20, and C_ws is set to 10. The other costs such as key generation cost C_key, encryption cost C_enc, decryption cost C_dec, verification cost C_ver, and hash functions cost C_hash are set to one unit. The results obtained from the handover cost analysis of each algorithm are shown in Fig. 20. The value of R is set to 0.1 km, and v and H vary from 2 to 5 km/h and 1 to 7 hop count, respectively. Increasing both v and H results in more average cost of handover in the standard and other algorithms compared to the proposed algorithm. This is a proof that the handover cost is effectively reduced by the proposed algorithm, which makes it suitable for such heterogeneous architecture.

Fig. 20

Handover cost when v and H vary

Figure 21 shows the handover cost when v is set to 2 km/h, and R and H vary from 0.1 to 0.8 km and 1 to 6 hop count, respectively.

Fig. 21

Handover cost when R and H vary

In general, when the hop count increases, the average handover cost is affected in the case of standard and other algorithms. In the case of the proposed algorithm, the handover cost is slightly affected, since the authentication process is performed by the local servers and no need to communicate with the 3AAAS (The hop count is assumed to be between the local servers and the 3AAAS). For instance, in the standard algorithm A1, the AHC increases from 1215 to 1349 when H increases from 1 to 7. Whereas in the proposed algorithm A4, the AHC increases from 1056 to 1100. The reduction of handover cost in the proposed algorithm A4 reaches up to 22%, 18%, and 11% compared to algorithms A1, A2, and A3, respectively.

6.5 Energy consumption

The 4G and the 5G networks serve a huge number of users; thus, energy consumption is one of the most important issues that should be addressed. The reduction of the generated keys and the number of exchanged messages during the authentication process result in reducing energy consumption [43] and [44]. In general, the amount of energy consumed by wireless networks can be obtained by a linear equation,

$$\begin{array}{*{20}l} {Energy} = {M}.~{N} +~{B} \end{array} $$

(45)

where N is the total number of bytes sent or received by the UE, M is the incremental cost, and B is the fixed cost. Both the incremental and fixed cost are linear coefficients which have been computed in [45] by using experimental results for point to point model. The energy consumption can be calculated based on the number of bytes sent and received by UE as follows:

$$\begin{array}{*{20}l} {E}_{{trans}} &= \text{0.48}{N} +431 \\ {E}_{{rec}} &= \text{0.12}{N}+316 \end{array} $$

(46)

The findings in [45] are utilized in this section to roughly calculate the energy consumed by the UE in each movements. Then, the calculations are applied on A1, A2, A3, and A4 algorithms. Figure 22 shows that the energy consumed in A1, A2, A3, and A4 algorithms is increased whenever the UE performs movement (either an inter HO or intra HO) in the LTE-WLAN-WiMAX environment. However, the energy consumption of A4 is slightly increased in each movements compared to A1, A2, and A3 algorithms, since the number of bytes sent and received by UE is reduced by inter and intra re-authentication protocols. The proposed algorithm A4 achieves 17%, 13%, and 11% as a reduction in energy consumption compared to A1, A2, and A3, respectively.

Fig. 22

Energy consumption in each user movement

7 Conclusion and future work

Authentication protocols provide secure communication in the wireless networks by preventing unauthorized users from using the network resources. Nevertheless, it adds delay and overhead to the communication. These problems become more pertinent in the environment of heterogeneous wireless networks such as 5G networks. For the sake of avoiding the single point of failure and providing secure and fast inter and intra handovers in the 4G and 5G networks, authentication and re-authentication protocols have been proposed in this work. The standard authentication protocols have been enhanced to be more secure against authentication and secrecy attacks such as UID and MITM attacks. A new method has been proposed to protect the user identity, and this method is simple and does not add processing capability to the UE or HSS. Then, new re-authentication protocols have been proposed to locally perform fast re-authentication processes during handovers to the same domain or to a previously visited domain. The results of the analytical model show that the proposed protocols achieve better performance than the standard and other protocols in terms of handover delay, cost, and energy consumption. In addition, the verification tools show that the proposed protocols are secure and prevent all types of authentication and secrecy attacks in such environment. The future work could be conducted in the area of designing fault tolerant procedures which will improve the performance of authentication protocols. Those procedures work in case of detecting errors during the authentication process. Instead of rejecting the authentication and restart the authentication from the first step, the fault-tolerant procedure restarts the authentication process from the step where errors occurred; thus, the delay and cost of authentication and re-authentication processes will be effectively reduced. Another future work direction is to design authentication protocols for heterogeneous wireless sensors networks (WSN) which support Internet of Things (IOT) notion. The idea could be extended to design authentication and re-authentication protocols in the interworking of 5G and WSN. In WSN networks, the sink is considering as an authentication server. Each sensor node and the sink exchange the keys and perform a mutual authentication.