Research on wireless distributed financial risk data stream mining based on dual privacy protection

Abstract

With the advancement of network technology and large-scale computing, distributed data streams have been widely used in the application of financial risk analysis. However, while data mining reveals financial models, it also increasingly poses a threat to privacy. Therefore, how to prevent privacy leakage during the efficient mining process poses new challenges to the data mining technology. This article is mainly aimed at the current privacy data leakage in financial data mining, combined with existing data mining technology to study data mining and privacy protection. First, a data mining model for dual privacy protection is defined, which can better meet the characteristics of distributed data streams while achieving privacy protection effects. Secondly, a privacy-oriented data stream mining algorithm is proposed, which uses random interference technology to effectively protect the original sensitive data. Finally, the analysis and discussion of the algorithm in this paper through simulation experiments show that the algorithm is feasible and effective, and can better adapt to the distributed data flow distribution and dynamic characteristics, while achieving better privacy protection effects, effectively reduced communication load.

1 Introduction

In recent years, the rapid development of information processing technology and storage technology has enabled relevant financial institutions to collect large amounts of data for data mining. In the process of data mining, multiple financial data owners may be required to publish or share the data they own [1, 2]. However, directly publishing and sharing the original financial data will lead to the disclosure of personal privacy information. In this case, the data owner is in a dilemma [3]. On the one hand, the privacy of personal data needs to be protected, and on the other hand, data availability needs to be guaranteed for data mining tasks. In order to solve this problem, the publication of privacy-protected data for data mining came into being and has become a very active research field [4, 5].

This field mainly studies how to publish data that does not disclose private information, while ensuring that the published data can be used for data mining. At present, a lot of research work has emerged for different data types, different application scenarios and different attack models. Agrawal et al. first proposed an encrypted data perturbation method for building decision tree classifiers. This method assumes that each client has a numeric attribute, and the data mining service needs to learn the distribution of these attribute values to build a classifier model [6]. Argueta et al. questioned the availability of encryption random noise and pointed out that addition noise can be filtered out in many cases, thus leading to privacy disclosure [7]. For more specific verification, a random matrix-based spectral filtering method is proposed to reconstruct the original data from the perturbed data. Experimental results show that the reconstructed data are very close to the original data. Huang et al. pointed out that the main factor that determines the accuracy of reconstructed data is the correlation between the attributes of the original data [8]. The research results show that the greater the correlation between the attributes of the original data, the higher the accuracy of the reconstructed data, so it will lead to the leakage of more private information. They further proposed two reconstruction methods based on data correlation, using principal component analysis technology and Bayesian estimation technology, respectively. Among them, the method based on data perturbation is easy to implement and has a strict mathematical theory foundation, which has strong practicability and reliability. However, there are still some problems to be solved [9, 10].

This paper takes financial risk as the research object and uses distributed privacy protection and random interference to design distributed data stream mining technology. By accessing the financial risk information platform, personal privacy data can be dynamically and accurately transmitted to the database, and real-time tracking and analysis of information can be achieved. This technology can improve the information transparency and data security of financial risk data mining, and provide an effective and implementable solution for promoting the development of financial undertakings.

2 Related work

The spillover effect of financial risks is inseparable from the study of individual specific risks. A single specific risk is the basis and source of financial risk spillover effects [11]. Therefore, when studying the spillover effect of financial risks, we should first study the specific financial risks. To study the risk spillover effect between the banking industry and the securities industry, first of all, the risks faced by the banking industry and the securities industry should be studied. On this basis, the mechanism of the risk spillover effect between the banking industry and the securities industry is elaborated [12]. This article will make a further theoretical analysis of the risk theory and the spillover effect between the banking industry and the securities industry. Regarding the causes of financial risks, many theories give their explanations. Schematic diagram of financial risk concept is shown in Fig. 1.

Fig. 1

Schematic diagram of financial risk concept

2.1 Financial instability theory

The main explanations for the theory of financial instability include the "cyclic explanation" group led by American economists Hyman Minsky and Charles Kinder Berger and the "monetarist explanation" group led by Friedman. The former believes that finance is an important part of various economies, so with the prosperity, recession, depression and recovery rhythm of the economic cycle, there will also be periodic instability in finance, namely financial risk [13]. This is consistent with the interpretation of the New Palgrave Dictionary of Monetary and Financial Affairs. According to the interpretation of the dictionary, the financial instability hypothesis refers to the inherent tendency of private credit creation institutions, especially commercial banks and related lenders, to experience cyclical crises and bankruptcy [14]. Because of the reality of its social life, that is, its close relationship with the real economy, finance has planted the seeds of financial instability during the period of economic and social prosperity.

2.2 Asymmetric information theory

Information economics scholars believe that the information in the economy is incomplete, that is, asymmetric, and one party of the economic actor knows more information than the other party. Moral hazard and adverse selection are the products of information asymmetry [15, 16]. Moral risk refers to the information asymmetry after the transaction occurs. There are three main manifestations of moral risk in the loan market: first, unauthorized changes in the use of funds. The second is that some borrowers with repayment ability conceal their true income status and delay or refrain from returning the loan. This situation often occurs in the absence of sanctions for breach of contract [17]. The third is that after the borrower obtains the funds, he does not pay attention to the use of the funds, so that the use efficiency of the funds cannot be guaranteed, resulting in losses of the borrowed funds.

2.3 The theory of sharp fluctuations in asset prices

Financial risk directly manifests as the loss of currency, so dramatic fluctuations in asset prices can directly cause instability in the financial system and trigger financial risks. As we all know, the price of financial assets always fluctuates from one peak to another, and this fluctuation is an important source of financial risk. The causes of financial asset price fluctuations are mainly concentrated in the following three aspects: first, there is excessive speculation; second, the uncontrolled use of credit and leveraged transactions; third, macroeconomic fluctuations.

2.4 International communication theory of financial risk

The globalization of international capital, the globalization of financial institutions and financial markets not only bring us a more dynamic economic structure, but also bring us a more complex financial risk environment. Financial globalization provides a free-flowing capital carrier for the international spread of finance. In particular, some developing countries have relaxed foreign exchange control in order to meet the needs of their own economic development and the integration of the world economy [18]. At the same time, however, the corresponding financial supervision system has not been formed, which has led to the larger-scale capital transfer becoming a carrier for the spread of financial risks internationally.

3 Privacy protection related technology

Before delving into the details of our proposed method, we first reexamine the existing methods, such as double privacy protection and random interference.

3.1 Double privacy protection

In collaborative data mining, there are two forms of data distribution, one is horizontal division and the other is vertical division. Horizontally divided data refer to the fact that the data held by each participant contain the same instance, but the data attributes contained in the instance are different. Vertically dividing data mean that the data held by each participant contain different instances, but the data attributes are the same. Figure 2 depicts the data perturbation during data level division [19, 20]. Suppose A and B are two participants in collaborative data mining. Since the data records are horizontally distributed in two places, A and B, in order to protect the privacy of their data, they need to randomly disturb the data they own before the data are released [21].

Fig. 2

Aiming at the horizontal division of privacy protection data sharing

Therefore, in order to ensure the uniformity of the projection results, A and B need to use the same random matrix for projection disturbance. In this way, the risk of random matrix leakage is faced. In order to ensure that the original data are not reconstructed, this paper proposes a noise random projection data perturbation method that meets differential privacy protection [22].

A key issue in the design of any privacy protection method is that the attacker will use prior knowledge of a certain amount of data to infer privacy protection content. However, when designing a privacy protection algorithm, it is impossible to accurately predict the amount of a priori knowledge that an attacker can grasp, so no strong proof can be given for an unknowable attack. Differential privacy provides a strong, provable and strict mathematical guarantee for privacy protection algorithm protection [23].

Aiming at the distributed data flow environment, a system model is constructed in which multiple nodes publish privacy protection data in real time, and integrate data from each node for data mining [24]. Figure 3 shows the privacy protection node model for distributed data mining. As shown in Fig. 3, the aggregation node is responsible for collecting the data of the subordinate terminal nodes and then transmitting them to the management service node Adm. Node Adm needs to classify, cluster and anomaly detect data. In order to protect the privacy of the original data, the original private data content of each terminal node is not exposed during the data transmission process or reaching the management service node. Suppose that when the aggregation node m broadcasts a data collection instruction to the terminal nodes in its area, n terminal nodes respond [25]. Then, next m will collect the data transmitted by the n terminal nodes. Data sent by n terminal nodes can be regarded as n asynchronously updated data streams. Before transmitting data to the aggregation node, the terminal node first disturbs the data.

Fig. 3

The privacy protection node model for distributed data mining

On the one hand, only the disturbed data are obtained, and the attacker cannot determine the original data. On the other hand, the disturbed data still maintain the measured distance between the original data, and the data mining algorithm can be directly applied to the disturbed data without accessing the original sensitive information.

3.2 Random interference technology

The symmetrical difference between the two data sets X, X' is defined as

$$X \ominus X^{\prime } = \left\{ {x|x \in X \cup X^{\prime } - X \cap X^{\prime } } \right\}$$

(1)

If you count for any two data sets $X, X \in D^{\prime }$, and any $\hat{D} \subseteq {\text{Range}}(A)$ is satisfied

$$\Pr \left[ {A(X) \in \hat{D}} \right] \leqq e^{{[X, X^{\prime}]}} \Pr \left[ {A\left( {X^{\prime } } \right) \in \hat{D}} \right]$$

(2)

Then, the algorithm satisfies differential privacy [26].

The point cloud models are expressed as $P_{1}$ and $P_{2}$, respectively, and the corresponding number of point clouds is counted. Calculate the positions of the global and local point clouds $P_{1}$ and $P_{2}$, respectively, projected in the financial distributed data stream [27].

Suppose that you have obtained the point pairs $P_{k1}$ and $P_{k2}$ in the point cloud models $P_{1}$ and $P_{2}$. By finding an approximate transformation T, the coordinate alignment of all corresponding point sets in $P_{k1}$ and $P_{k2}$ is completed. The transformation T is composed of a translation vector L, a rotation matrix R and a scaling factor s, so the transformation objective function problem of the transformation T problem is solved.

$$E\left( {P_{k1} ,P_{k2} } \right) = \left\| {P_{k1} - \left( {s \cdot P_{k1} \cdot R + L} \right)} \right\|$$

(3)

Equation (3) shows that the optimization problem of the function is the Procrustes problem. Use the following steps to gradually solve the corresponding transformation, and use Eq. (4) to calculate the geometric centers of $P_{k1}$ and $P_{k2}$.

$$O = \mathop \sum \limits_{i = 1}^{N} X_{i} /N$$

(4)

In which, O represents the geometric center, $X_{i}$ represents the coordinate of the ith point, and N represents the number of point clouds. After the geometric centers $O_{1}$ and $O_{2}$ of $P_{k1}$ and $P_{k2}$ are obtained, the translation vector L is calculated using Eq. (5).

$$L = \left( {O_{1} - O_{2} } \right) + \left( {O - O_{1} } \right)$$

(5)

The coordinates of $P_{k1}$ and $P_{k2}$ are normalized, without considering the scaling factor, and expressed by Eq. (6).

$$E\left( {P_{k1} ,P_{k2} } \right) = \left\| {P_{k1} - P_{k2} \cdot R} \right\|$$

(6)

$$R = U \cdot V^{{\text{T}}}$$

(7)

In order to meet the invariance of the internal structure during the conversion process, the rotation matrix R needs to satisfy det (R) = 1, and the minimization problem is solved using Procrustes orthogonal problem theory; the optimal rotation matrix R is solved by Eq. (8).

$$s = {\text{tr}}\left( {P_{K1}^{{\text{T}}} \cdot P_{k2} \cdot R} \right)/{\text{tr}}\left( {P_{K2}^{{\text{T}}} \cdot P_{k2} } \right)$$

(8)

In which, U and $V^{{\text{T}}}$ are the eigenvector matrix results obtained by singular value decomposition, and the scaling factor is calculated using Eq. (9).

$$\tilde{P}_{2} = s \cdot P_{2} \cdot R + L$$

(9)

Figure 4 is a schematic diagram of disturbance data released by financial risk distributed terminals. First determine the projection dimension, then perturb each updated attribute value and finally calculate the perturbation data. The figure shows an example of data mining performed by n terminals issuing disturbance data. Each terminal disturbs its original data stream and sends it to the base station. The base station summarizes the disturbance data coming from each terminal and integrates it into an aggregate data set for subsequent data mining work [28].

Fig. 4

Schematic diagram of the disturbance data released by the terminal

4 Methods

4.1 Test environment and data

The experiment selects artificial data and real data separately for testing. The real data contain two data sets. One is the financial risk distributed data set, which contains some financial risk data obtained from a financial platform. Use the data structure commonly used in text data mining to represent the data into a document matrix. In the experiment, the financial risk data set with 200 attributes after feature selection is selected [29]. Another real data set is a financial risk data set published by a financial analysis agency. By selecting the frequency, a sparse integer type data set is generated, and the data record has an attribute dimension of 1000. In addition, the experiment generated an artificial data set with a data attribute dimension of 100 and sparsity from 1 to 100. The experimental environment is: the hardware environment is Intel(R) Core (TM) i7 CPU @2.40 GHz, the installed memory is 4 GB, the software environment is Windows 10 operating system, and the algorithms are implemented in Python.

4.2 Experiment analysis

Data sparsity and reconstruction data quality experiments first selected a data record with the most nonzero elements from a financial platform and a financial analysis institution data set [29].

Among them, the data selected from a financial platform contain 10 nonzero items with a data sparsity of 4.37%, and the data selected from a financial analysis institution contain 82 nonzero items with a data sparsity of 8.2%. First, the random projection data perturbation method is used to perturb the data to generate perturbation data with 50% projection ratio, and then, the original data are reconstructed from the perturbation data. Figures 5 shows the comparison between the original data and the reconstructed data in the data set of a financial platform and a financial analysis institution. It can be seen that the original data records are accurately reconstructed [30].

Fig. 5

Comparison of original data and reconstructed data in a financial platform data set

4.3 Results

The following will specifically analyze the relationship between the sparsity of the original financial data and the quality of the reconstructed data, and the relationship between the projection dimension of the disturbance data and the quality of the reconstructed data. In the test of the sparsity of the original financial data and the quality of the reconstructed data, an artificial data set containing data records of different sparsity was generated [31]. The original record dimension in the data set is 100, the projection disturbance dimension is 50, and the data types include continuous type, Boolean type and discrete integer type. Among them, the unknown original financial data type is first tested, and the quality of the original data is approximately reconstructed, and then, the quality of the original data is accurately reconstructed when the original data type is known. In the experiment, the quality of the original data is evaluated by the degree of distortion of the reconstructed data [32]. Boolean unrefined reconstruction data distortion is shown in Fig. 6. Discrete integer unreconstructed reconstruction data distortion is shown in Fig. 7. The repair effect after the original data is disturbed is shown in Fig. 8. Figure 9 shows the repair effect after the reconstructed data is disturbed [33].

Fig. 6

Boolean unrefined reconstruction data distortion

Fig. 7

Discrete integer unreconstructed reconstruction data distortion

Fig. 8

Repair effect after interference-original data

Fig. 9

Repair effect after interference-reconstruction data

As can be seen from Figs. 7, 8 and 9, when the sparsity is about 20%, after the data sparsity reaches about 20%, the distortion of the reconstructed data starts to increase; the degree of distortion is 0 or approximately 0. This is because, when the projection dimension is constant, as long as the number of nonzero items contained in the data record is less than a certain value, the data reconstruction method based on minimization can find a unique sparse solution. This means that data records with sparseness within this range can be accurately or approximately reconstructed. Comparing Figs. 7, 8 and 9, it can be found that if the original data is known to be Boolean or discrete integer, then the precision can slightly increase the number of accurate reconstruction data and the quality of the reconstruction data. This is because the reconstructed data will have a slight error compared to the original data [34]. When the original data type is known, the reconstruction algorithm can remove this error, making the reconstruction result more accurate. Of course, even if the original data type is unknown, this small error does not affect the accuracy of the reconstruction result [35].

5 Discussion

This research focuses on the dual privacy protection of the original data and rules, as well as the high efficiency and privacy protection required by large-scale distributed computing. Based on the analysis of financial risks, this paper presents a distributed data stream mining processing model for dual privacy protection. While solving the problem of privacy leakage, it can better mine the knowledge contained in the distributed data stream. In addition, this paper proposes a privacy oriented distributed data mining algorithm. According to the given processing model, random interference and double privacy protection technology are used to protect the original sensitive data, while increasing the incremental judgment and transmission of critical closure. Finally, simulation experiments are carried out on the algorithm proposed in this paper. The results show that the algorithm is getting better privacy protection effect. In addition, the distributed data stream mining for privacy protection researched in this paper is still in the theoretical research stage and should be more closely integrated with the reality to make further improvements.