یک روش جدید برای ترکیب شبکه های بیزی، تجزیه و تحلیل نظری، و برنامه های کاربردی آن

Effective knowledge integration plays a very important role in knowledge engineering and knowledge-based machine learning. The combination of Bayesian networks (BNs) has shown a promising technique in knowledge fusion and the way of combining BNs remains a challenging research topic. An effective method of BNs combination should not impose any particular constraints on the underlying BNs such that the method is applicable to a variety of knowledge engineering scenarios. In general, a sound method of BNs combination should satisfy three fundamental criteria, that is, avoiding cycles, preserving the conditional independencies of BN structures, and preserving the characteristics of individual BN parameters, respectively. However, none of the existing BNs combination method satisfies all the aforementioned criteria. Accordingly, there are only marginal theoretical contributions and limited practical values of previous research on BNs combination. In this paper, following the approach adopted by existing BNs combination methods, we assume that there is an ancestral ordering shared by individual BNs that helps avoid cycles. We first design and develop a novel BNs combination method that focuses on the following two aspects: (1) a generic method for combining BNs that does not impose any particular constraints on the underlying BNs, and (2) an effective approach ensuring that the last two criteria of BNs combination are satisfied. Further through a formal analysis, we compare the properties of the proposed method and that of three classical BNs combination methods, and hence to demonstrate the distinctive advantages of the proposed BNs combination method. Finally, we apply the proposed method in recommender systems for estimating users' ratings based on their implicit preferences, bank direct marketing for predicting clients' willingness of deposit subscription, and disease diagnosis for assessing patients' breast cancer risk.

A Bayesian Network (BN) is a graphical representation of knowledge with intuitive structures and parameters [1], [2] and [3]. For many real world applications, a BN may be just a local model of the whole knowledge domain. It is desirable to combine the local models of the whole domain to create a global and more general representation of the knowledge domain [4] and [5]. The combination of BNs has been shown to be an effective approach for combining knowledge structures to some extent [6], [7] and [8]. A good method of combining BNs should be a generic one that ensures a combined BN meets three important criteria of avoiding cycles, preserving conditional independencies and preserving the characteristics of individual BN parameters. A method is generic if it does not impose any constraints on two given BNs to be combined, i.e. the method can be used to combine any two BNs. For instance, no newly generated v-structure (Definition 3 in Section 2) is the necessary condition for combining two BNs in all existing methods. Moreover, the first two criteria are qualitative, related to combining BN structures, and the third is about quantitative combination of BNs' parameters. The necessity of meeting these three criteria when combining any two BNs is threefold. First, the combination of two BNs may generate a cycle. Therefore, it should avoid any generated cycles when combining two BNs that are actually Directly Acyclic Graphs (DAGs) by nature. Second, because a BN is an accurate and concise presentation of a Joint Probability Distribution (JPD) in terms of conditional independencies, it is naturally required to preserve the original conditional independencies in individual BNs. Finally, preserving the characteristics of individual BN parameters ensures the probability inference with a high degree of accuracy. According to our best knowledge, all existing methods of combining BNs assume that the original BNs are equally important, i.e. without considering how many data samples the original BNs are generated from, since a BN itself does not include this kind of information. Further, in those methods, an ancestral ordering is assigned in advance by experts to avoid cycles. This assignation is easy and the problem is actually not a critical issue in combination of BNs. Thus, most of these methods mainly focus on the last two criteria. They combine either the structures, or the parameters, or both. In preserving conditional independencies, which is essential for combining the structures, there are two different perspectives. The first aims at preserving all conditional independencies in the two BNs to be combined while the second preserves only their common conditional independencies. These two approaches represent two extreme cases of preservation. Del Sagrado and Moral [6] prove that the intersection and the union of two BNs satisfy the two perspectives of preservations when the two BNs meet certain conditions. However, the result of the intersection or the union of two BNs is incomplete because these methods combine only the structures, i.e. unification of BN parameters is missing. More importantly, the intersection and the union stand for two extreme cases of preservation of conditional independencies only. This makes a combined BN degenerate quickly into a completely isolated graph or expanding to a completely connected graph. The reason is that the intersection takes the preservation of all conditional independencies for granted and the union greedily preserves as many dependencies as possible, but preserves only common conditional independencies. For instance, consider two simple BNs in Fig. 1(a): their intersection and union results, shown in Fig. 1(b) and (c), are an isolated graph and a directly complete graph, respectively. Full-size image (14 K) Fig. 1. Two BNs and their intersection and union results. (a) Two original BNs, (b) intersection and (c) union. Figure options Li et al. [8] propose a method which deals with preservation of both conditional independencies and characteristics of individual parameters. Nevertheless, their method, unlike a generic one, has a considerably rigid constraint. As a result, its application area is limited to only those BNs that are generated from third Normal Form (3NF) schemes of a relational database [8]. To the best of our knowledge, none of the existing methods of combining BNs is a generic method that also supports combination of structures and parameters. Following other existing BN combination methods, this paper assumes that there is an ancestral ordering shared by individual BNs that helps avoid cycles. Focusing on the other two main criteria of combining BNs (i.e. preserving conditional independencies and the characteristics of BN parameters), we develop a novel method for combining BNs to address the weakness of existing methods, i.e. a method that is generic and able to combine both BN structures and BN parameters. Further, we conduct a theoretical analysis of the necessary conditions such that meeting of the above criteria is guaranteed. In short, the proposed generic BNs combination method has the following three advantages compared to existing methods. First, the rigid constraint imposed by the method developed by Li et al. [8] is removed. As a result, our proposed method is generic and can be applied to integrate two BNs with any structures and parameters. Second, the proposed fuzzy fusion method of network parameters makes it possible to apply the method to various types of combinations including intersection and union of BNs. Finally, the set of conditional independencies preserved by the proposed method strives for a good balance between the two extremes produced by intersection- and union-based combinations, which is rather based on the sound principle of preserving the most important conditional independencies among individual BNs. Let us now consider the two BNs of Fig. 1(a) again. Two possible BNs that are constructed according to the proposed method are shown in Fig. 2. Essentially, each of the BNs is a compromise between an isolated graph shown in Fig. 1(b) and a complete graph depicted in Fig. 1(c). Full-size image (8 K) Fig. 2. Two combination results of Fig. 1(a) by the proposed method. Figure options Finally, to explicitly demonstrate the practical significance of the novel combination method, the method has been applied in recommender systems for estimating users' ratings based on their implicit preferences, bank direct marketing for predicting clients' willingness of deposit subscription, and disease diagnosis for assessing patients' breast cancer risk. The performance of the proposed BNs combination method is validated and investigated by comparing it with other methods based on the real world applications in recommender systems, bank direct marketing, and disease diagnosis. For more details, refer to Section 5. The main contributions of this paper can be summarized as follows: • A generic BNs combination method is developed to combine any BN structures and parameters. The proposed method not only removes the rigid constraint imposed by the method of Li et al. [8], but is also a good tradeoff between the two extremes of intersection and union methods (Section 3). • A theoretical analysis is conducted so that the relationships among the proposed combination method, the intersection method, the union method, and the method presented by Li et al. [8] are theoretically proved based on the property of conditional independencies, which demonstrates the distinctive advantages of the proposed BNs combination method (Section 4). • In the real world applications of recommender systems, bank direct marketing, and disease diagnosis, the proposed BNs combination significantly outperforms several well-known classifiers and their ensembles, the intersection method, and the union method in terms of classification accuracy. Further, the performance of the proposed BNs combination method is more stable than other methods. Last but not least, both training time and prediction time of the proposed method are only slightly higher than other individual classifiers (Section 5). The remainder of this paper is organized as follows. In Section 2 we review the related research work on some basic concepts of Bayesian networks and the existing methods for combining BNs. The proposed BNs combination method is illustrated in Section 3. In Section 4, we compare the proposed method and the three existing common methods through discussing their relationships. In Section 5, we present experiments using three real world applications in recommender systems, bank direct marketing and disease diagnosis, which mainly include the empirical evaluation of the proposed method and discussions on experimental results. Finally, we offer our concluding remarks and discuss future directions of our research work in Section 6.

The combination of Bayesian networks has been shown as an effective approach for combining knowledge. Motivated by the weakness of existing methods to combine BNs, two main criteria, that is, preserving conditional independencies and preserving the characteristics of individual BNs parameters, are exploited to develop a generic method to combine BNs. To remove the considerably rigid constraint of existing BNs combination methods, two rules are developed to guarantee that the proposed method is a generic one that can be applied to combine any BNs. These two rules focus on two different cases: interior variables and exterior variables in two BNs. The relationships of the sets of conditional independencies preserved by the proposed method, the intersection method, the union method, and the method of Li et al. [8] have been analyzed from a theoretical perspective. Finally, the performance of the proposed BNs combination method is validated and investigated in three real world applications of recommender systems, bank direct marketing, and disease diagnosis. Experimental results verified the effectiveness and efficiency of the method proposed in this paper. Future work involves further evaluation of the proposed method based on other real world applications. For example, in a vehicle-to-vehicle system, each vehicle communicates with others to exchange the traffic data for predicting traffic states [37] and [38]. However, the amount of raw data is usually huge and the communication is constrained by unreliable wireless links and packet collisions. It is planned to investigate the way by which individual vehicles exchange predictive models instead of raw data, and an associated approach for synthesizing those models.