Lam Bacchus Figure 1 Graph G1 Joint Probabilities CSC 720 Spring 2001 A B C P(C|A,B) P(B|A) P(A) P(A,B,C) T T T 0.7 0.8 0.3 0.168 T T F 0.3 0.8 0.3 0.072 T F T 0.1 0.2 0.3 0.006 T F F 0.9 0.2 0.3 0.054 F T T 0.8 0.1 0.7 0.056 F T F 0.2 0.1 0.7 0.014 F F T 0.2 0.9 0.7 0.126 F F F 0.8 0.9 0.7 0.504 1.0000

P(B) = .168 + .072 + .056 + .014 = .31
P(C) = .168 + .006 + .056 + .126 = .356

Mutual Information in G1:
W(A,B) = .24 * log2(.24/(.3*.31)) + .06 * log2(.06/(.3*.69)) + .07 * log2(.07/(.7*.31)) + .63 * log2(.63/(.7*.69)) = .348
W(B,C) = .200
W(A,C) = .066

 Lam Bacchus Figure 1 Graph G2 Joint Probabilities CSC 720 Spring 2001 A B C P(C|B) P(B|A) P(A) P(A,B,C) T T T 0.75 0.8 0.3 0.180 T T F 0.25 0.8 0.3 0.060 T F T 0.15 0.2 0.3 0.009 T F F 0.85 0.2 0.3 0.051 F T T 0.75 0.1 0.7 0.0525 F T F 0.25 0.1 0.7 0.0175 F F T 0.15 0.9 0.7 0.0945 F F F 0.85 0.9 0.7 0.5355 1.0000

P(B) = .180 + .060 + .0525 + .0175 = .31
P(C) = .180 + .009 + .0525 + .0945 = .336

Mutual Information in G2:
W(A,B) = same as G1
W(B,C) = .249
W(A,C) = .117

Cross-Entropy: C(G1,G2) = .168 * log2(.168/.180) + .072 * log2(.072/.06) + .006 * log2(.006/.009) + .054 * log2(.054/.051) + .056 * log2(.056/.0525) + .014 * log2(.014/.0175) + .126 * log2(.126/.0945) + .504 * log2(.504/.5355) = .008
Cross-Entropy: C(G2,G1) = .008