Lam Bacchus Figure 1 Graph G1 Joint Probabilities CSC 720 Spring 2001

A B C P(C|A,B) P(B|A) P(A) P(A,B,C)
T T T 0.7 0.8 0.3 0.168
T T F 0.3 0.8 0.3 0.072
T F T 0.1 0.2 0.3 0.006
T F F 0.9 0.2 0.3 0.054
F T T 0.8 0.1 0.7 0.056
F T F 0.2 0.1 0.7 0.014
F F T 0.2 0.9 0.7 0.126
F F F 0.8 0.9 0.7 0.504
            1.0000

P(B) = .168 + .072 + .056 + .014 = .31
P(C) = .168 + .006 + .056 + .126 = .356

Mutual Information in G1:
W(A,B) = .24 * log2(.24/(.3*.31)) + .06 * log2(.06/(.3*.69)) + .07 * log2(.07/(.7*.31)) + .63 * log2(.63/(.7*.69)) = .348
W(B,C) = .200
W(A,C) = .066

Lam Bacchus Figure 1 Graph G2 Joint Probabilities CSC 720 Spring 2001

A B C P(C|B) P(B|A) P(A) P(A,B,C)
T T T 0.75 0.8 0.3 0.180
T T F 0.25 0.8 0.3 0.060
T F T 0.15 0.2 0.3 0.009
T F F 0.85 0.2 0.3 0.051
F T T 0.75 0.1 0.7 0.0525
F T F 0.25 0.1 0.7 0.0175
F F T 0.15 0.9 0.7 0.0945
F F F 0.85 0.9 0.7 0.5355
            1.0000

P(B) = .180 + .060 + .0525 + .0175 = .31
P(C) = .180 + .009 + .0525 + .0945 = .336

Mutual Information in G2:
W(A,B) = same as G1
W(B,C) = .249
W(A,C) = .117

Cross-Entropy: C(G1,G2) = .168 * log2(.168/.180) + .072 * log2(.072/.06) + .006 * log2(.006/.009) + .054 * log2(.054/.051) + .056 * log2(.056/.0525) + .014 * log2(.014/.0175) + .126 * log2(.126/.0945) + .504 * log2(.504/.5355) = .008
Cross-Entropy: C(G2,G1) = .008