|
||||||||||
PREV LETTER NEXT LETTER | FRAMES NO FRAMES |
I(P||Q) = sum_over_all_X's assignments{P(x)log(P(x)/Q(x))}, only picks those assignments when P(x)>0
if P(X)!(<<)Q(X), then:
I(P||Q) = positive infinity
Generally,I(P||Q) != I(Q||P)
Note: When P is dominated by Q, we have: 0/0 = 0 and 0*log(0/0) = 0.
see also:
http://en.wikipedia.org/wiki/Kullback-Leibler_divergence
http://en.wikipedia.org/wiki/Cross_entropy
Assume: P(X) and Q(X) are valid distributions, w.r.t our implementation here.- CrossEntropy(JointProbDistribution, JointProbDistribution) - Constructor for class umbc.ebiquity.BayesOWL.commonMethod.CrossEntropy
- Constructs a new class for cross entropy.
|
||||||||||
PREV LETTER NEXT LETTER | FRAMES NO FRAMES |