Cross-entropy

Information theory
Entropy Differential entropy Conditional entropy Joint entropy Mutual information Directed information Conditional mutual information Relative entropy Entropy rate Limiting density of discrete points
Asymptotic equipartition property Rate–distortion theory
Shannon's source coding theorem Channel capacity Noisy-channel coding theorem Shannon–Hartley theorem
v t e

In information theory, the cross-entropy between two probability distributions ${\displaystyle p}$ and ${\displaystyle q}$, over the same underlying set of events, measures the average number of bits needed to identify an event drawn from the set when the coding scheme used for the set is optimized for an estimated probability distribution ${\displaystyle q}$, rather than the true distribution ${\displaystyle p}$.