Fisher information metric

In information geometry, the Fisher information metric[1] is a particular Riemannian metric which can be defined on a smooth statistical manifold, i.e., a smooth manifold whose points are probability measures defined on a common probability space. It can be used to calculate the informational difference between measurements.[clarification needed]

The metric is interesting in several aspects. By Chentsov’s theorem, the Fisher information metric on statistical models is the only Riemannian metric (up to rescaling) that is invariant under sufficient statistics.[2][3]

It can also be understood to be the infinitesimal form of the relative entropy (i.e., the Kullback–Leibler divergence); specifically, it is the Hessian of the divergence. Alternately, it can be understood as the metric induced by the flat space Euclidean metric, after appropriate changes of variable. When extended to complex projective Hilbert space, it becomes the Fubini–Study metric; when written in terms of mixed states, it is the quantum Bures metric.

Considered purely as a matrix, it is known as the Fisher information matrix. Considered as a measurement technique, where it is used to estimate hidden parameters in terms of observed random variables, it is known as the observed information.

  1. ^ Nielsen, Frank (2023). "A Simple Approximation Method for the Fisher–Rao Distance between Multivariate Normal Distributions". Entropy. 25 (4): 654. arXiv:2302.08175. Bibcode:2023Entrp..25..654N. doi:10.3390/e25040654. PMC 10137715. PMID 37190442.
  2. ^ Amari, Shun-ichi; Nagaoka, Horishi (2000). "Chentsov's theorem and some historical remarks". Methods of Information Geometry. New York: Oxford University Press. pp. 37–40. ISBN 0-8218-0531-2.
  3. ^ Dowty, James G. (2018). "Chentsov's theorem for exponential families". Information Geometry. 1 (1): 117–135. arXiv:1701.08895. doi:10.1007/s41884-018-0006-4. S2CID 5954036.