Minimum message length

Minimum message length (MML) is a Bayesian information-theoretic method for statistical model comparison and selection.[1] It provides a formal information theory restatement of Occam's Razor: even when models are equal in their measure of fit-accuracy to the observed data, the one generating the most concise explanation of data is more likely to be correct (where the explanation consists of the statement of the model, followed by the lossless encoding of the data using the stated model). MML was invented by Chris Wallace, first appearing in the seminal paper "An information measure for classification".[2] MML is intended not just as a theoretical construct, but as a technique that may be deployed in practice.[3] It differs from the related concept of Kolmogorov complexity in that it does not require use of a Turing-complete language to model data.[4]

  1. ^ Wallace, C. S. (Christopher S.), -2004. (2005). Statistical and inductive inference by minimum message length. New York: Springer. ISBN 9780387237954. OCLC 62889003.{{cite book}}: CS1 maint: multiple names: authors list (link) CS1 maint: numeric names: authors list (link)
  2. ^ Wallace, C. S.; Boulton, D. M. (1968-08-01). "An Information Measure for Classification". The Computer Journal. 11 (2): 185–194. doi:10.1093/comjnl/11.2.185. ISSN 0010-4620.
  3. ^ Allison, Lloyd. (2019). Coding Ockham's Razor. Springer. ISBN 978-3030094881. OCLC 1083131091.
  4. ^ Wallace, C. S.; Dowe, D. L. (1999-01-01). "Minimum Message Length and Kolmogorov Complexity". The Computer Journal. 42 (4): 270–283. doi:10.1093/comjnl/42.4.270. ISSN 0010-4620.