Wake-sleep algorithm

Layers of the neural network. R, G are weights used by the wake-sleep algorithm to modify data inside the layers.

The wake-sleep algorithm[1] is an unsupervised learning algorithm for deep generative models, especially Helmholtz Machines.[2] The algorithm is similar to the expectation-maximization algorithm,[3] and optimizes the model likelihood for observed data.[4] The name of the algorithm derives from its use of two learning phases, the “wake” phase and the “sleep” phase, which are performed alternately.[1] It can be conceived as a model for learning in the brain,[5] but is also being applied for machine learning.[6]

  1. ^ a b Hinton, Geoffrey E.; Dayan, Peter; Frey, Brendan J.; Neal, Radford (1995-05-26). "The wake-sleep algorithm for unsupervised neural networks". Science. 268 (5214): 1158–1161. Bibcode:1995Sci...268.1158H. doi:10.1126/science.7761831. PMID 7761831. S2CID 871473.
  2. ^ Dayan, Peter. "Helmholtz Machines and Wake-Sleep Learning" (PDF). Retrieved 2015-11-01.
  3. ^ Ikeda, Shiro; Amari, Shun-ichi; Nakahara, Hiroyuki (1998). "Convergence of the Wake-Sleep Algorithm". Advances in Neural Information Processing Systems. 11. MIT Press.
  4. ^ Frey, Brendan J.; Hinton, Geoffrey E.; Dayan, Peter (1996-05-01). "Does the wake-sleep algorithm produce good density estimators?" (PDF). Advances in Neural Information Processing Systems.
  5. ^ Katayama, Katsuki; Ando, Masataka; Horiguchi, Tsuyoshi (2004-04-01). "Models of MT and MST areas using wake–sleep algorithm". Neural Networks. 17 (3): 339–351. doi:10.1016/j.neunet.2003.07.004. PMID 15037352.
  6. ^ Cite error: The named reference :0 was invoked but never defined (see the help page).