Neural architecture search

Part of a series on
Machine learning
and data mining
Paradigms
Problems
Clustering
Dimensionality reduction
Structured prediction
Anomaly detection
Artificial neural network
Reinforcement learning
Learning with humans
Model diagnostics
Mathematical foundations
Journals and conferences
Related articles

Neural architecture search (NAS)[1][2] is a technique for automating the design of artificial neural networks (ANN), a widely used model in the field of machine learning. NAS has been used to design networks that are on par with or outperform hand-designed architectures.[3][4] Methods for NAS can be categorized according to the search space, search strategy and performance estimation strategy used:[1]

  • The search space defines the type(s) of ANN that can be designed and optimized.
  • The search strategy defines the approach used to explore the search space.
  • The performance estimation strategy evaluates the performance of a possible ANN from its design (without constructing and training it).

NAS is closely related to hyperparameter optimization[5] and meta-learning[6] and is a subfield of automated machine learning (AutoML).[7]

  1. ^ a b Elsken, Thomas; Metzen, Jan Hendrik; Hutter, Frank (August 8, 2019). "Neural Architecture Search: A Survey". Journal of Machine Learning Research. 20 (55): 1–21. arXiv:1808.05377.
  2. ^ Wistuba, Martin; Rawat, Ambrish; Pedapati, Tejaswini (2019-05-04). "A Survey on Neural Architecture Search". arXiv:1905.01392 [cs.LG].
  3. ^ Cite error: The named reference Zoph 2016 was invoked but never defined (see the help page).
  4. ^ Zoph, Barret; Vasudevan, Vijay; Shlens, Jonathon; Le, Quoc V. (2017-07-21). "Learning Transferable Architectures for Scalable Image Recognition". arXiv:1707.07012 [cs.CV].
  5. ^ Matthias Feurer and Frank Hutter. Hyperparameter optimization. In: AutoML: Methods, Systems, Challenges, pages 3–38.
  6. ^ Vanschoren, Joaquin (2019). "Meta-Learning". Automated Machine Learning. The Springer Series on Challenges in Machine Learning. pp. 35–61. doi:10.1007/978-3-030-05318-5_2. ISBN 978-3-030-05317-8. S2CID 239362577.
  7. ^ Salehin, Imrus; Islam, Md. Shamiul; Saha, Pritom; Noman, S. M.; Tuni, Azra; Hasan, Md. Mehedi; Baten, Md. Abu (2024-01-01). "AutoML: A systematic review on automated machine learning with neural architecture search". Journal of Information and Intelligence. 2 (1): 52–81. doi:10.1016/j.jiixd.2023.10.002. ISSN 2949-7159.