TY - JOUR
T1 - AutoML
T2 - A survey of the state-of-the-art
AU - He, Xin
AU - Zhao, Kaiyong
AU - Chu, Xiaowen
N1 - Funding Information:
The research was supported by the grant RMGS2019_1_23 from Hong Kong Research Matching Grant Scheme . We would like to thank the anonymous reviewers for their valuable comments.
Publisher Copyright:
© 2020 Elsevier B.V.
PY - 2021/1/5
Y1 - 2021/1/5
N2 - Deep learning (DL) techniques have obtained remarkable achievements on various tasks, such as image recognition, object detection, and language modeling. However, building a high-quality DL system for a specific task highly relies on human expertise, hindering its wide application. Meanwhile, automated machine learning (AutoML) is a promising solution for building a DL system without human assistance and is being extensively studied. This paper presents a comprehensive and up-to-date review of the state-of-the-art (SOTA) in AutoML. According to the DL pipeline, we introduce AutoML methods – covering data preparation, feature engineering, hyperparameter optimization, and neural architecture search (NAS) – with a particular focus on NAS, as it is currently a hot sub-topic of AutoML. We summarize the representative NAS algorithms’ performance on the CIFAR-10 and ImageNet datasets and further discuss the following subjects of NAS methods: one/two-stage NAS, one-shot NAS, joint hyperparameter and architecture optimization, and resource-aware NAS. Finally, we discuss some open problems related to the existing AutoML methods for future research.
AB - Deep learning (DL) techniques have obtained remarkable achievements on various tasks, such as image recognition, object detection, and language modeling. However, building a high-quality DL system for a specific task highly relies on human expertise, hindering its wide application. Meanwhile, automated machine learning (AutoML) is a promising solution for building a DL system without human assistance and is being extensively studied. This paper presents a comprehensive and up-to-date review of the state-of-the-art (SOTA) in AutoML. According to the DL pipeline, we introduce AutoML methods – covering data preparation, feature engineering, hyperparameter optimization, and neural architecture search (NAS) – with a particular focus on NAS, as it is currently a hot sub-topic of AutoML. We summarize the representative NAS algorithms’ performance on the CIFAR-10 and ImageNet datasets and further discuss the following subjects of NAS methods: one/two-stage NAS, one-shot NAS, joint hyperparameter and architecture optimization, and resource-aware NAS. Finally, we discuss some open problems related to the existing AutoML methods for future research.
KW - Automated machine learning (autoML)
KW - Deep learning
KW - Hyperparameter optimization (HPO)
KW - Neural architecture search (NAS)
UR - http://www.scopus.com/inward/record.url?scp=85097576363&partnerID=8YFLogxK
U2 - 10.1016/j.knosys.2020.106622
DO - 10.1016/j.knosys.2020.106622
M3 - Journal article
SN - 0950-7051
VL - 212
JO - Knowledge-Based Systems
JF - Knowledge-Based Systems
M1 - 106622
ER -