自动化机器学习中的超参调优方法

Translated title of the contribution: Hyperparameter tuning methods in automated machine learning

张爱军*, 杨泽斌

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

1 Citation (Scopus)

Abstract

In this paper, we make a comprehensive review of the challenging task of hyperparameter optimization in automated machine learning. The commonly used hyperparameter optimization methods include single-shot sampling strategies, e.g., grid search, random search and sequential strategies where new trials are gradually augmented based on existing information, including Bayesian optimization, evolutionary algorithms, reinforcement learning-based methods.We find the sequential number-theoretic optimization (SNTO) algorithm proposed by Kai-Tai Fang and Yuan Wang in 1990 can also be applied in hyperparameter optimization, where sequential uniform designs are utilized to search the global optima in complex response surfaces. For illustration, various hyperparameter optimization methods are tested with two widely-used machine learning models including the support vector machine (SVM) and extreme gradient boosting (XGBoost), on two classical binary classification datasets. By analyzing the experimental results, we find a modified SNTO algorithm is quite promising in the hyperparameter optimization task.

Translated title of the contributionHyperparameter tuning methods in automated machine learning
Original languageChinese (Simplified)
Pages (from-to)695-710
Number of pages16
Journal中国科学: 数学
Volume50
Issue number5
DOIs
Publication statusPublished - May 2020

Scopus Subject Areas

  • Mathematics(all)

User-Defined Keywords

  • AutoML
  • Bayesian optimization
  • Hyperparameter optimization
  • Sequential uniform designs

Fingerprint

Dive into the research topics of 'Hyperparameter tuning methods in automated machine learning'. Together they form a unique fingerprint.

Cite this