KGTuner: Efficient Hyper-parameter Search for Knowledge Graph Learning

Yongqi Zhang; Zhanke Zhou; Quanming Yao; Yong Li

doi:10.18653/v1/2022.acl-long.194

KGTuner: Efficient Hyper-parameter Search for Knowledge Graph Learning

Yongqi Zhang, Zhanke Zhou, Quanming Yao^*, Yong Li

^*Corresponding author for this work

Department of Computer Science

Research output: Chapter in book/report/conference proceeding › Conference proceeding › peer-review

12 Citations (Scopus)

Abstract

While hyper-parameters (HPs) are important for knowledge graph (KG) learning, existing methods fail to search them efficiently. To solve this problem, we first analyze the properties of different HPs and measure the transfer ability from small subgraph to the full graph. Based on the analysis, we propose an efficient two-stage search algorithm KGTuner, which efficiently explores HP configurations on small subgraph at the first stage and transfers the top-performed configurations for fine-tuning on the large full graph at the second stage. Experiments show that our method can consistently find better HPs than the baseline algorithms within the same time budget, which achieves 9.1% average relative improvement for four embedding models on the large-scale KGs in open graph benchmark. Our code is released in https://github.com/AutoML-Research/KGTuner.

Original language	English
Title of host publication	ACL 2022 - 60th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers)
Editors	Smaranda Muresan, Preslav Nakov, Aline Villavicencio
Publisher	Association for Computational Linguistics (ACL)
Pages	2715-2735
Number of pages	21
ISBN (Electronic)	9781955917216
DOIs	https://doi.org/10.18653/v1/2022.acl-long.194
Publication status	Published - May 2022
Event	60th Annual Meeting of the Association for Computational Linguistics, ACL 2022 - Dublin, Ireland Duration: 22 May 2022 → 27 May 2022 https://www.2022.aclweb.org/

Publication series

Name	Proceedings of the Annual Meeting of the Association for Computational Linguistics
Volume	1
ISSN (Print)	0736-587X

Conference

Conference	60th Annual Meeting of the Association for Computational Linguistics, ACL 2022
Country/Territory	Ireland
City	Dublin
Period	22/05/22 → 27/05/22
Internet address	https://www.2022.aclweb.org/

Access to Document

10.18653/v1/2022.acl-long.194

Cite this

Zhang, Y., Zhou, Z., Yao, Q., & Li, Y. (2022). KGTuner: Efficient Hyper-parameter Search for Knowledge Graph Learning. In S. Muresan, P. Nakov, & A. Villavicencio (Eds.), ACL 2022 - 60th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers) (pp. 2715-2735). (Proceedings of the Annual Meeting of the Association for Computational Linguistics; Vol. 1). Association for Computational Linguistics (ACL). https://doi.org/10.18653/v1/2022.acl-long.194

Zhang, Yongqi ; Zhou, Zhanke ; Yao, Quanming et al. / KGTuner: Efficient Hyper-parameter Search for Knowledge Graph Learning. ACL 2022 - 60th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers). editor / Smaranda Muresan ; Preslav Nakov ; Aline Villavicencio. Association for Computational Linguistics (ACL), 2022. pp. 2715-2735 (Proceedings of the Annual Meeting of the Association for Computational Linguistics).

@inproceedings{9c53b335ab244d5e88eb3fc25f517d46,

title = "KGTuner: Efficient Hyper-parameter Search for Knowledge Graph Learning",

abstract = "While hyper-parameters (HPs) are important for knowledge graph (KG) learning, existing methods fail to search them efficiently. To solve this problem, we first analyze the properties of different HPs and measure the transfer ability from small subgraph to the full graph. Based on the analysis, we propose an efficient two-stage search algorithm KGTuner, which efficiently explores HP configurations on small subgraph at the first stage and transfers the top-performed configurations for fine-tuning on the large full graph at the second stage. Experiments show that our method can consistently find better HPs than the baseline algorithms within the same time budget, which achieves 9.1% average relative improvement for four embedding models on the large-scale KGs in open graph benchmark. Our code is released in https://github.com/AutoML-Research/KGTuner.",

author = "Yongqi Zhang and Zhanke Zhou and Quanming Yao and Yong Li",

note = "This work was supported in part by The National Key Research and Development Program of China under grant 2020AAA0106000. Publisher Copyright: {\textcopyright} 2022 Association for Computational Linguistics.; 60th Annual Meeting of the Association for Computational Linguistics, ACL 2022 ; Conference date: 22-05-2022 Through 27-05-2022",

year = "2022",

month = may,

doi = "10.18653/v1/2022.acl-long.194",

language = "English",

series = "Proceedings of the Annual Meeting of the Association for Computational Linguistics",

publisher = "Association for Computational Linguistics (ACL)",

pages = "2715--2735",

editor = "Smaranda Muresan and Preslav Nakov and Aline Villavicencio",

booktitle = "ACL 2022 - 60th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers)",

address = "Australia",

url = "https://www.2022.aclweb.org/",

}

Zhang, Y, Zhou, Z, Yao, Q & Li, Y 2022, KGTuner: Efficient Hyper-parameter Search for Knowledge Graph Learning. in S Muresan, P Nakov & A Villavicencio (eds), ACL 2022 - 60th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers). Proceedings of the Annual Meeting of the Association for Computational Linguistics, vol. 1, Association for Computational Linguistics (ACL), pp. 2715-2735, 60th Annual Meeting of the Association for Computational Linguistics, ACL 2022, Dublin, Ireland, 22/05/22. https://doi.org/10.18653/v1/2022.acl-long.194

KGTuner: Efficient Hyper-parameter Search for Knowledge Graph Learning. / Zhang, Yongqi; Zhou, Zhanke; Yao, Quanming et al.
ACL 2022 - 60th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers). ed. / Smaranda Muresan; Preslav Nakov; Aline Villavicencio. Association for Computational Linguistics (ACL), 2022. p. 2715-2735 (Proceedings of the Annual Meeting of the Association for Computational Linguistics; Vol. 1).

Research output: Chapter in book/report/conference proceeding › Conference proceeding › peer-review

TY - GEN

T1 - KGTuner: Efficient Hyper-parameter Search for Knowledge Graph Learning

AU - Zhang, Yongqi

AU - Zhou, Zhanke

AU - Yao, Quanming

AU - Li, Yong

PY - 2022/5

Y1 - 2022/5

N2 - While hyper-parameters (HPs) are important for knowledge graph (KG) learning, existing methods fail to search them efficiently. To solve this problem, we first analyze the properties of different HPs and measure the transfer ability from small subgraph to the full graph. Based on the analysis, we propose an efficient two-stage search algorithm KGTuner, which efficiently explores HP configurations on small subgraph at the first stage and transfers the top-performed configurations for fine-tuning on the large full graph at the second stage. Experiments show that our method can consistently find better HPs than the baseline algorithms within the same time budget, which achieves 9.1% average relative improvement for four embedding models on the large-scale KGs in open graph benchmark. Our code is released in https://github.com/AutoML-Research/KGTuner.

AB - While hyper-parameters (HPs) are important for knowledge graph (KG) learning, existing methods fail to search them efficiently. To solve this problem, we first analyze the properties of different HPs and measure the transfer ability from small subgraph to the full graph. Based on the analysis, we propose an efficient two-stage search algorithm KGTuner, which efficiently explores HP configurations on small subgraph at the first stage and transfers the top-performed configurations for fine-tuning on the large full graph at the second stage. Experiments show that our method can consistently find better HPs than the baseline algorithms within the same time budget, which achieves 9.1% average relative improvement for four embedding models on the large-scale KGs in open graph benchmark. Our code is released in https://github.com/AutoML-Research/KGTuner.

UR - http://www.scopus.com/inward/record.url?scp=85133377036&partnerID=8YFLogxK

U2 - 10.18653/v1/2022.acl-long.194

DO - 10.18653/v1/2022.acl-long.194

M3 - Conference proceeding

AN - SCOPUS:85133377036

T3 - Proceedings of the Annual Meeting of the Association for Computational Linguistics

SP - 2715

EP - 2735

BT - ACL 2022 - 60th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers)

A2 - Muresan, Smaranda

A2 - Nakov, Preslav

A2 - Villavicencio, Aline

PB - Association for Computational Linguistics (ACL)

T2 - 60th Annual Meeting of the Association for Computational Linguistics, ACL 2022

Y2 - 22 May 2022 through 27 May 2022

ER -

Zhang Y, Zhou Z, Yao Q, Li Y. KGTuner: Efficient Hyper-parameter Search for Knowledge Graph Learning. In Muresan S, Nakov P, Villavicencio A, editors, ACL 2022 - 60th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers). Association for Computational Linguistics (ACL). 2022. p. 2715-2735. (Proceedings of the Annual Meeting of the Association for Computational Linguistics). doi: 10.18653/v1/2022.acl-long.194

KGTuner: Efficient Hyper-parameter Search for Knowledge Graph Learning

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this