Unleashing Mask: Explore the Intrinsic Out-of-distribution Detection Capability

Jianing Zhu, Hengzhuang Li, Jiangchao Yao*, Tongliang Liu, Jianliang Xu, Bo Han*

*Corresponding author for this work

Research output: Chapter in book/report/conference proceedingConference proceedingpeer-review

Abstract

Out-of-distribution (OOD) detection is an indispensable aspect of secure AI when deploying machine learning models in real-world applications. Previous paradigms either explore better scoring functions or utilize the knowledge of outliers to equip the models with the ability of OOD detection. However, few of them pay attention to the intrinsic OOD detection capability of the given model. In this work, we generally discover the existence of an intermediate stage of a model trained on in-distribution (ID) data having higher OOD detection performance than that of its final stage across different settings, and further identify one critical data-level attribution to be learning with the atypical samples. Based on such insights, we propose a novel method, Unleashing Mask, which aims to restore the OOD discriminative capabilities of the well-trained model with ID data. Our method utilizes a mask to figure out the memorized atypical samples, and then finetune the model or prune it with the introduced mask to forget them. Extensive experiments and analysis demonstrate the effectiveness of our method. The code is available at: https://github.com/tmlr-group/Unleashing-Mask.
Original languageEnglish
Title of host publicationProceedings of the 40th International Conference on Machine Learning, ICML 2023
EditorsAndreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, Jonathan Scarlett
PublisherML Research Press
Pages43068-43104
Number of pages37
Volume202
Publication statusPublished - Jul 2023
Event40th International Conference on Machine Learning, ICML 2023 - Honolulu, United States
Duration: 23 Jul 202329 Jul 2023
https://icml.cc/Conferences/2023
https://proceedings.mlr.press/v202/
https://openreview.net/group?id=ICML.cc/2023/Conference

Publication series

NameProceedings of Machine Learning Research
Volume202
ISSN (Print)2640-3498

Conference

Conference40th International Conference on Machine Learning, ICML 2023
Country/TerritoryUnited States
CityHonolulu
Period23/07/2329/07/23
Internet address

Scopus Subject Areas

  • Software
  • Artificial Intelligence
  • Control and Systems Engineering
  • Statistics and Probability

Fingerprint

Dive into the research topics of 'Unleashing Mask: Explore the Intrinsic Out-of-distribution Detection Capability'. Together they form a unique fingerprint.

Cite this