Instance-dependent Label-noise Learning under a Structural Causal Model

Yu Yao; Tongliang Liu; Mingming Gong; Bo Han; Gang Niu; Kun Zhang

Instance-dependent Label-noise Learning under a Structural Causal Model

Yu Yao, Tongliang Liu^*, Mingming Gong, Bo Han, Gang Niu, Kun Zhang

^*Corresponding author for this work

Department of Computer Science

Research output: Chapter in book/report/conference proceeding › Conference proceeding › peer-review

34 Citations (Scopus)

Abstract

Label noise generally degenerates the performance of deep learning algorithms because deep neural networks easily overfit label errors. Let X and Y denote the instance and clean label, respectively. When Y is a cause of X, according to which many datasets have been constructed, e.g., SVHN and CIFAR, the distributions of P(X) and P(Y |X) are generally entangled. This means that the unsupervised instances are helpful to learn the classifier and thus reduce the side effect of label noise. However, it remains elusive on how to exploit the causal information to handle the label-noise problem. We propose to model and make use of the causal process in order to correct the label-noise effect. Empirically, the proposed method outperforms all state-of-the-art methods on both synthetic and real-world label-noise datasets.

Original language	English
Title of host publication	35th Conference on Neural Information Processing Systems (NeurIPS 2021)
Editors	Marc'Aurelio Ranzato, Alina Beygelzimer, Yann Dauphin, Percy S. Liang, Jenn Wortman Vaughan
Publisher	Neural Information Processing Systems Foundation
Pages	4409-4420
Number of pages	12
Volume	6
ISBN (Print)	9781713845393
Publication status	Published - 6 Dec 2021
Event	35th Conference on Neural Information Processing Systems, NeurIPS 2021 - Virtual Duration: 6 Dec 2021 → 14 Dec 2021 https://nips.cc/Conferences/2021 (Conference website) https://neurips.cc/Conferences/2021 (Conference website) https://papers.nips.cc/paper_files/paper/2021 (Conference proceedings) https://proceedings.neurips.cc/paper/2021 (Conference proceedings)

Publication series

Name	Advances in Neural Information Processing Systems
Volume	34
ISSN (Print)	1049-5258

Name	NeurIPS Proceedings

Conference

Conference	35th Conference on Neural Information Processing Systems, NeurIPS 2021
Period	6/12/21 → 14/12/21
Internet address	https://nips.cc/Conferences/2021 (Conference website) https://neurips.cc/Conferences/2021 (Conference website) https://papers.nips.cc/paper_files/paper/2021 (Conference proceedings) https://proceedings.neurips.cc/paper/2021 (Conference proceedings)

Scopus Subject Areas

Computer Networks and Communications
Information Systems
Signal Processing

Cite this

Yao, Y., Liu, T., Gong, M., Han, B., Niu, G., & Zhang, K. (2021). Instance-dependent Label-noise Learning under a Structural Causal Model. In MA. Ranzato, A. Beygelzimer, Y. Dauphin, P. S. Liang, & J. Wortman Vaughan (Eds.), 35th Conference on Neural Information Processing Systems (NeurIPS 2021) (Vol. 6, pp. 4409-4420). (Advances in Neural Information Processing Systems; Vol. 34), (NeurIPS Proceedings). Neural Information Processing Systems Foundation.

Yao, Yu ; Liu, Tongliang ; Gong, Mingming et al. / Instance-dependent Label-noise Learning under a Structural Causal Model. 35th Conference on Neural Information Processing Systems (NeurIPS 2021). editor / Marc'Aurelio Ranzato ; Alina Beygelzimer ; Yann Dauphin ; Percy S. Liang ; Jenn Wortman Vaughan. Vol. 6 Neural Information Processing Systems Foundation, 2021. pp. 4409-4420 (Advances in Neural Information Processing Systems). (NeurIPS Proceedings).

@inproceedings{3cececb109c648e4be9ebeb5bd6044b6,

title = "Instance-dependent Label-noise Learning under a Structural Causal Model",

abstract = "Label noise generally degenerates the performance of deep learning algorithms because deep neural networks easily overfit label errors. Let X and Y denote the instance and clean label, respectively. When Y is a cause of X, according to which many datasets have been constructed, e.g., SVHN and CIFAR, the distributions of P(X) and P(Y |X) are generally entangled. This means that the unsupervised instances are helpful to learn the classifier and thus reduce the side effect of label noise. However, it remains elusive on how to exploit the causal information to handle the label-noise problem. We propose to model and make use of the causal process in order to correct the label-noise effect. Empirically, the proposed method outperforms all state-of-the-art methods on both synthetic and real-world label-noise datasets.",

author = "Yu Yao and Tongliang Liu and Mingming Gong and Bo Han and Gang Niu and Kun Zhang",

note = "Funding Information: TL was partially supported by Australian Research Council Projects DP-180103424, DE-190101473, and IC-190100031. GM was supported by Australian Research Council Project DE210101624. BH was supported by supported by the RGC Early Career Scheme No. 22200720 and NSFC Young Scientists Fund No. 62006202. GN was supported by JST AIP Acceleration Research Grant Number JPMJCR20U3, Japan. KZ was supported in part by the National Institutes of Health (NIH) under Contract R01HL159805, by the United States Air Force under Contract No. FA8650-17-C7715, by the NSF-Convergence Accelerator Track-D award #2134901, and by a grant from Apple. The NIH or NSF is not responsible for the views reported in this article. The authors thank the reviewers and the meta-reviewer for their helpful and constructive comments on this work. Publisher Copyright: {\textcopyright} 2021 Neural information processing systems foundation. All rights reserved.; 35th Conference on Neural Information Processing Systems, NeurIPS 2021 ; Conference date: 06-12-2021 Through 14-12-2021",

year = "2021",

month = dec,

day = "6",

language = "English",

isbn = "9781713845393",

volume = "6",

series = "Advances in Neural Information Processing Systems",

publisher = "Neural Information Processing Systems Foundation",

pages = "4409--4420",

editor = "Marc'Aurelio Ranzato and Alina Beygelzimer and Yann Dauphin and Liang, {Percy S.} and {Wortman Vaughan}, Jenn",

booktitle = "35th Conference on Neural Information Processing Systems (NeurIPS 2021)",

url = "https://nips.cc/Conferences/2021, https://neurips.cc/Conferences/2021, https://papers.nips.cc/paper_files/paper/2021, https://proceedings.neurips.cc/paper/2021",

}

Yao, Y, Liu, T, Gong, M, Han, B, Niu, G & Zhang, K 2021, Instance-dependent Label-noise Learning under a Structural Causal Model. in MA Ranzato, A Beygelzimer, Y Dauphin, PS Liang & J Wortman Vaughan (eds), 35th Conference on Neural Information Processing Systems (NeurIPS 2021). vol. 6, Advances in Neural Information Processing Systems, vol. 34, NeurIPS Proceedings, Neural Information Processing Systems Foundation, pp. 4409-4420, 35th Conference on Neural Information Processing Systems, NeurIPS 2021, 6/12/21.

Instance-dependent Label-noise Learning under a Structural Causal Model. / Yao, Yu; Liu, Tongliang; Gong, Mingming et al.
35th Conference on Neural Information Processing Systems (NeurIPS 2021). ed. / Marc'Aurelio Ranzato; Alina Beygelzimer; Yann Dauphin; Percy S. Liang; Jenn Wortman Vaughan. Vol. 6 Neural Information Processing Systems Foundation, 2021. p. 4409-4420 (Advances in Neural Information Processing Systems; Vol. 34), (NeurIPS Proceedings).

Research output: Chapter in book/report/conference proceeding › Conference proceeding › peer-review

TY - GEN

T1 - Instance-dependent Label-noise Learning under a Structural Causal Model

AU - Yao, Yu

AU - Liu, Tongliang

AU - Gong, Mingming

AU - Han, Bo

AU - Niu, Gang

AU - Zhang, Kun

N1 - Funding Information: TL was partially supported by Australian Research Council Projects DP-180103424, DE-190101473, and IC-190100031. GM was supported by Australian Research Council Project DE210101624. BH was supported by supported by the RGC Early Career Scheme No. 22200720 and NSFC Young Scientists Fund No. 62006202. GN was supported by JST AIP Acceleration Research Grant Number JPMJCR20U3, Japan. KZ was supported in part by the National Institutes of Health (NIH) under Contract R01HL159805, by the United States Air Force under Contract No. FA8650-17-C7715, by the NSF-Convergence Accelerator Track-D award #2134901, and by a grant from Apple. The NIH or NSF is not responsible for the views reported in this article. The authors thank the reviewers and the meta-reviewer for their helpful and constructive comments on this work. Publisher Copyright: © 2021 Neural information processing systems foundation. All rights reserved.

PY - 2021/12/6

Y1 - 2021/12/6

N2 - Label noise generally degenerates the performance of deep learning algorithms because deep neural networks easily overfit label errors. Let X and Y denote the instance and clean label, respectively. When Y is a cause of X, according to which many datasets have been constructed, e.g., SVHN and CIFAR, the distributions of P(X) and P(Y |X) are generally entangled. This means that the unsupervised instances are helpful to learn the classifier and thus reduce the side effect of label noise. However, it remains elusive on how to exploit the causal information to handle the label-noise problem. We propose to model and make use of the causal process in order to correct the label-noise effect. Empirically, the proposed method outperforms all state-of-the-art methods on both synthetic and real-world label-noise datasets.

AB - Label noise generally degenerates the performance of deep learning algorithms because deep neural networks easily overfit label errors. Let X and Y denote the instance and clean label, respectively. When Y is a cause of X, according to which many datasets have been constructed, e.g., SVHN and CIFAR, the distributions of P(X) and P(Y |X) are generally entangled. This means that the unsupervised instances are helpful to learn the classifier and thus reduce the side effect of label noise. However, it remains elusive on how to exploit the causal information to handle the label-noise problem. We propose to model and make use of the causal process in order to correct the label-noise effect. Empirically, the proposed method outperforms all state-of-the-art methods on both synthetic and real-world label-noise datasets.

UR - https://proceedings.neurips.cc/paper/2021/file/23451391cd1399019fa0421129066bc6-Paper.pdf

UR - https://proceedings.neurips.cc/paper/2021/hash/23451391cd1399019fa0421129066bc6-Abstract.html

UR - http://www.scopus.com/inward/record.url?scp=85131763047&partnerID=8YFLogxK

M3 - Conference proceeding

AN - SCOPUS:85131763047

SN - 9781713845393

VL - 6

T3 - Advances in Neural Information Processing Systems

SP - 4409

EP - 4420

BT - 35th Conference on Neural Information Processing Systems (NeurIPS 2021)

A2 - Ranzato, Marc'Aurelio

A2 - Beygelzimer, Alina

A2 - Dauphin, Yann

A2 - Liang, Percy S.

A2 - Wortman Vaughan, Jenn

PB - Neural Information Processing Systems Foundation

T2 - 35th Conference on Neural Information Processing Systems, NeurIPS 2021

Y2 - 6 December 2021 through 14 December 2021

ER -

Yao Y, Liu T, Gong M, Han B, Niu G, Zhang K. Instance-dependent Label-noise Learning under a Structural Causal Model. In Ranzato MA, Beygelzimer A, Dauphin Y, Liang PS, Wortman Vaughan J, editors, 35th Conference on Neural Information Processing Systems (NeurIPS 2021). Vol. 6. Neural Information Processing Systems Foundation. 2021. p. 4409-4420. (Advances in Neural Information Processing Systems). (NeurIPS Proceedings).

Instance-dependent Label-noise Learning under a Structural Causal Model

Abstract

Publication series

Conference

Scopus Subject Areas

Other files and links

Fingerprint

Cite this