Robust Weight Perturbation for Adversarial Training

Chaojian Yu, Bo Han, Mingming Gong, Li Shen, Shiming Ge, Du Bo, Tongliang Liu*

*Corresponding author for this work

Research output: Chapter in book/report/conference proceedingConference contributionpeer-review

Abstract

Overfitting widely exists in adversarial robust training of deep networks. An effective remedy is adversarial weight perturbation, which injects the worst-case weight perturbation during network training by maximizing the classification loss on adversarial examples. Adversarial weight perturbation helps reduce the robust generalization gap; however, it also undermines the robustness improvement. A criterion that regulates the weight perturbation is therefore crucial for adversarial training. In this paper, we propose such a criterion, namely Loss Stationary Condition (LSC) for constrained perturbation. With LSC, we find that it is essential to conduct weight perturbation on adversarial data with small classification loss to eliminate robust overfitting. Weight perturbation on adversarial data with large classification loss is not necessary and may even lead to poor robustness. Based on these observations, we propose a robust perturbation strategy to constrain the extent of weight perturbation. The perturbation strategy prevents deep networks from overfitting while avoiding the side effect of excessive weight perturbation, significantly improving the robustness of adversarial training. Extensive experiments demonstrate the superiority of the proposed method over the state-of-the-art adversarial training methods.
Original languageEnglish
Title of host publicationProceedings of 31th International Joint Conference on Artificial Intelligence, IJCAI 2022
EditorsLuc De Raedt
PublisherInternational Joint Conferences on Artificial Intelligence
Pages3688-3694
Number of pages7
ISBN (Electronic)9781956792003
DOIs
Publication statusPublished - 23 Jul 2022
Event31th International Joint Conference on Artificial Intelligence, IJCAI 2022 - Messe Wien, Vienna, Austria
Duration: 23 Jul 202229 Jul 2022
https://ijcai-22.org/
https://www.ijcai.org/proceedings/2022/

Publication series

NameIJCAI International Joint Conference on Artificial Intelligence
ISSN (Print)1045-0823

Conference

Conference31th International Joint Conference on Artificial Intelligence, IJCAI 2022
Country/TerritoryAustria
CityMesse Wien, Vienna
Period23/07/2229/07/22
Internet address

User-Defined Keywords

  • Machine Learning
  • Adversarial Machine Learning

Fingerprint

Dive into the research topics of 'Robust Weight Perturbation for Adversarial Training'. Together they form a unique fingerprint.

Cite this