Exploring Model Dynamics for Accumulative Poisoning Discovery

Jianing Zhu, Xiawei Guo, Jiangchao Yao*, Chao Du, Li He, Shuo Yuan, Tongliang Liu, Liang Wang, Bo Han*

*Corresponding author for this work

Research output: Chapter in book/report/conference proceedingConference proceedingpeer-review


Adversarial poisoning attacks pose huge threats to various machine learning applications. Especially, the recent accumulative poisoning attacks show that it is possible to achieve irreparable harm on models via a sequence of imperceptible attacks followed by a trigger batch. Due to the limited data-level discrepancy in real-time data streaming, current defensive methods are indiscriminate in handling the poison and clean samples. In this paper, we dive into the perspective of model dynamics and propose a novel information measure, namely, Memorization Discrepancy, to explore the defense via the model-level information. By implicitly transferring the changes in the data manipulation to that in the model outputs, Memorization Discrepancy can discover the imperceptible poison samples based on their distinct dynamics from the clean samples. We thoroughly explore its properties and propose Discrepancy-aware Sample Correction (DSC) to defend against accumulative poisoning attacks. Extensive experiments comprehensively characterized Memorization Discrepancy and verified its effectiveness. The code is publicly available at: https://github.com/tmlr-group/Memorization-Discrepancy.
Original languageEnglish
Title of host publicationProceedings of the 40th International Conference on Machine Learning, ICML 2023
EditorsAndreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, Jonathan Scarlett
PublisherML Research Press
Number of pages22
Publication statusPublished - Jul 2023
Event40th International Conference on Machine Learning, ICML 2023 - Honolulu, United States
Duration: 23 Jul 202329 Jul 2023

Publication series

NameProceedings of Machine Learning Research
ISSN (Print)2640-3498


Conference40th International Conference on Machine Learning, ICML 2023
Country/TerritoryUnited States
Internet address

Scopus Subject Areas

  • Software
  • Artificial Intelligence
  • Control and Systems Engineering
  • Statistics and Probability


Dive into the research topics of 'Exploring Model Dynamics for Accumulative Poisoning Discovery'. Together they form a unique fingerprint.

Cite this