Abstract
Recent years have witnessed a burgeoning interest in federated learning (FL). However, the contexts in which clients engage in sequential learning remain under- explored. Bridging FL and continual learning (CL) gives rise to a challenging practical problem: federated continual learning (FCL). Existing research in FCL primarily focuses on mitigating the catastrophic forgetting issue of continual learning while collaborating with other clients. We argue that forgetting phenomena are not invariably detrimental. In this paper, we consider a more practical and challenging FCL setting characterized by potentially unrelated or even antagonistic data/tasks across different clients. In the FL scenario, statistical heterogeneity and data noise among clients may exhibit spurious correlations which result in biased feature learning. While existing CL strategies focus on the complete utilization of previous knowledge, we found that forgetting biased information was beneficial in our study. Therefore, we propose a new concept accurate forgetting (AF) and develop a novel generative-replay method AF-FCL that selectively utilizes previous knowledge in federated networks. We employ a probabilistic framework based on a normalizing flow model to quantify the credibility of previous knowledge. Comprehensive experiments affirm the superiority of our method over baselines.
Original language | English |
---|---|
Title of host publication | Proceedings of the Twelfth International Conference on Learning Representations, ICLR 2024 |
Publisher | International Conference on Learning Representations |
Pages | 1-19 |
Number of pages | 19 |
Publication status | Published - May 2024 |
Event | 12th International Conference on Learning Representations, ICLR 2024 - Messe Wien Exhibition and Congress Center, Vienna, Austria Duration: 7 May 2024 → 11 May 2024 https://iclr.cc/Conferences/2024 (Conference website) https://iclr.cc/virtual/2024/calendar (Conference schedule ) https://openreview.net/group?id=ICLR.cc/2024/Conference#tab-accept-oral (Conference proceedings) |
Publication series
Name | Proceedings of the International Conference on Learning Representations, ICLR |
---|
Conference
Conference | 12th International Conference on Learning Representations, ICLR 2024 |
---|---|
Country/Territory | Austria |
City | Vienna |
Period | 7/05/24 → 11/05/24 |
Internet address |
|
Scopus Subject Areas
- Language and Linguistics
- Computer Science Applications
- Education
- Linguistics and Language