GossipFL: A Decentralized Federated Learning Framework With Sparsified and Adaptive Communication

Zhenheng Tang, Shaohuai Shi*, Bo Li, Xiaowen Chu*

*Corresponding author for this work

Research output: Contribution to journalJournal articlepeer-review

17 Citations (Scopus)

Abstract

Recently, federated learning (FL) techniques have enabled multiple users to train machine learning models collaboratively without data sharing. However, existing FL algorithms suffer from the communication bottleneck due to network bandwidth pressure and/or low bandwidth utilization of the participating clients in both centralized and decentralized architectures. To deal with the communication problem while preserving the convergence performance, we introduce a communication-efficient decentralized FL framework GossipFL. In GossipFL, we 1) design a novel sparsification algorithm to enable that each client only needs to communicate with one peer with a highly sparsified model, and 2) propose a new and novel gossip matrix generation algorithm that can better utilize the bandwidth resources while preserving the convergence property. We also theoretically prove that GossipFL has convergence guarantees. We conduct experiments with three convolutional neural networks on two datasets (IID and non-IID) under two distributed environments (14 clients and 100 clients) to verify the effectiveness of GossipFL. Experimental results show that GossipFL takes less communication traffic for 38.5% and less communication time for 49.8% than state-of-the-art solutions while achieving comparative model accuracy.
Original languageEnglish
Pages (from-to)909-922
Number of pages14
JournalIEEE Transactions on Parallel and Distributed Systems
Volume34
Issue number3
Early online date21 Dec 2022
DOIs
Publication statusPublished - 1 Mar 2023

Scopus Subject Areas

  • Signal Processing
  • Hardware and Architecture
  • Computational Theory and Mathematics

User-Defined Keywords

  • Deep learning
  • communication efficiency
  • federated learning

Fingerprint

Dive into the research topics of 'GossipFL: A Decentralized Federated Learning Framework With Sparsified and Adaptive Communication'. Together they form a unique fingerprint.

Cite this