MG-WFBP: Merging gradients wisely for efficient communication in distributed deep learning

Keyphrases

• Merged-gradient 100%
• Distributed Deep Learning 100%
• Efficient Communication 100%
• Backpropagation 60%
• Wait-free 60%
• Deep Neural Network 40%
• Distributed Training 40%
• Training Performance 20%
• Popular 20%
• Optimization Problem 20%
• Computational Power 20%
• Training Time 20%
• Caffe 20%
• System Scalability 20%
• Cluster Computing 20%
• Network Communication 20%
• Communication Time 20%
• Communication Tasks 20%
• Power Network 20%
• Small Data 20%
• Multi-GPU 20%
• Stochastic Gradient Descent 20%
• Short Communication 20%
• Trace-driven Simulation 20%
• Training Process 20%
• Training Algorithm 20%
• Scalable Performance 20%
• PyTorch 20%
• Potential Scaling 20%

Computer Science

• Graphics Processing Unit 100%
• Deep Learning Method 100%
• Deep Neural Network 100%
• Experimental Result 50%
• Communication Network 50%
• Optimization Problem 50%
• Computational Power 50%
• Computer Cluster 50%
• Training Process 50%
• training algorithm 50%
• Pipelining 50%
• Stochastic Gradient Descent 50%