MG-WFBP: Merging gradients wisely for efficient communication in distributed deep learning

Mathematics

• Wait-free 100%
• Merging 97%
• Back Propagation 94%
• Training 88%
• Communication 76%
• Learning 75%
• Gradient 63%
• Neural Networks 39%
• Pipelining 35%
• Scaling 35%
• Training Algorithm 34%
• Network Communication 33%
• Distributed Algorithms 27%
• Stochastic Gradient 27%
• Scalability 26%
• Gradient Descent 25%
• Performance 24%
• Overlap 21%
• Experimental Results 20%
• Optimal Solution 17%
• Verify 16%
• Trace 16%
• Minimise 16%
• Experiment 15%
• Optimization Problem 14%
• Simulation 13%

Engineering & Materials Science

• Merging 81%
• Backpropagation 77%
• Deep learning 74%
• Deep neural networks 53%
• Graphics processing unit 51%
• Communication 41%
• Telecommunication networks 24%
• Scalability 22%
• Experiments 9%