Exploiting simultaneous communications to accelerate data parallel distributed deep learning

Keyphrases

• Distributed Deep Learning 100%
• Data Parallelism 100%
• Tensor Fusion 100%
• Simultaneous Communication 100%
• Communication Efficiency 40%
• Communication Tasks 40%
• Stochastic Gradient Descent 40%
• Allreduce 40%
• Optimization Problem 20%
• Deep Learning Model 20%
• Design Space 20%
• Distributed Systems 20%
• Communication Cost 20%
• Multi-GPU 20%
• Iteration Times 20%
• Real-world Experiment 20%
• Computing Task 20%
• Training Algorithm 20%
• Ethernet 20%
• Distributed Training 20%
• Fusion Technique 20%
• PyTorch 20%
• Deep Learning Training 20%
• 10Gbps 20%
• Horovod 20%
• Optimal Scheduling 20%
• Scheduling Solutions 20%
• Fusion Solution 20%

Computer Science

• Graphics Processing Unit 100%
• Deep Learning Method 100%
• Gradient Descent 100%
• Experimental Result 50%
• Deep Learning Model 50%
• Optimization Problem 50%
• Distributed System 50%
• Communication Cost 50%
• Iteration Time 50%
• Deep Neural Network 50%
• training algorithm 50%
• Fusion Technique 50%
• Data Parallelism 50%