On the Overlooked Structure of Stochastic Gradients

Keyphrases

• Stochastic Gradient 100%
• Heavy Tails 62%
• Deep Learning 50%
• Gradient Noise 50%
• Power Law 37%
• Statistical Test 37%
• Deep Neural Network 25%
• Stochastic Optimization 12%
• Mathematical Structure 12%
• Covariance 12%
• Work Challenges 12%
• Mini-batch Training 12%
• Covariance Spectrum 12%
• Anisotropic Structure 12%
• Tail Property 12%

Mathematics

• Stochastics 100%
• Heavy Tail 55%
• Deep Learning Method 44%
• Power Law 33%
• Statistical Test 33%
• Covariance 22%
• Deep Neural Network 22%
• Mathematical Structure 11%

Chemical Engineering

• Deep Learning Method 100%
• Deep Neural Network 50%