Preconditioned Optimizers: Shampoo, K-FAC, and Natural Gradient

2 selectedDifficulty 3-32 unseenView topic

FoundationNew

0 answered

2 foundationAdapts to your performance

Question 1 of 2

120sfoundation (3/10)conceptual

Why can Adam help when different parameters have gradients on very different scales?