Beta. Content is under active construction and has not been peer-reviewed. Report errors on GitHub.Disclaimer

Scaling Compute-Optimal Training

2 questionsDifficulty 5-5View topic
Intermediate
0 / 2
2 intermediateAdapts to your performance
1 / 2
intermediate (5/10)conceptual
The Chinchilla scaling law (Hoffmann et al., 2022) found that compute-optimal training allocates a fixed ratio between model parameters and training tokens . What is the approximate relationship?