Beta. Content is under active construction and has not been peer-reviewed. Report errors on
GitHub
.
Disclaimer
Theorem
Path
Curriculum
Paths
Demos
Diagnostic
Search
Quiz Hub
/
Implicit Bias and Modern Generalization
Implicit Bias and Modern Generalization
3 questions
Difficulty 7-10
View topic
Advanced
0 / 3
3 advanced
Adapts to your performance
1 / 3
advanced (7/10)
conceptual
In double descent, test error spikes at the interpolation threshold (where model capacity equals training set size) and then decreases in the overparameterized regime. What mechanism causes test error to decrease again after the spike?
Hide and think first
A.
Double descent is an artifact of specific architectures, and with careful hyperparameter tuning it disappears entirely
B.
Many interpolating solutions exist past the threshold, and implicit bias selects a smooth one
C.
The extra parameters act as an implicit form of regularization that effectively shrinks the model capacity
D.
Once overparameterized the model memorizes every training point, and memorization is equivalent to generalization for huge models
Submit Answer