Beta. Content is under active construction and has not been peer-reviewed. Report errors on
GitHub
.
Disclaimer
Theorem
Path
Curriculum
Paths
Demos
Diagnostic
Search
Quiz Hub
/
Lazy vs Feature Learning
Lazy vs Feature Learning
2 questions
Difficulty 7-9
View topic
Advanced
0 / 2
2 advanced
Adapts to your performance
1 / 2
advanced (7/10)
conceptual
In the Neural Tangent Kernel (NTK) regime (infinite width, standard parameterization), the network behaves like a linear model. Why does this fail to explain the success of deep learning on structured data?
Hide and think first
A.
The kernel stays constant during training, so no data-dependent features are learned
B.
The NTK theory assumes zero activation functions, so it only models linear networks with no nonlinearity at all
C.
The NTK regime requires infinite width, so it is computationally infeasible and irrelevant to real deep networks
D.
The NTK kernel cannot approximate complex functions, so it fundamentally lacks expressivity needed for real data
Submit Answer