Beta. Content is under active construction and has not been peer-reviewed. Report errors on
GitHub
.
Disclaimer
Theorem
Path
Curriculum
Paths
Demos
Diagnostic
Search
Quiz Hub
/
Neural Tangent Kernel
Neural Tangent Kernel
3 questions
Difficulty 7-8
View topic
Advanced
0 / 3
3 advanced
Adapts to your performance
1 / 3
advanced (7/10)
conceptual
In the neural tangent kernel (NTK) regime, a neural network behaves like a linear model in function space. When does this regime apply?
Hide and think first
A.
When the network is trained with a sufficiently small learning rate on any architecture, regardless of the network width or initialization scheme
B.
At infinite width with standard initialization scaling, so the kernel
K
(
x
,
x
′
)
=
⟨
∇
θ
f
(
x
)
,
∇
θ
f
(
x
′
)⟩
stays approximately constant
C.
When the training data is linearly separable in the input feature space, allowing the network to find a solution without feature learning
D.
When the network uses ReLU activations specifically, as the NTK convergence result is restricted to piecewise-linear activation functions
Submit Answer