Beta. Content is under active construction and has not been peer-reviewed. Report errors on
GitHub
.
Disclaimer
Theorem
Path
Curriculum
Paths
Demos
Diagnostic
Search
Quiz Hub
/
Skip Connections and ResNets
Skip Connections and ResNets
1 questions
Difficulty 5-5
View topic
Intermediate
0 / 1
1 intermediate
Adapts to your performance
1 / 1
intermediate (5/10)
conceptual
In a residual network, the output of a block is
y
=
F
(
x
)
+
x
where
F
(
x
)
is the learned residual function. Why is learning the residual
F
(
x
)
=
y
−
x
easier than learning the full mapping
H
(
x
)
=
y
directly?
Hide and think first
A.
If the optimal mapping is close to identity, the residual
F
(
x
)
≈
0
is easier to learn than
H
(
x
)
≈
x
, and the skip connection provides a gradient highway
B.
The skip connection doubles the feature dimensionality available to downstream layers by concatenating the input with the learned features
C.
Residual learning reduces the total number of trainable parameters needed to achieve equivalent representational capacity in the network
D.
The skip connection transforms the non-convex loss landscape into a convex optimization problem, guaranteeing convergence to the global minimum
Submit Answer