Beta. Content is under active construction and has not been peer-reviewed. Report errors on
GitHub
.
Disclaimer
Theorem
Path
Curriculum
Paths
Demos
Diagnostic
Search
Quiz Hub
/
Fine-Tuning and Adaptation
Fine-Tuning and Adaptation
3 questions
Difficulty 5-6
View topic
Intermediate
0 / 3
3 intermediate
Adapts to your performance
1 / 3
intermediate (5/10)
compare
Fine-tuning a pretrained model on a new task can cause catastrophic forgetting of the original capabilities. Which intervention directly addresses the forgetting problem?
Hide and think first
A.
KL-regularization or parameter-elastic anchoring against the pretrained model, explicitly penalizing distance from the original weights
B.
Increasing the learning rate by an order of magnitude to escape local minima near the pretrained weights
C.
Switching the optimizer from Adam to SGD with momentum, which reduces the amount of adaptation per step
D.
Training on a mix of base-task and new-task data simultaneously, which trades off final performance against forgetting risk
Submit Answer