Lesson 2 of 3 · About 5 minutes
Constrain the fit, then check it
A more flexible model is not always a better predictor. Try a simpler fit, or penalize large coefficients with regularization. Either change needs a validation check.
Try it
Move the penalty strength, lambda, in both directions. Which setting gives the lowest validation error? Does a stronger penalty help these particular examples?
Compare settings using validation error. These fixed datasets show how the fit changes; they do not measure bias or variance across repeated training samples.
Degree 8 stays fixed. Ridge adds λ times the sum of squared coefficients to the sum of squared residuals. The basis is 1, x, …, x⁸ with x in [0, 1]; the intercept is not penalized. Stronger penalties need not improve validation error.
This experiment keeps the same degree-eight features and training examples. Ridge penalizes squared coefficients, leaving the intercept unpenalized. Here the weakest offered penalty gives the best validation result: stronger regularization is not the right fix for these data. The useful strength depends on your data and feature scaling. Test the idea instead of assuming it will help.