Sparse Autoencoders for Interpretability: TopK, JumpReLU, Matryoshka, and Scaling

4 selectedDifficulty 5-64 unseenView topic

IntermediateNew

0 answered

4 intermediateAdapts to your performance

Question 1 of 4

120sintermediate (5/10)spot the error

A student says: "The unit-norm constraint ∥ W_{dec} [:, j] ∥_{2} = 1 on SAE decoder columns is purely cosmetic, used so feature directions can be plotted on a unit sphere; you can train a perfectly good L1 SAE without it." What is wrong with this claim?