Unlock: Singular Learning Theory

Singular Learning Theory (SLT), developed by Sumio Watanabe, is the Bayesian asymptotic theory of models whose Fisher information matrix is degenerate at the true parameter. Neural networks, mixture models, and hidden Markov models all fall in this class. The Real Log Canonical Threshold (RLCT) replaces half the parameter count in the Bayes free-energy expansion, and the Local Learning Coefficient (LLC) gives an empirical proxy that the developmental-interpretability community uses to study trained networks.