Beta. Content is under active construction and has not been peer-reviewed. Report errors on GitHub.Disclaimer

Key Theorems

The must-know results across ML theory, concentration, optimization, and RL. Each links to the topic page where it is stated and proved.

Tier 1Hoeffding's Inequalityin concentration inequalities Tier 2Bernstein's Inequalityin concentration inequalities Tier 1McDiarmid's Inequalityin mcdiarmids inequality Tier 1Chernoff Boundin chernoff bounds Tier 1Sauer-Shelah Lemmain vc dimension Tier 1VC Generalization Boundin vc dimension Tier 1Rademacher Generalization Boundin rademacher complexity Tier 1Symmetrization Lemmain symmetrization inequality Tier 2Contraction Lemmain rademacher complexity Tier 1Matrix Bernstein Inequalityin matrix concentration Tier 2Hanson-Wright Inequalityin hanson wright inequality Tier 2Davis-Kahan Sin-Thetain matrix concentration Tier 2Weyl's Inequalityin matrix concentration Tier 1Representer Theoremin kernels and rkhs Tier 1Bellman Optimality Equationin markov decision processes Tier 1Policy Gradient Theoremin policy gradient theorem Tier 2Marchenko-Pastur Lawin random matrix theory overview Tier 2BBP Phase Transitionin random matrix theory overview Tier 1GD Convergence (Smooth Convex)in convex optimization basics Tier 1GD Convergence (Strongly Convex)in convex optimization basics Tier 2K-Means++ Approximationin k means clustering Tier 1Gauss-Markov Theoremin linear regression