Learning Track

ML Research Interview Prep

The theory you'll actually be asked at top ML labs. Five phases, from foundations assumed cold through the 2026 research frontier. Every topic linked to its exact theorem statements, proofs, and failure modes. No hand-waving.

Find your gaps first Take the diagnostic

1/5

Phase 1: Foundations assumed cold

Often asked as warm-ups. Weak answers here usually end the round early, regardless of how strong later material is.

Expectation, variance, momentsFirst-round warm-up question Probability distributionsKnow the Gaussian, Bernoulli, Exponential by heart Bayesian estimationPosteriors, priors, MAP. Come up in generative modeling rounds Eigenvalues & SVDPCA, spectral methods, conditioning Gradient descentBatch, stochastic, mini-batch: convergence tradeoffs

2/5

Phase 2: Learning theory classics

The "why do models generalize" thread. Expect precise statements of VC, Rademacher, PAC, and uniform convergence, not verbal summaries.

Bias-variance tradeoffState the decomposition exactly ERM frameworkWhy does minimizing training loss work?VC dimensionKnow the shattering argument Rademacher complexityData-dependent generalization bound PAC learningPrecise notion of learnability Concentration inequalitiesHoeffding, Bernstein, McDiarmid Uniform convergenceThe bridge from concentration to generalization

3/5

Phase 3: Optimization and training

Practical questions with theoretical answers. Can you reason about convergence rates and failure modes?

Convex optimizationStrong convexity, smoothness, convergence rates SGD convergenceVariance, step size schedules, strong convexity vs non-convex AdamWhen does it beat SGD? When does it fail?Batch normalizationWhat does it actually do? Internal covariate shift is not the answer.RegularizationL1 vs L2, early stopping, dropout as regularizer BackpropagationDerive from scratch on a 2-layer MLP

4/5

Phase 4: Modern deep learning

What research teams care about right now: transformer internals, generalization puzzles, and scaling behavior.

Transformer architectureDraw it; explain every block; why residual stream?AttentionQK^T/sqrt(d): why the sqrt? What about softmax temperature?Double descentClassical bias-variance is incomplete Implicit biasWhy does SGD find generalizing solutions?Neural tangent kernelInfinite-width limit; connection to kernels Scaling lawsKaplan vs Chinchilla; compute-optimal training

5/5

Phase 5: Research frontier

Asked at alignment, interpretability, and frontier-lab research roles. Know the questions; know what's open.

RLHFBradley-Terry, PPO vs DPO, reward hacking GrokkingDelayed generalization; what's the mechanism?Induction headsIn-context learning circuit; why two layers?Mechanistic interpretabilityWhat are the open problems?Sparse autoencodersSuperposition hypothesis; dictionary learning Constitutional AIRLAIF; self-critique loops

Representative questions

A sample of what gets asked. Each links to the page that answers it.

Strategy

Breadth before depth. Know every Phase 1-3 topic well enough to state the theorem and sketch the proof. Interviewers follow up on what you state.
Pick one frontier area. You cannot cover everything in Phase 5. Pick scaling, interpretability, or alignment and go deep.
Expect failure-mode questions. "When does X fail?" is the most common follow-up. Every page here has a FailureMode section for a reason.
Start with the gap finder. Pick the topic you want to reach. It walks prerequisites backward and gives you a reading list.