Distributions Atlas

Why This Matters

The named distributions are not a list. They are a graph. A Bernoulli trial summed $n$ times is a Binomial; a Binomial with $n$ large and mean held fixed is a Poisson; the time between Poisson events is an Exponential; the sum of $k$ independent Exponentials with the same rate is a Gamma; a Gamma with shape $k/2$ and rate $1/2$ is a Chi-squared with $k$ degrees of freedom; a standard Normal divided by a square-rooted Chi-squared is a Student-t; the ratio of two scaled Chi-squareds is an F.

Knowing these edges turns a long memorization task into a short one. The sampling distribution of the standardized sample mean is Student-t because the numerator is Normal, the denominator squared is Chi-squared, and they are independent. The Pearson Chi-squared statistic has its name because under the null hypothesis it converges to a sum of squared Normals. The F statistic in analysis of variance is a ratio of two variance estimates, and each variance estimate is a scaled Chi-squared. Each new test follows from the same construction: identify the building blocks, identify the transformation, read off the limiting distribution.

This page is the atlas. It states each connection precisely once and links to the per-distribution page that proves it.

The Atlas

Discrete families

Continuous families

The variance entries with degree-of-freedom restrictions ($\nu>2$, $d_2>2$, $\alpha>1$) reflect the fact that those distributions have polynomially decaying tails, so low-order moments only exist past a threshold.

The Connection Graph

Each row of this table is a named transformation. The "Direction" column reads from the building block to the result.

Theorem

Sum of i.i.d. Exponentials is Gamma

Statement

If $Y_1,\dots,Y_k$ are independent $\operatorname{Exp}(\lambda)$ random variables, then $T_k = Y_1 + \cdots + Y_k \sim \operatorname{Gamma}(k,\lambda),$ with density $f_{T_k}(t) = \frac{\lambda^k t^{k-1} e^{-\lambda t}}{(k-1)!}, \qquad t\ge 0.$

Intuition

A Poisson process with rate $\lambda$ has Exponential inter-arrival times. The time of the $k$-th event is the sum of the first $k$ gaps. Each gap contributes a factor of $\lambda$ to the density and one power of $t$ to the polynomial term; the factorial is the volume of the ordered-arrival simplex.

Proof Sketch

By induction or by MGF. The MGF of $\operatorname{Exp}(\lambda)$ is $\lambda/(\lambda-s)$ for $s<\lambda$. By independence the MGF of $T_k$ is $[\lambda/(\lambda-s)]^k$, which is the MGF of $\operatorname{Gamma}(k,\lambda)$. MGF uniqueness identifies the distribution.

Why It Matters

Every Chi-squared random variable is a sum of squared Normals, hence a Gamma with half-integer shape. Every F is a ratio of two Chi-squareds, hence built from Gammas. The Gamma family is the load-bearing wall behind half of the classical sampling distributions.

Failure Mode

The result requires independence and a common rate. Sums of Exponentials with different rates do not give a Gamma; they give a hypoexponential distribution, which has a different density.

report a correction →

Theorem

Ratio of Normal and Root Chi-squared is Student-t

Statement

Let $Z\sim\mathcal{N}(0,1)$ and $V\sim\chi^2_k$ be independent. Then $T = \frac{Z}{\sqrt{V/k}} \sim t_k,$ with density $f_T(t) = \frac{\Gamma((k+1)/2)}{\sqrt{k\pi}\,\Gamma(k/2)}\left(1+\frac{t^2}{k}\right)^{-(k+1)/2}.$

Intuition

The numerator is the unit-variance source of randomness. The denominator is a scale estimate, normalized so that $V/k\to 1$ as $k\to\infty$. Dividing $Z$ by a noisy estimate of the unit scale inflates the tails by a polynomial amount; the heavier the noise (smaller $k$), the heavier the tails.

Proof Sketch

Joint density of $(Z,V)$ factors by independence. Change variables to $T = Z/\sqrt{V/k}$ and $V$, integrate out $V$. The integrand is a Gamma in $V$ once collected, so the integral evaluates by the Gamma normalizing constant.

Why It Matters

The sample mean $\bar X$ of an i.i.d. Normal sample has numerator $(\bar X-\mu)/(\sigma/\sqrt n)$ that is standard Normal, and denominator $S/\sigma$ with $S^2$ proportional to a Chi-squared. The standardized statistic $(\bar X-\mu)/(S/\sqrt n)$ is therefore exactly $t_{n-1}$. This is the one-sample t-test in one line.

Failure Mode

Independence of $Z$ and $V$ is essential. In the t-test the relevant $Z$ is built from $\bar X-\mu$ and the relevant $V$ is built from $\sum(X_i-\bar X)^2$; their independence is a special fact about Normal samples and Cochran's theorem, not a generic phenomenon.

report a correction →

Conjugate Priors

A prior on $\theta$ is conjugate to a likelihood $p(x|\theta)$ when the posterior is in the same parametric family as the prior. For the named families the conjugate pairs are short:

The Bernoulli-Beta and Poisson-Gamma pairs are derived in their respective pages; the Normal pairs are derived in Bayesian estimation.

Three Recurring Tricks

Trick 1: Sum identifies via MGF

If $X$ and $Y$ are independent and you want the law of $X+Y$, compute the MGF of $X+Y = M_X(s)M_Y(s)$ and match it to a known MGF. This is how sum-of-Exponentials-is-Gamma is proved, how sum-of-Normals-is-Normal is proved, and how Binomial-additivity is proved. The MGF table in moment generating functions is the lookup index.

Trick 2: Limit identifies via characteristic-function convergence

If you want the limiting law of a sequence $X_n$, compute the characteristic function $\varphi_{X_n}(t)$ and take the limit. This is how Binomial-to-Poisson is proved (Poisson's theorem), how the central limit theorem is proved, and how $t_\nu\to\mathcal{N}(0,1)$ as $\nu\to\infty$ is proved.

Trick 3: Transformation identifies via change of variables

If $Y = g(X)$ for a smooth invertible $g$, then $f_Y(y) = f_X(g^{-1}(y))|(g^{-1})'(y)|$. This is how Lognormal is derived from Normal, how the Pareto-log-Exponential connection is verified, how the Student-t density is computed from the joint $(Z,V)$ density, and how the F density is computed from the joint $(V_1,V_2)$ density.

Two Common Confusions

Watch Out

Rate versus scale parameterization

Exponential and Gamma both have two conventions. Rate uses $\lambda$ with density $\lambda e^{-\lambda x}$ and mean $1/\lambda$. Scale uses $\theta = 1/\lambda$ with density $e^{-x/\theta}/\theta$ and mean $\theta$. SciPy and many engineering texts default to scale; mathematical-statistics texts default to rate. Always check which parameterization a software library uses before plugging in. The pages in this atlas use rate by default and note where scale is more natural.

Watch Out

Chi-squared as a sampling distribution versus a concentration class

The Chi-squared distribution in this atlas is the exact sampling distribution of $\sum Z_i^2$. The phrase "Chi-squared concentration" elsewhere on the site refers to the Laurent-Massart sub-Gamma tail bounds for Chi-squared random variables, which is a finite-sample inequality, not a description of the law. See chi-squared concentration for the bound; see chi-squared distribution and tests for the law and its uses.

How to Use This Atlas

Pick a target distribution; read down the connection-graph table; follow the edges to the building blocks. Each distribution page derives its connections from this atlas and links back here.

• Want to know why the sample variance has a Chi-squared distribution? Read chi-squared distribution and tests.
• Want to derive the t-statistic from first principles? Read Student-t distribution and t-test.
• Want the F statistic for ANOVA? Read F distribution and ANOVA.
• Want a Bayesian update for a Bernoulli sequence? Read beta distribution.
• Want the time-to-failure model for a Poisson process? Read exponential distribution and gamma distribution.

Exercises

ExerciseCore

Problem

Let $X_1,\dots,X_n$ be i.i.d. $\operatorname{Exp}(\lambda)$ with $n=10$ and $\lambda = 2$. Identify the distribution of $S = \sum_{i=1}^{10} X_i$ and compute $\mathbb{E}[S]$ and $\operatorname{Var}(S)$.

Hint

Reveal Solution

ExerciseCore

Problem

A Poisson process with rate $\lambda = 3$ events per minute is observed. Let $Y$ be the time, in minutes, of the third event. Identify the distribution of $Y$ and compute $\mathbb{P}(Y > 2)$ in terms of an incomplete Gamma function.

Hint

Reveal Solution

ExerciseAdvanced

Problem

Let $Z\sim\mathcal{N}(0,1)$ and $V\sim\chi^2_k$ be independent, and let $T = Z/\sqrt{V/k}$. Show that as $k\to\infty$, the distribution of $T$ converges to $\mathcal{N}(0,1)$.

Hint

Reveal Solution

References

Canonical:

• Casella and Berger, Statistical Inference (2002), Chapters 3 and 5.
• Bickel and Doksum, Mathematical Statistics, Volume I (2015), Chapter 1.
• Johnson, Kotz, and Balakrishnan, Continuous Univariate Distributions, Volumes 1 and 2 (1994 and 1995).
• Johnson, Kemp, and Kotz, Univariate Discrete Distributions (2005).

Probability foundations:

• Blitzstein and Hwang, Introduction to Probability (2019), Chapters 3 through 8.
• Durrett, Probability: Theory and Examples (2019), Chapters 2 and 3.
• Grimmett and Stirzaker, Probability and Random Processes (2020), Chapters 3 through 6.

Bayesian framing:

• Gelman et al., Bayesian Data Analysis (2013), Chapter 2.
• Robert, The Bayesian Choice (2007), Chapter 3.