Statistical Estimation

Basu's Theorem

A complete sufficient statistic is independent of every ancillary statistic. This provides the cleanest method for proving independence between statistics without computing joint distributions.

AdvancedTier 3Stable~35 min

Prerequisites

Sufficient Statistics and Exponential Families

Why This Matters

Proving that two statistics are independent usually requires computing their joint distribution and factoring it. This can be painful. Basu's theorem gives a shortcut: if one statistic is complete sufficient and the other is ancillary, they are independent. No joint distribution computation needed.

The classic application: in a normal sample, the sample mean $\bar{X}$ is independent of the sample variance $S^2$ . This fact underpins the derivation of the t-test. Basu's theorem proves it in two lines.

Formal Setup

Definition

Ancillary Statistic

A statistic $A(X_1, \ldots, X_n)$ is ancillary for a parameter $\theta$ if its distribution does not depend on $\theta$ . It carries no information about $\theta$ by itself, but may carry information in combination with other statistics.

Definition

Complete Statistic

A statistic $T(X_1, \ldots, X_n)$ is complete if for every measurable function $g$ :

$\mathbb{E}_\theta[g(T)] = 0 \text{ for all } \theta \implies P_\theta(g(T) = 0) = 1 \text{ for all } \theta$

Completeness means there are no nontrivial unbiased estimators of zero based on $T$ . Informally, $T$ contains no "wasted" information.

Main Theorems

Theorem

Basu's Theorem

Statement

If $T$ is a complete sufficient statistic for $\theta$ and $A$ is ancillary for $\theta$ , then $T$ and $A$ are independent (under every $P_\theta$ ).

Intuition

Sufficiency means that the conditional distribution of the data given $T$ does not depend on $\theta$ . Ancillarity means the marginal distribution of $A$ does not depend on $\theta$ . Completeness forces these two facts to combine into independence: the conditional distribution of $A$ given $T$ must equal the marginal distribution of $A$ .

Proof Sketch

Let $B$ be any measurable set. Define $g(t) = P(A \in B \mid T = t) - P(A \in B)$ . By sufficiency, $P(A \in B \mid T = t)$ does not depend on $\theta$ . By ancillarity, $P(A \in B)$ does not depend on $\theta$ . So $g(t)$ does not depend on $\theta$ , and $\mathbb{E}_\theta[g(T)] = P_\theta(A \in B) - P(A \in B) = 0$ for all $\theta$ (using ancillarity again). By completeness, $g(T) = 0$ a.s., meaning $P(A \in B \mid T) = P(A \in B)$ a.s. This is independence.

Why It Matters

Without this theorem, proving independence of $\bar{X}$ and $S^2$ in normal sampling requires computing the joint density via a change of variables. With Basu's theorem, you only need three facts: (1) $\bar{X}$ is complete sufficient for $\mu$ when $\sigma^2$ is known, (2) $S^2/\sigma^2$ is ancillary for $\mu$ , (3) apply the theorem. This pattern extends to many other settings.

Failure Mode

If the sufficient statistic is not complete, the theorem fails. For example, in a uniform distribution on $[\theta - 1, \theta + 1]$ , the order statistics $(X_{(1)}, X_{(n)})$ are sufficient but not complete. The range $X_{(n)} - X_{(1)}$ is ancillary but not independent of the midrange $(X_{(1)} + X_{(n)})/2$ .

Canonical Examples

Example

Normal sampling: mean and variance independence

Let $X_1, \ldots, X_n \sim N(\mu, \sigma^2)$ with $\sigma^2$ known. The sample mean $\bar{X}$ is complete sufficient for $\mu$ (this follows from the normal distribution being an exponential family). The statistic $S^2 = \frac{1}{n-1}\sum_{i=1}^n (X_i - \bar{X})^2$ has a distribution that depends only on $\sigma^2$ , not on $\mu$ . So $S^2$ is ancillary for $\mu$ . By Basu's theorem, $\bar{X}$ and $S^2$ are independent. This is the fact that makes the t-statistic $(\bar{X} - \mu)/(S/\sqrt{n})$ have a t-distribution.

Example

Exponential distribution: mean and coefficient of variation

Let $X_1, \ldots, X_n \sim \text{Exp}(\lambda)$ . The sample sum $T = \sum X_i$ is complete sufficient for $\lambda$ . The vector of ratios $(X_1/T, \ldots, X_n/T)$ is ancillary (its distribution is uniform on the simplex, independent of $\lambda$ ). By Basu's theorem, $T$ is independent of all the ratios $X_i/T$ .

Common Confusions

Watch Out

Ancillary does not mean useless

An ancillary statistic carries no information about $\theta$ by itself. But conditionally, given the ancillary, the precision of estimation can change. This is the basis of conditional inference. Basu's theorem says: if you have a complete sufficient statistic, you cannot improve estimation by conditioning on the ancillary.

Watch Out

Completeness is doing the heavy lifting

Sufficiency alone does not imply independence from ancillary statistics. Completeness is the key condition. Think of completeness as saying the sufficient statistic has no redundancy: there is no function of $T$ that is itself ancillary.

Summary

Complete sufficient + ancillary implies independent
The proof uses completeness to upgrade "same expectation" to "equal a.s."
The main application is proving independence without computing joint distributions
Fails without completeness: sufficiency alone is not enough

Exercises

ExerciseCore

Problem

Let $X_1, \ldots, X_n \sim N(\mu, 1)$ . Identify a complete sufficient statistic and an ancillary statistic. State what Basu's theorem tells you.

ExerciseAdvanced

Problem

Give an example where a sufficient statistic $T$ and an ancillary statistic $A$ are not independent. What condition of Basu's theorem fails?

References

Canonical:

Casella & Berger, Statistical Inference, Chapter 6.2
Lehmann & Casella, Theory of Point Estimation, Chapter 4

Current:

Keener, Theoretical Statistics, Chapter 3
van der Vaart, Asymptotic Statistics (1998), Chapters 2-8

Next Topics

Fisher information: the natural next step in estimation theory

Last reviewed: April 2026

Prerequisites

Foundations this topic depends on.

Sufficient Statistics and Exponential FamiliesLayer 0B
Maximum Likelihood EstimationLayer 0B
Common Probability DistributionsLayer 0A
Sets, Functions, and RelationsLayer 0A
Basic Logic and Proof TechniquesLayer 0A
Differentiation in RnLayer 0A

Next Topics

Fisher InformationContinue →