Poisson Distribution

Let $X_n\sim\operatorname{Bin}(n,p_n)$ with $np_n\to\lambda > 0$ as $n\to\infty$. Then for every fixed $k\in\{0,1,2,\dots\}$, $\mathbb{P}(X_n = k)\to\frac{\lambda^k e^{-\lambda}}{k!}.$

The Binomial PMF $\binom{n}{k}p_n^k(1-p_n)^{n-k}$ has three factors. The binomial coefficient grows like $n^k/k!$. The factor $p_n^k$ is $(np_n/n)^k\to(\lambda/n)^k$ times $n^k$, so the two cancel to give $\lambda^k/k!$. The factor $(1-p_n)^{n-k}\to e^{-\lambda}$ by the calculus identity $(1-x/n)^n\to e^{-x}$.

This is the classical justification for using a Poisson model when you have a large number of nearly impossible independent trials: defects on a manufactured chip, mutations along a long DNA sequence, hits on a server in a one-second window. The convergence is pointwise in $k$, but it can be strengthened to total-variation convergence; the rate is $O(\lambda^2/n)$ in TV distance, which is the basis of Le Cam's Poisson-approximation theorem.

The limit requires independence and constant per-trial probability $p_n$. Real-world counts of rare events often violate one or both: hospital admissions cluster across patients with the same flu; defects cluster within a single manufacturing batch. When events are positively correlated, the Poisson under-disperses the data (the empirical variance exceeds the mean), and a Negative Binomial or compound Poisson is the right model.

For $X\sim\operatorname{Pois}(\lambda)$ and every $s\in\mathbb{R}$, $M_X(s) = \mathbb{E}[e^{sX}] = \exp\!\left(\lambda(e^s - 1)\right).$

The exponential generating function of the PMF is the same as the MGF after the substitution $z = e^s$. Identifying $\sum_k \lambda^k z^k/k! = e^{\lambda z}$ gives the result up to the $e^{-\lambda}$ normalization.

Differentiating once at $s = 0$ gives $\mathbb{E}[X] = \lambda$; differentiating twice gives $\mathbb{E}[X^2] = \lambda + \lambda^2$, so $\operatorname{Var}(X) = \lambda$. The mean equals the variance, and this is a sharp diagnostic: if a count data set has empirical variance significantly larger than its mean, the Poisson model is misspecified.

The Poisson MGF is finite for every $s$, but only narrowly so: the log-MGF $\lambda(e^s-1)$ grows doubly exponentially in $s$, which makes the Poisson sub-exponential rather than sub-Gaussian. Tail bounds for the Poisson are tighter than the generic sub-exponential bound; see Bennett's and Bernstein's inequalities.

1. Additivity. If $X_i\sim\operatorname{Pois}(\lambda_i)$ are independent, then $\sum X_i \sim \operatorname{Pois}(\sum\lambda_i)$.
2. Conditional binomiality. Conditional on $X_1+\cdots+X_k = N$, the joint distribution of $(X_1,\dots,X_k)$ is multinomial with parameters $N$ and $(\lambda_1/\Lambda,\dots,\lambda_k/\Lambda)$ where $\Lambda = \sum\lambda_i$.
3. Thinning. If $X\sim\operatorname{Pois}(\lambda)$ and each event is independently classified as type $A$ with probability $p$ and type $B$ with probability $1-p$, then the type-$A$ count is $\operatorname{Pois}(\lambda p)$, the type-$B$ count is $\operatorname{Pois}(\lambda(1-p))$, and the two are independent.

Independent Poisson processes merge ("superposition") into a Poisson process whose rate is the sum. Splitting events of one Poisson process into types based on independent coin flips ("thinning") gives independent Poisson processes whose rates partition the original. The conditional-binomial statement is the discrete-time consequence of the same construction.

Superposition justifies pooling counts from independent sources with potentially different rates. Thinning justifies splitting a single count stream into independent sub-streams. The conditional-binomial result is what makes Pearson's Chi-squared test for cell counts valid: under the null hypothesis of independence, observed cell counts are conditionally multinomial with cell probabilities equal to row times column marginals. See chi-squared distribution and tests.

All three results require independence. Two count streams that interact (the second is triggered by the first) are not the superposition of independent Poissons; their merged process is not Poisson. Thinning with state-dependent rates produces a non-Poisson type-$A$ count.

For an i.i.d. sample $X_1,\dots,X_n$ from $\operatorname{Pois}(\lambda)$, the MLE is $\hat\lambda = \bar X_n = \frac{1}{n}\sum_{i=1}^n X_i.$ The MLE is unbiased and achieves the Cramer-Rao lower bound exactly: $\operatorname{Var}(\hat\lambda) = \lambda/n$.

The Poisson is a one-parameter exponential family with sufficient statistic $\sum X_i$. The MLE of the natural parameter is the empirical mean of the sufficient statistic.

The Poisson MLE is one of the few MLEs that achieves the Cramer-Rao bound exactly in finite samples. The asymptotic theory of MLEs is unnecessary here; the result holds at $n = 1$. The estimator is unbiased, consistent, and efficient, which together is most of what point-estimation theory asks for. See maximum likelihood estimation for the general framework.

The MLE assumes Poisson data. With overdispersed counts (variance exceeding mean), $\bar X_n$ is still consistent for the mean but the model is misspecified; standard errors based on $\hat\lambda/n$ underestimate the true sampling variance. The fix is a Negative Binomial regression or a Quasi-Poisson approach. See maximum likelihood estimation for the QMLE / sandwich-variance treatment of misspecification.