Exponential Distribution

For $X\sim\operatorname{Exp}(\lambda)$ and $s < \lambda$, $M_X(s) = \mathbb{E}[e^{sX}] = \frac{\lambda}{\lambda - s}.$ For $s \ge \lambda$ the MGF is infinite.

The MGF is the Laplace transform of the density at $-s$. The Laplace transform of $\lambda e^{-\lambda x}$ is finite exactly when the exponent $-(\lambda - s)$ is negative, that is, when $s < \lambda$.

Differentiating twice and evaluating at zero gives $\mathbb{E}[X] = 1/\lambda$ and $\mathbb{E}[X^2] = 2/\lambda^2$, so $\operatorname{Var}(X) = 1/\lambda^2$. The MGF has a pole at $s = \lambda$, which means the Exponential is light-tailed but not sub-Gaussian: it is sub-exponential. See sub-exponential random variables for the resulting tail bound.

The MGF is finite only on the half-line $s<\lambda$. Chernoff bounds for the Exponential must keep the parameter inside this region; pushing $s$ to $\lambda$ blows up the MGF and gives no useful bound. The Gamma and Chi-squared inherit this restriction because they are sums of Exponentials.

For $X\sim\operatorname{Exp}(\lambda)$ and every $s,t\ge 0$, $\mathbb{P}(X > s + t \mid X > s) = \mathbb{P}(X > t).$

A light bulb whose failure time is Exponential has no memory of how long it has been on. Given it has not failed by time $s$, the distribution of the remaining time looks identical to the distribution of a fresh bulb.

The memoryless property is the defining feature of the Exponential family among continuous distributions on $[0,\infty)$. It is what makes Poisson processes Markovian: the future does not depend on the past beyond the current time. It also makes Exponential models inappropriate when the hazard rate of the underlying process changes over time; in those cases use the Gamma distribution or a Weibull distribution.

The memoryless property does not hold for the Gamma distribution with shape parameter different from one, the Lognormal, or the Weibull with shape parameter different from one. Reliability data with increasing or decreasing hazard rates should not be modeled by an Exponential; the wrong parametric family will systematically misprice tail risk.

If $X$ is a nonnegative continuous random variable that is memoryless and not identically zero, then there exists a unique $\lambda > 0$ with $X\sim\operatorname{Exp}(\lambda)$.

Memorylessness is the multiplicative functional equation $G(s+t) = G(s)G(t)$ for the survival function $G(x) = \mathbb{P}(X > x)$. The only nonzero continuous solutions are the decaying exponentials, $G(x) = e^{-\lambda x}$.

The characterization is what makes "memoryless waiting time" a one-line argument for choosing the Exponential family: no other continuous distribution on $[0,\infty)$ has it. Combined with the discrete analogue (geometric is the only memoryless distribution on $\{1,2,\dots\}$), the result anchors a clean decision: if you have evidence the hazard rate is constant in time, the Exponential is forced; if not, do not use it.

The characterization assumes continuity. Replacing it with general right-continuity allows for trivial distributions concentrated at zero. The discrete analogue with geometric distributions on $\{1,2,\dots\}$ uses a different functional equation; the two characterizations are parallel but not interchangeable.

Let $X_i\sim\operatorname{Exp}(\lambda_i)$ for $i=1,\dots,n$ be independent. Then $M = \min(X_1,\dots,X_n) \sim \operatorname{Exp}\!\left(\sum_{i=1}^n \lambda_i\right),$ and $\mathbb{P}(M = X_j) = \frac{\lambda_j}{\sum_{i=1}^n \lambda_i},\qquad j=1,\dots,n.$

The probability that no event has happened by time $t$ is the product of the survival probabilities, which is exponential in the sum of rates. The probability that event $j$ is the first equals the relative rate $\lambda_j/\sum\lambda_i$, by symmetry of the joint density.

This identity is what competing-risks models, queueing systems, and the Gillespie algorithm for simulating continuous-time Markov chains depend on. Each event clock is exponential; the first to fire determines the next state, and the time of the first firing is itself exponential.

The result requires independence and exponential marginals. For dependent or non-exponential lifetimes, the minimum is not exponential and the relative-rate identification of which clock fires first fails. The classical competing-risks formula generalizes via cause-specific hazards, not by the elementary calculation here.

Given an i.i.d. sample $X_1,\dots,X_n$ from $\operatorname{Exp}(\lambda)$, the MLE is $\hat\lambda = \frac{n}{\sum_{i=1}^n X_i} = \frac{1}{\bar X_n}.$ The MLE for the scale parameterization is $\hat\theta = \bar X_n$.

The log-likelihood is concave in $\lambda$ with a single critical point. The MLE for the rate is the reciprocal of the sample mean; the MLE for the mean is the sample mean.

$\hat\lambda$ is biased upward in finite samples: $\mathbb{E}[\hat\lambda] = n\lambda/(n-1)$ for $n\ge 2$, computed from the fact that $n/(n\bar X_n) = 1/\bar X_n$ and $n\bar X_n \sim \operatorname{Gamma}(n,\lambda)$ has expected reciprocal $\lambda n/(n-1)$. The bias-corrected estimator is $(n-1)/(n\bar X_n)$. For large $n$ the bias is negligible.

The MLE is undefined if every $X_i = 0$, which happens with probability zero for continuous data but can happen with quantized or censored data. Survival analysis with censoring requires a modified likelihood; see survival analysis.