Energy Distance and Distance Covariance

Why This Matters

Means and covariances can agree while distributions differ. Pearson correlation can be zero while variables remain strongly dependent. Energy statistics address both problems using pairwise distances:

• energy distance measures whether two probability distributions differ;
• distance covariance measures whether two random vectors are dependent;
• permutation tests turn both quantities into finite-sample tests under exchangeability;
• distance-induced kernels connect energy distance to maximum mean discrepancy (MMD);
• the same population discrepancy yields a strictly proper multivariate forecast score.

This use of energy is separate from energy-based models. There is no learned energy landscape or partition function here. See Energy in Statistics and Machine Learning for the terminology map.

Mental Model

Pool two samples and inspect all pairwise distances.

• If the samples come from the same distribution, a cross-sample pair should look much like a within-sample pair.
• If the distributions differ, cross-sample distances tend to be larger than the within-sample distances after the comparison is centered correctly.

Energy distance is twice the average cross-distance minus the two average within-distances. Distance covariance applies a related idea to paired data: center the distance matrix for $X$, center the distance matrix for $Y$, then measure whether large and small distances align across the two matrices.

Population Energy Distance

Let $(\mathcal X,\rho)$ be a space with a symmetric nonnegative semimetric $\rho$ satisfying $\rho(x,x)=0$. Let

$$X,X'\overset{\mathrm{iid}}{\sim}P, \qquad Y,Y'\overset{\mathrm{iid}}{\sim}Q,$$

with all four variables mutually independent.

Definition

Energy Distance${\mathcal{E}}_{\rho }(P,Q)$

Under the convention used on this page, the squared-form energy distance is

$$\mathcal E_\rho(P,Q) =2\mathbb E\rho(X,Y) -\mathbb E\rho(X,X') -\mathbb E\rho(Y,Y').$$

This page follows the common Székely-Rizzo statistical convention and calls $\mathcal E_\rho$ the energy distance. Some metric treatments instead apply the name to $\sqrt{\mathcal E_\rho(P,Q)}$. Always check the convention. For Euclidean energy distance, $\rho(x,y)=\|x-y\|_2$ rather than squared Euclidean distance.

The expectations are finite when $P$ and $Q$ have finite first $\rho$-moments, meaning $\mathbb E\rho(X,x_0)+\mathbb E\rho(Y,x_0)<\infty$ for some $x_0$.

Theorem

Energy Distance Characterizes Equality of Distributions

Statement

$$\mathcal E_\rho(P,Q)\ge0, \qquad \mathcal E_\rho(P,Q)=0\ \Longleftrightarrow\ P=Q.$$

Euclidean distance on $\mathbb R^d$ is of strong negative type, so the result holds for random vectors with finite first moments.

Intuition

The cross-distance term is compared with the natural within-distribution spread of each law. Strong negative type says that this centered comparison separates probability measures, not merely individual points.

Proof Sketch

Negative type gives a Hilbert-space embedding $\varphi:\mathcal X\to\mathcal H$ such that $\rho(x,y)=\|\varphi(x)-\varphi(y)\|_{\mathcal H}^2$. Expanding the three expectations gives

$$\mathcal E_\rho(P,Q) =2\left\|\mathbb E_P\varphi(X)-\mathbb E_Q\varphi(Y)\right\|_{\mathcal H}^2.$$

This is nonnegative. Strong negative type makes the mean embedding injective on probability measures with finite first moments, giving the equality claim.

Why It Matters

Energy distance detects any distributional difference allowed by the assumptions, not only a change in mean or covariance. It supplies a nonparametric two-sample target without density estimation.

Failure Mode

Negative type alone gives nonnegativity but may not make zero identify the distribution. Without finite moments, one or more expectations can be infinite. Replacing $\|x-y\|$ with an arbitrary dissimilarity can invalidate both conclusions.

report a correction →

Example

Two point masses

Let $P=\delta_0$ and $Q=\delta_a$ on $\mathbb R^d$. Then $\mathbb E\|X-Y\|=\|a\|$, while both within-distribution expectations are zero. Hence

$\mathcal E(P,Q)=2\|a\|.$

The discrepancy increases with separation, even though neither distribution has within-sample spread.

Sample Estimators

Given independent observations $x_1,\ldots,x_n\sim P$ and $y_1,\ldots,y_m\sim Q$, define pairwise distances $a_{ij}=\rho(x_i,x_j)$, $b_{ij}=\rho(y_i,y_j)$, and $c_{ij}=\rho(x_i,y_j)$.

Biased V-statistic

The plug-in estimator is

$\widehat{\mathcal E}_{V}=\frac{2}{nm}\sum_{i=1}^{n}\sum_{j=1}^{m}c_{ij}-\frac{1}{n^2}\sum_{i=1}^{n}\sum_{j=1}^{n}a_{ij}-\frac{1}{m^2}\sum_{i=1}^{m}\sum_{j=1}^{m}b_{ij}.$

For a negative-type semimetric this estimator is nonnegative. It is biased because empirical within-sample expectations include diagonal pairs, although $a_{ii}=b_{jj}=0$ for ordinary distances.

Unbiased U-statistic

Removing diagonal pairs and changing the denominators gives

$\widehat{\mathcal E}_{U}=\frac{2}{nm}\sum_{i=1}^{n}\sum_{j=1}^{m}c_{ij}-\frac{1}{n(n-1)}\sum_{i\ne j}a_{ij}-\frac{1}{m(m-1)}\sum_{i\ne j}b_{ij}.$

This estimator is unbiased for $\mathcal E_\rho(P,Q)$ when $n,m\ge2$. Unlike the population distance and the biased estimator, it can be negative in a finite sample. That is not a contradiction: unbiasedness does not impose the range of the estimand on every realization.

Watch Out

Biased does not mean invalid

The nonnegative V-statistic and the unbiased U-statistic answer slightly different finite-sample needs. Either can be used as an ordering statistic in a correctly constructed permutation test. Do not truncate a negative U-statistic to zero unless the chosen method explicitly calls for that transformation.

The Two-Sample Permutation Test

To test $H_0:P=Q$ against $H_1:P\ne Q$:

1. Compute $\widehat{\mathcal E}$ on the observed group labels.
2. Pool all $n+m$ observations.
3. Reassign $n$ observations to the first group and $m$ to the second.
4. Recompute the statistic for each reassignment, or for a random set of reassignments.
5. Reject for an unusually large observed statistic.

Under $H_0$, the pooled observations are exchangeable with respect to the group labels. Enumerating every label allocation gives a conditional test of level at most $\alpha$, with randomized boundary handling available for exact size. For $B$ random permutations, use the corrected one-sided value

$\widehat p=\frac{1+\sum_{b=1}^{B}\mathbf 1\{\widehat{\mathcal E}^{(b)}\ge\widehat{\mathcal E}^{\mathrm{obs}}\}}{B+1}.$

The correction prevents a zero Monte Carlo p-value and preserves finite-$B$ validity under the standard exchangeability construction. See Permutation Tests for the general argument.

Watch Out

Exchangeability is the load-bearing assumption

Independent samples from the same distribution satisfy the usual two-sample null. Clustered data, time series, matched pairs, and covariate-dependent assignment may not permit unrestricted label shuffling. Use a permutation scheme that respects the design, or use a method derived for that dependence structure.

Equivalence to Maximum Mean Discrepancy

Energy distance and kernel MMD can be two coordinate systems for the same population discrepancy.

For a semimetric $\rho$ of negative type and a fixed base point $x_0$, define the distance-induced kernel

$$k_{x_0}(x,y) =\frac12\left[ \rho(x,x_0)+\rho(y,x_0)-\rho(x,y) \right].$$

Changing $x_0$ changes the kernel representation but not the MMD between probability measures.

Theorem

Energy Distance Equals Twice Squared MMD

Statement

$$\mathcal E_\rho(P,Q)=2\,\operatorname{MMD}_k^2(P,Q).$$

If $\rho$ is of strong negative type, the associated mean embedding is injective on the relevant probability measures, matching the $\mathcal E_\rho(P,Q)=0\iff P=Q$ property.

Intuition

Both quantities compare a cross expectation with two within-distribution expectations. Substituting the distance generated by $k$ makes the marginal terms cancel, leaving the same centered discrepancy up to a factor of two.

Proof Sketch

Start from

$$\operatorname{MMD}_k^2 =\mathbb E k(X,X')+\mathbb E k(Y,Y')-2\mathbb E k(X,Y).$$

Replace each $\rho$ in $\mathcal E_\rho$ by $k(u,u)+k(v,v)-2k(u,v)$. All diagonal expectation terms cancel, and the remaining expression is twice the displayed MMD quantity.

Why It Matters

Results and implementations can move between distance and kernel language. Choosing a ground semimetric corresponds to choosing a kernel geometry; both choices control test power.

Failure Mode

An arbitrary distance need not generate a positive-definite kernel, and an arbitrary kernel need not be characteristic. The equivalence does not say that every distance test has power against every alternative.

report a correction →

Distance Covariance and Distance Correlation

Energy distance compares two distributions. Distance covariance instead tests whether paired random vectors $X\in\mathbb R^p$ and $Y\in\mathbb R^q$ are independent.

Given paired observations $(x_i,y_i)_{i=1}^n$, form

$$a_{ij}=\|x_i-x_j\|_2, \qquad b_{ij}=\|y_i-y_j\|_2.$$

Double-center both distance matrices:

$$A_{ij}=a_{ij}-\bar a_{i\cdot}-\bar a_{\cdot j}+\bar a_{\cdot\cdot}, \qquad B_{ij}=b_{ij}-\bar b_{i\cdot}-\bar b_{\cdot j}+\bar b_{\cdot\cdot}.$$

Here $\bar a_{i\cdot}$ is row $i$'s mean, $\bar a_{\cdot j}$ is column $j$'s mean, and $\bar a_{\cdot\cdot}$ is the grand mean. The usual biased sample squared distance covariance is

$\operatorname{dCov}_n^2(X,Y)=\frac{1}{n^2}\sum_{i=1}^n\sum_{j=1}^n A_{ij}B_{ij}.$

For $n\ge4$, an unbiased estimator uses U-centering rather than the ordinary double-centering above. For $i\ne j$, define

$\widetilde A_{ij}=a_{ij}-\frac{1}{n-2}\sum_{\ell=1}^n a_{i\ell}-\frac{1}{n-2}\sum_{k=1}^n a_{kj}+\frac{1}{(n-1)(n-2)}\sum_{k=1}^n\sum_{\ell=1}^n a_{k\ell},\qquad \widetilde A_{ii}=0,$

and define $\widetilde B$ analogously. Then

$\operatorname{dCov}_{U}^2(X,Y)=\frac{1}{n(n-3)}\sum_{i\ne j}\widetilde A_{ij}\widetilde B_{ij}$

is unbiased for squared population distance covariance. Like the unbiased energy-distance estimator, it can be negative in a finite sample. The ordinary double-centered statistic remains common for permutation testing; the estimator and null calibration should be reported explicitly.

The normalized distance correlation is

$\operatorname{dCor}_n^2(X,Y)=\frac{\operatorname{dCov}_n^2(X,Y)}{\sqrt{\operatorname{dCov}_n^2(X,X)\operatorname{dCov}_n^2(Y,Y)}},$

when the denominator is positive, and is defined as zero otherwise. For ordinary Euclidean distances, the denominator vanishes when at least one sample has no distance variation, as when every observed value on that side is identical.

Theorem

Distance Covariance Characterizes Independence

Statement

The population distance covariance satisfies

$$\operatorname{dCov}^2(X,Y)\ge0, \qquad \operatorname{dCov}^2(X,Y)=0 \ \Longleftrightarrow\ X\ \text{and}\ Y\ \text{are independent}.$$

More generally, the equivalence holds when the marginal spaces are of strong negative type and the required moments exist.

Intuition

If two observations are far apart in $X$ whenever they are far apart in $Y$, the centered distance matrices align. Independence removes this systematic alignment. Strong negative type prevents a dependent joint law from having the same embedding as the product of its marginals.

Proof Sketch

In Euclidean spaces, population distance covariance has a weighted characteristic-function representation:

$$\operatorname{dCov}^2(X,Y) =\int \left|\varphi_{X,Y}(t,s)-\varphi_X(t)\varphi_Y(s)\right|^2 w(t,s)\,dt\,ds.$$

The nonnegative integral is zero exactly when the joint characteristic function factors everywhere, which is equivalent to independence. The metric space result replaces characteristic functions with injective distance or kernel embeddings.

Why It Matters

Ordinary covariance detects linear association. At the population level, distance covariance separates independence from dependence under its assumptions while remaining computable from pairwise distances. Finite-sample power still depends on the alternative, dimension, tails, and calibration.

Failure Mode

In a general metric space lacking strong negative type, zero distance covariance need not imply independence. A finite sample value near zero is not itself a calibrated test result; use an appropriate permutation or asymptotic null procedure.

report a correction →

Example

Zero Pearson correlation but perfect dependence

Let $X$ be uniform on $[-1,1]$ and let $Y=X^2$. Symmetry gives $\operatorname{Cov}(X,Y)=\mathbb E[X^3]=0$, yet $Y$ is determined by $X$. Population distance covariance is positive because $X$ and $Y$ are not independent.

The Proper Energy Score

Let a forecast distribution be $F$ on $\mathbb R^d$, let $y$ be the realized outcome, and let $X,X'\overset{\mathrm{iid}}{\sim}F$. For $0<\beta<2$, the energy-score family in loss orientation is

$$\operatorname{ES}_\beta(F,y) =\mathbb E\|X-y\|_2^\beta -\frac12\mathbb E\|X-X'\|_2^\beta.$$

The standard $\beta=1$ member is

$$\operatorname{ES}(F,y) =\mathbb E\|X-y\|_2 -\frac12\mathbb E\|X-X'\|_2.$$

Lower is better. If the true outcome distribution is $G$, then

$$\mathbb E_{Y\sim G}\operatorname{ES}(F,Y) -\mathbb E_{Y\sim G}\operatorname{ES}(G,Y) =\frac12\mathcal E(F,G)\ge0.$$

For $\beta=1$, the energy-distance characterization makes the score strictly proper on the finite-first-moment class: the true distribution uniquely minimizes expected loss. This is the multivariate generalization of the continuous ranked probability score, and in one dimension the $\beta=1$ member equals the CRPS. For general $\beta$, the corresponding moment condition and energy distance use $\rho(x,y)=\|x-y\|_2^\beta$. This score is not the log-sum-exp OOD score sometimes called classifier energy.

Energy Distance Versus Nearby Methods

The MMD row can be mathematically equivalent to energy distance after matching the kernel and semimetric. Wasserstein distance is not: it optimizes over couplings rather than averaging independent cross-pairs.

Practical Workflow

For a two-sample analysis:

1. State the null as equality of distributions, not merely equality of means.
2. Choose a ground semimetric suited to the data and verify its negative-type property if using the general theorem.
3. Inspect scale and preprocessing. Euclidean distance can be dominated by high-variance coordinates.
4. Choose the U- or V-statistic deliberately.
5. Calibrate by a permutation scheme valid for the sampling design.
6. Report the statistic, sample sizes, permutation count, p-value convention, and the exact preprocessing.
7. Treat rejection as evidence that the distributions differ, not as a diagnosis of how they differ.

For an independence analysis, preserve the $(x_i,y_i)$ pairings and permute one side across pairs only when those permutations are justified by the null and design.

Common Confusions

Watch Out

Energy statistics are not energy-based models

Energy statistics operate on distances among observations. EBMs assign an energy to each configuration and normalize $e^{-E}$. Their formulas, algorithms, and inferential questions are different.

Watch Out

Squared Euclidean distance is not the standard Euclidean energy distance

The classical formula uses $\rho(x,y)=\|x-y\|_2$. Using $\rho(x,y)=\|x-y\|_2^2$ collapses the population expression to a multiple of the squared mean difference, so it no longer detects arbitrary distributional changes.

Watch Out

A significant test does not measure practical importance

With enough data, a small distributional difference can be detected. Report the energy statistic in context, inspect what changed, and avoid interpreting a p-value as an effect size or operational consequence.

Exercises

ExerciseCore

Problem

Compute $\mathcal E(P,Q)$ when $P=\delta_0$ and $Q=\delta_3$ on the real line using $\rho(x,y)=|x-y|$.

Hint

Reveal Solution

ExerciseAdvanced

Problem

Show that using $\rho(x,y)=\|x-y\|_2^2$ gives

$$\mathcal E_\rho(P,Q) =2\|\mathbb E X-\mathbb E Y\|_2^2$$

when second moments exist.

Hint

Reveal Solution

ExerciseAdvanced

Problem

Explain why a negative value of $\widehat{\mathcal E}_U$ does not refute the nonnegativity of population energy distance.

Hint

Reveal Solution

ExerciseResearch

Problem

Design a valid permutation test for paired observations $(x_i,y_i)_{i=1}^n$ using distance covariance. State what is permuted and the null assumption needed.

Hint

Reveal Solution

References

Energy distance and two-sample testing:

• Székely and Rizzo, "Testing for Equal Distributions in High Dimension" (InterStat, 2004), Sections 2 and 3.
• Székely and Rizzo, "Energy Statistics: A Class of Statistics Based on Distances" (Journal of Statistical Planning and Inference, 2013), Sections 2 through 4. DOI: 10.1016/j.jspi.2013.03.018.
• Baringhaus and Franz, "On a New Multivariate Two-Sample Test" (Journal of Multivariate Analysis, 2004), Sections 2 and 3. DOI: 10.1016/S0047-259X(03)00079-4.

Distance covariance and metric spaces:

• Székely, Rizzo, and Bakirov, "Measuring and Testing Dependence by Correlation of Distances" (Annals of Statistics, 2007), Theorems 1 through 3. DOI: 10.1214/009053607000000505.
• Székely and Rizzo, "Brownian Distance Covariance" (Annals of Applied Statistics, 2009), Sections 2 and 3. DOI: 10.1214/09-AOAS312.
• Székely and Rizzo, "Partial Distance Correlation with Methods for Dissimilarities" (Annals of Statistics, 2014), Section 3.1. DOI: 10.1214/14-AOS1255.
• Lyons, "Distance Covariance in Metric Spaces" (Annals of Probability, 2013), Section 3 (strong-negative-type characterization). DOI: 10.1214/12-AOP803.

Kernel equivalence, tests, and scores:

• Sejdinovic, Sriperumbudur, Gretton, and Fukumizu, "Equivalence of Distance-Based and RKHS-Based Statistics in Hypothesis Testing" (Annals of Statistics, 2013), Sections 2, 3, and Theorem 22.
• Gretton, Borgwardt, Rasch, Schölkopf, and Smola, "A Kernel Two-Sample Test" (Journal of Machine Learning Research, 2012), Sections 2 through 6.
• Gneiting and Raftery, "Strictly Proper Scoring Rules, Prediction, and Estimation" (Journal of the American Statistical Association, 2007), Section 4.3. DOI: 10.1198/016214506000001437.
• Phipson and Smyth, "Permutation P-values Should Never Be Zero: Calculating Exact P-values When Permutations Are Randomly Drawn" (Statistical Applications in Genetics and Molecular Biology, 2010), Sections 2 and 3.

Next Topics

• Kernel Two-Sample Tests: the RKHS and MMD formulation of distribution comparison.
• Proper Scoring Rules: why forecast losses should make honest probability reports optimal.
• Wasserstein Distances: comparison through optimal transport rather than independent pairwise distances.