Bayesian State Estimation

The posterior $p(x_t | z_{1:t})$ satisfies the predict-update recursion:

Predict:

$p(x_t | z_{1:t-1}) = \int p(x_t | x_{t-1}) \, p(x_{t-1} | z_{1:t-1}) \, dx_{t-1}$

Update:

$p(x_t | z_{1:t}) = \frac{p(z_t | x_t) \, p(x_t | z_{1:t-1})}{p(z_t | z_{1:t-1})}$

where $p(z_t | z_{1:t-1}) = \int p(z_t | x_t) \, p(x_t | z_{1:t-1}) \, dx_t$ is the normalizing constant.

The predict step propagates uncertainty forward through the dynamics model: if I was uncertain about $x_{t-1}$, I am even more uncertain about $x_t$ before seeing $z_t$. The update step sharpens the belief using the new observation: states that explain $z_t$ well get higher probability. This is just Bayes' rule applied recursively.

This recursion is the foundation of all Bayesian filters. Every specific algorithm (Kalman, extended Kalman, unscented Kalman, particle filter) is an approximation to this exact recursion under different assumptions about the state space and distributions.

The predict step requires computing an integral that is analytically tractable only in special cases (linear Gaussian). For nonlinear dynamics or non-Gaussian noise, you must approximate. The quality of the filter depends entirely on how well you approximate this integral.

Under linear Gaussian assumptions, the Bayes filter recursion has a closed-form solution. The belief $p(x_t | z_{1:t})$ is Gaussian $\mathcal{N}(\mu_t, \Sigma_t)$ at every time step, with:

Predict:

$\mu_{t|t-1} = A\mu_{t-1}, \qquad \Sigma_{t|t-1} = A\Sigma_{t-1}A^\top + Q$

Update:

$K_t = \Sigma_{t|t-1} H^\top (H\Sigma_{t|t-1}H^\top + R)^{-1}$

$\mu_t = \mu_{t|t-1} + K_t(z_t - H\mu_{t|t-1})$

$\Sigma_t = (I - K_tH)\Sigma_{t|t-1}$

where $K_t$ is the Kalman gain, $Q$ is the process noise covariance, and $R$ is the observation noise covariance.

The Kalman gain $K_t$ controls how much you trust the new observation versus your prediction. If observation noise $R$ is large, $K_t$ is small: you mostly trust the prediction. If prediction uncertainty $\Sigma_{t|t-1}$ is large, $K_t$ is large: you mostly trust the observation. The filter automatically balances these two sources of information.

The Kalman filter is one of the most widely used algorithms in engineering: GPS navigation, spacecraft tracking, econometrics. Its importance comes from being the exact Bayesian solution (not an approximation) for linear Gaussian systems, and from being computationally cheap ($O(d^3)$ per step where $d$ is the state dimension).

Linearity and Gaussianity are strong assumptions. Real systems with nonlinear dynamics (robot arm kinematics), multi-modal beliefs (robot unsure which side of a wall it is on), or heavy-tailed noise (financial data) violate these assumptions. The extended Kalman filter (EKF) linearizes locally, but this can diverge for highly nonlinear systems.