Prospect Theory

Why This Matters

Expected utility theory (EUT) prescribes how a rational agent should decide. Prospect theory describes how people actually decide. The gap between these is large and systematic. People overweight small probabilities, are more sensitive to losses than to gains, and evaluate outcomes relative to a reference point rather than in absolute terms.

For ML, this matters in at least three ways. First, RLHF relies on human preference judgments, and those judgments follow prospect theory, not EUT. Second, any system that presents risk information to users (medical AI, financial models) must account for how users perceive probabilities. Third, reward function design in RL implicitly assumes a value function shape; prospect theory shows why symmetric reward functions may not match human preferences.

The Failures of Expected Utility

Before building prospect theory, we need to see why EUT fails as a descriptive model.

Definition

Allais Paradox

Consider two choice problems:

Problem 1: Choose between A ($1M for certain) and B ($5M with probability 0.10, $1M with probability 0.89,$0 with probability 0.01).

Problem 2: Choose between C ($1M with probability 0.11,$0 with probability 0.89) and D ($5M with probability 0.10,$0 with probability 0.90).

Most people choose A over B and D over C. Under EUT with any utility function, choosing A over B implies choosing C over D (by the independence axiom). The observed pattern violates EUT.

The Allais paradox reveals the certainty effect: people overweight outcomes that are certain relative to outcomes that are merely probable. Moving from certainty (100%) to near-certainty (99%) feels much worse than moving from 11% to 10%, even though the probability difference is the same.

Definition

Reflection Effect

The risk preferences observed for gains reverse for losses. People are risk-averse for gains (preferring a sure $500 over a 50/50 chance at$1000) but risk-seeking for losses (preferring a 50/50 chance of losing $1000 over a sure loss of$500). Under EUT with a concave utility function, the agent should be risk-averse everywhere.

The Prospect Theory Model

Kahneman and Tversky (1979) proposed a two-phase model:

Phase 1: Editing. The decision-maker frames outcomes as gains or losses relative to a reference point, simplifies dominated options, and isolates shared components. This phase is informal and context-dependent.

Phase 2: Evaluation. The edited prospects are evaluated using a value function $v$ and a probability weighting function $w$, and the prospect with the highest weighted value is chosen.

For a simple prospect $(x, p; y, q)$ yielding $x$ with probability $p$ and $y$ with probability $q$ (with $p + q \leq 1$ and outcome $0$ otherwise):

$V(x, p; y, q) = w(p) \cdot v(x) + w(q) \cdot v(y)$

Proposition

Prospect Theory Value Function Properties

Statement

The value function $v: \mathbb{R} \to \mathbb{R}$ in prospect theory has three key properties:

1. Reference dependence: $v(0) = 0$. Outcomes are evaluated as deviations from a reference point, not as final wealth levels.
2. Diminishing sensitivity: $v$ is concave for gains ($v''(x) < 0$ for $x > 0$) and convex for losses ($v''(x) > 0$ for $x < 0$).
3. Loss aversion: $v$ is steeper for losses than for gains. $|v(-x)| > v(x)$ for all $x > 0$. The loss aversion coefficient $\lambda = -v(-x)/v(x)$ is typically around 2.25 empirically.

A commonly used parametric form is:

$v(x) = \begin{cases} x^\alpha & \text{if } x \geq 0 \\ -\lambda(-x)^\beta & \text{if } x < 0 \end{cases}$

with $\alpha \approx 0.88$, $\beta \approx 0.88$, and $\lambda \approx 2.25$.

Intuition

The concavity for gains and convexity for losses follows from diminishing sensitivity. The difference between $10 and$20 feels larger than the difference between $1010 and$1020, whether these are gains or losses. Loss aversion is a separate phenomenon: losing $100 hurts more than gaining$100 feels good. The asymmetry is roughly 2:1.

Why It Matters

This value function explains the reflection effect (risk aversion for gains, risk seeking for losses), the endowment effect (people demand more to give up an object than they would pay to acquire it), and the status quo bias (losses from switching loom larger than gains). These are not irrational quirks; they are systematic patterns that any model of human preferences must accommodate.

Failure Mode

The reference point is not always well-defined. In complex multi-attribute decisions, what counts as "the reference point" can change with framing. The parametric form with $\alpha = \beta$ is a simplification; individual heterogeneity is large. Loss aversion may also be domain-dependent, with different coefficients for money, health, and time.

Probability Weighting

Definition

Probability Weighting Function

The probability weighting function $w: [0,1] \to [0,1]$ transforms objective probabilities into decision weights. It satisfies $w(0) = 0$ and $w(1) = 1$ but is not linear. Key properties:

1. Overweighting of small probabilities: $w(p) > p$ for small $p$. This explains why people buy lottery tickets (overweight the small chance of winning) and insurance (overweight the small chance of a catastrophe).
2. Underweighting of moderate to large probabilities: $w(p) < p$ for moderate to large $p$.
3. Subcertainty: $w(p) + w(1-p) < 1$ for $0 < p < 1$. Decision weights do not sum to 1.

A common parametric form (Prelec, 1998):

$w(p) = \exp(-(-\ln p)^\gamma)$

with $\gamma \approx 0.65$ estimated empirically.

The weighting function is concave near 0 (overweighting) and convex near 1 (underweighting), producing an inverse-S shape. This single function explains both gambling (overweight rare large gains) and insurance (overweight rare large losses).

Cumulative Prospect Theory

Original prospect theory (1979) had a problem: it could violate stochastic dominance. If prospect $A$ yields outcomes at least as good as prospect $B$ in every state, a rational agent should prefer $A$. But the probability weighting in the original model could reverse this.

Definition

Cumulative Prospect Theory

Cumulative prospect theory (Tversky and Kahneman, 1992) fixes this by applying the weighting function to cumulative probabilities rather than individual probabilities. For a prospect with outcomes $x_1 \leq \cdots \leq x_n$ occurring with probabilities $p_1, \ldots, p_n$, separate gains and losses:

For gains ($x_i > 0$), the decision weight is:

$\pi_i^+ = w^+(p_i + \cdots + p_n) - w^+(p_{i+1} + \cdots + p_n)$

For losses ($x_i < 0$), the decision weight is:

$\pi_i^- = w^-(p_1 + \cdots + p_i) - w^-(p_1 + \cdots + p_{i-1})$

The overall value is $V = \sum_{x_i \geq 0} \pi_i^+ v(x_i) + \sum_{x_i < 0} \pi_i^- v(x_i)$.

Cumulative prospect theory preserves stochastic dominance while retaining the key empirical predictions: loss aversion, diminishing sensitivity, and probability weighting. It allows different weighting functions $w^+$ and $w^-$ for gains and losses.

The Fourfold Pattern of Risk Attitudes

Prospect theory predicts a distinctive pattern:

	High probability	Low probability
Gains	Risk averse	Risk seeking
Losses	Risk seeking	Risk averse

• High-probability gains: Diminishing sensitivity makes the value function concave; risk aversion.
• Low-probability gains: Overweighting of small probabilities dominates; risk seeking (lottery tickets).
• High-probability losses: Diminishing sensitivity makes the value function convex; risk seeking (gambling to avoid a near-certain loss).
• Low-probability losses: Overweighting of small probabilities dominates; risk aversion (insurance).

EUT with a fixed utility function cannot produce all four patterns simultaneously.

Connections to ML

RLHF and reward modeling. Human evaluators in RLHF express preferences between model outputs. These preferences are typically modeled as following a Bradley-Terry model, which assumes a consistent utility scale. But prospect theory predicts that human preferences are reference-dependent, loss-averse, and sensitive to framing. A model output that is a clear improvement over the current response may be judged differently depending on whether the evaluator frames the comparison as "how much better is A?" versus "what is wrong with B?"

Calibration and risk communication. ML models that output calibrated probabilities assume users interpret probabilities linearly. Prospect theory says they do not. A model reporting 2% cancer risk will be perceived as more alarming than a linear model would predict, because users overweight small probabilities.

Reward shaping in RL. If the reward function is symmetric around zero, prospect theory predicts that an agent trained to match human preferences should weight negative rewards more heavily than positive ones. The optimal reward shaping for human-aligned behavior may need a loss-aversion coefficient.

Common Confusions

Watch Out

Prospect theory is descriptive, not normative

EUT says what you should do given rationality axioms. Prospect theory says what you will do given cognitive biases. A well-calibrated Bayesian should not follow prospect theory. But if you are modeling human behavior, predicting human preferences, or designing interfaces for human users, prospect theory is the better model.

Watch Out

Loss aversion is not risk aversion

Risk aversion (concave utility for gains) and loss aversion (steeper value for losses than gains) are different phenomena. A risk-neutral agent can still be loss-averse. Risk aversion is about the curvature of the value function; loss aversion is about the slope difference at the reference point. Prospect theory includes both.

Watch Out

The reference point is not always the status quo

The reference point can be an expectation, a goal, or a social comparison. If you expect a $5000 bonus and receive$4000, the reference point is $5000 and you code the outcome as a$1000 loss, even though you received $4000. The reference point is context-dependent and can be manipulated by framing.

Exercises

ExerciseCore

Problem

Under the prospect theory value function $v(x) = x^{0.88}$ for $x \geq 0$ and $v(x) = -2.25(-x)^{0.88}$ for $x < 0$, compute $v(100)$, $v(-100)$, and the ratio $|v(-100)|/v(100)$. Verify that the ratio equals the loss aversion coefficient.

Hint

Reveal Solution

ExerciseCore

Problem

The Allais paradox: A decision-maker with utility function $u(x) = x^{0.5}$ (EUT with square-root utility) is presented with the two choice problems from the Allais paradox. Show that if the decision-maker prefers A over B in Problem 1, then EUT requires preferring C over D in Problem 2. What does this imply about the observed pattern (A over B and D over C)?

Hint

Reveal Solution

ExerciseAdvanced

Problem

Show that the probability weighting function $w(p) = p^\gamma / (p^\gamma + (1-p)^\gamma)^{1/\gamma}$ (Tversky-Kahneman form) satisfies $w(0) = 0$, $w(1) = 1$, and has an inverse-S shape (overweights small $p$, underweights large $p$) for $\gamma < 1$. Find the fixed point $p^*$ where $w(p^*) = p^*$.

Hint

Reveal Solution

ExerciseResearch

Problem

Consider an RLHF setting where human evaluators compare pairs of model outputs. The standard Bradley-Terry model assumes $P(A \succ B) = \sigma(r(A) - r(B))$ where $r$ is a reward function and $\sigma$ is the logistic function. Propose a modification that incorporates prospect-theoretic features (reference dependence and loss aversion). What would the training objective look like, and what empirical signatures would distinguish it from the standard model?

Hint

Reveal Solution

References

Original papers:

• Kahneman and Tversky, "Prospect Theory: An Analysis of Decision under Risk," Econometrica 47(2), 1979, pp. 263-291
• Tversky and Kahneman, "Advances in Prospect Theory: Cumulative Representation of Uncertainty," Journal of Risk and Uncertainty 5(4), 1992, pp. 297-323

Probability weighting:

• Prelec, "The Probability Weighting Function," Econometrica 66(3), 1998, pp. 497-527

Textbook treatments:

• Wakker, Prospect Theory: For Risk and Ambiguity (2010), Chapters 3-9
• Kahneman, Thinking, Fast and Slow (2011), Chapters 25-29
• Mas-Colell, Whinston, and Green, Microeconomic Theory (1995), Chapter 6

Next Topics

Prospect theory connects to several directions:

• Decision theory foundations: the normative framework that prospect theory departs from
• Game theory: strategic interaction where agents may follow prospect-theoretic rather than EUT preferences