Bayes's Rule

Bayes’s Rule is a fundamental principle in probability theory that describes how to update the probability of a hypothesis based on new evidence. It is named after Reverend Thomas Bayes (1701–1761), whose essay on the subject was published posthumously, and later formalized by Pierre-Simon Laplace. The rule states:

( P(h|d) = \frac{P(d|h) \times P(h)}{P(d)} )

where ( P(h) ) is the prior probability of hypothesis ( h ), ( P(d|h) ) is the likelihood of observing data ( d ) given ( h ), and ( P(d) ) is the marginal probability of ( d ) (the sum over all hypotheses of ( P(d|h)P(h) )). This formula provides a coherent way to incorporate new evidence into existing beliefs.

Applications and related prediction rules depend on their assumptions:

• Laplace’s Law: A specific case for binary events, where the probability of success after ( w ) wins and ( n ) attempts is ( \frac{w+1}{n+2} ). This assumes a uniform prior. For example, if you win a lottery on your first try, your estimated probability of winning next time is ( \frac{2}{3} ).
• Prediction Rules: Depending on the underlying distribution, different prediction rules emerge. For a memoryless waiting-time model (exponential/geometric), the Additive Rule applies: the expected remaining wait stays constant, so expected total duration is elapsed time plus that remaining wait. For power-law distributions, a Multiplicative Rule gives different predictions.
• Copernican Principle: J. Richard Gott applied this reasoning to predict the remaining lifespan of the Berlin Wall by assuming his observation time was random within the wall’s existence.
• Real-world examples: The German Tank Problem, where Bayesian inference was used to estimate the number of tanks from serial numbers; the analysis of car accidents vs. plane crash fatalities; and studies on how children’s prior experiences affect their ability to delay gratification (marshmallow test).