Beta Distribution

The beta distribution is a continuous probability distribution defined on the interval \( [0, 1] \). It is widely used in Bayesian inference, statistical modeling of proportions, and machine learning. Characterized by two positive shape parameters \(\alpha\) and \(\beta\), it offers remarkable flexibility, capable of assuming diverse shapes ranging from uniform and U-shaped to highly skewed or bell-like forms.

Probability Density Function

The probability density function (PDF) of the beta distribution is given by:

\[ f(x; \alpha, \beta) = \frac{x^{\alpha-1}(1-x)^{\beta-1}}{B(\alpha, \beta)}, \quad \text{for } 0 \leq x \leq 1 \]

where \( B(\alpha, \beta) \) is the beta function, which acts as a normalization constant to ensure the total probability integrates to 1:

\[ B(\alpha, \beta) = \int_0^1 t^{\alpha-1}(1-t)^{\beta-1} \, dt = \frac{\Gamma(\alpha)\Gamma(\beta)}{\Gamma(\alpha+\beta)} \]

Here, \(\Gamma(\cdot)\) denotes the gamma function, a generalization of the factorial function to real and complex numbers.

Key Properties

The statistical behavior of the beta distribution is governed entirely by \(\alpha\) and \(\beta\). Below are its fundamental moments and characteristics:

Moments

\[ \text{Mean: } \quad \mu = \frac{\alpha}{\alpha + \beta} \] \[ \text{Variance: } \quad \sigma^2 = \frac{\alpha\beta}{(\alpha + \beta)^2(\alpha + \beta + 1)} \] \[ \text{Mode: } \quad \frac{\alpha-1}{\alpha+\beta-2} \quad (\alpha,\beta > 1) \]

๐Ÿ’ก Insight: The parameters \(\alpha\) and \(\beta\) can be interpreted as "prior successes" and "prior failures" in a binomial process. Larger values concentrate the distribution around the mean, while small values near 1 produce flatter or U-shaped curves.

Shape Behavior

  • \(\alpha = \beta = 1\): Uniform distribution on \([0,1]\)
  • \(\alpha, \beta < 1\): U-shaped with mass at boundaries
  • \(\alpha > \beta\): Right-skewed, mass near 1
  • \(\alpha < \beta\): Left-skewed, mass near 0
  • \(\alpha = \beta > 1\): Symmetric, bell-shaped around 0.5

Applications

The beta distribution's bounded support and shape flexibility make it indispensable across multiple domains:

  • Bayesian Statistics: Conjugate prior for the binomial, beta-binomial, geometric, and negative binomial distributions.
  • Project Management: Used in PERT analysis to model task completion times where uncertainty is bounded.
  • Machine Learning: Models probabilities in reinforcement learning, bandit problems, and probabilistic graphical models.
  • Finance & Risk: Represents loss rates, default probabilities, and portfolio weight allocations constrained to \([0,1]\).
  • Biology & Ecology: Describes gene frequencies, species abundance distributions, and reaction rates.

References & Further Reading

  1. 1 Johnson, N. L., Kotz, S., & Balakrishnan, N. (1995). Continuous Univariate Distributions, Vol. 2. Wiley.
  2. 2 Gelman, A. et al. (2020). Bayesian Data Analysis (3rd ed.). CRC Press. Chapter 3: Conjugate Priors.
  3. 3 Wikipedia Contributors. "Beta distribution." Wikipedia. Free Foundation.
  4. 4 NIST Engineering Statistics Handbook. "Beta Distribution." National Institute of Standards and Technology.