Home › Mathematics › Probability Theory › Probability Distributions
Mathematics Statistics 📅 Updated: Oct 12, 2025 ⏱️ 14 min read ✍️ Dr. Elena Rostova & AI Editorial

Probability Distributions

A probability distribution is a mathematical function that describes the likelihood of obtaining the possible values that a random variable can take. It forms the foundational framework for statistical inference, stochastic modeling, and modern machine learning.

1. Introduction

In probability theory and statistics, a probability distribution specifies how probabilities are distributed over the values of a random variable. It provides a complete characterization of the stochastic behavior of a system, enabling prediction, risk assessment, and hypothesis testing.

Distributions are broadly categorized into discrete (defined on countable sets) and continuous (defined on intervals of real numbers). Each type is described by a corresponding probability mass function (PMF) or probability density function (PDF), alongside a cumulative distribution function (CDF) that applies universally.

Core Concept

The area under a PDF curve between two points represents the probability that the random variable falls within that interval. For discrete variables, the sum of all probabilities equals exactly 1.

2. Discrete vs. Continuous Distributions

Discrete Distributions

Discrete distributions apply to random variables that can only take specific, isolated values (e.g., integers). They are described by a Probability Mass Function \( P(X = x) = p(x) \), where \( 0 \leq p(x) \leq 1 \) and \( \sum_{x} p(x) = 1 \).

Continuous Distributions

Continuous distributions model variables that can take any value within a continuous range. They are described by a Probability Density Function \( f(x) \), where probabilities are obtained via integration: \( P(a \leq X \leq b) = \int_{a}^{b} f(x) \, dx \).

\[ \text{CDF: } F(x) = P(X \leq x) = \begin{cases} \sum_{t \leq x} p(t) & \text{(discrete)} \\ \int_{-\infty}^{x} f(t) \, dt & \text{(continuous)} \end{cases} \]

3. Common Probability Distributions

Bernoulli Discrete

Models a single binary trial (success/failure). Parameters: \( p \in [0,1] \). PMF: \( P(X=k) = p^k(1-p)^{1-k} \).

Binomial Discrete

Counts successes in \( n \) independent Bernoulli trials. PMF: \( \binom{n}{k}p^k(1-p)^{n-k} \). Mean: \( np \), Variance: \( np(1-p) \).

Poisson Discrete

Models rare events over fixed intervals. Parameter \( \lambda > 0 \). PMF: \( \frac{\lambda^k e^{-\lambda}}{k!} \). Mean & Variance: \( \lambda \).

Normal (Gaussian) Continuous

The bell curve governing natural variation. PDF: \( \frac{1}{\sigma\sqrt{2\pi}}e^{-\frac{(x-\mu)^2}{2\sigma^2}} \). Central to the CLT.

Exponential Continuous

Models time between events in a Poisson process. PDF: \( \lambda e^{-\lambda x} \). Memoryless property. Mean: \( 1/\lambda \).

Beta Continuous

Flexible distribution on \( [0,1] \). Widely used in Bayesian statistics as a conjugate prior for Bernoulli/Binomial likelihoods.

4. Mathematical Properties

Every probability distribution is characterized by its moments and shape parameters. The most fundamental include:

  • Expected Value (Mean): \( \mathbb{E}[X] = \sum x p(x) \) or \( \int x f(x) dx \)
  • Variance: \( \text{Var}(X) = \mathbb{E}[(X - \mu)^2] \)
  • Skewness: Measures asymmetry. Positive = right-tailed, negative = left-tailed.
  • Kurtosis: Measures tail heaviness. Leptokurtic distributions exhibit fat tails and higher peak.

Law of Large Numbers & Central Limit Theorem

The LLN guarantees sample means converge to the population mean. The CLT states that the distribution of sample means approaches normality regardless of the underlying distribution, provided \( n \) is sufficiently large.

5. Applications

Probability distributions are indispensable across disciplines:

  • Machine Learning: Bayesian priors, probabilistic graphical models, generative adversarial networks, and variational inference rely heavily on distributional assumptions.
  • Finance & Risk Management: Asset returns often follow heavy-tailed distributions (e.g., Student's t, Log-normal). Value-at-Risk (VaR) calculations depend on tail probability estimation.
  • Quality Control: Manufacturing processes use Poisson and Normal distributions to monitor defect rates and process capability indices.
  • Reliability Engineering: Weibull and Exponential distributions model failure times and system lifespans.
  • Bioinformatics: Gene expression levels and mutation rates are modeled using negative binomial and gamma distributions.

6. References & Further Reading

  1. Feller, W. (1970). An Introduction to Probability Theory and Its Applications, Vol. 1. Wiley.
  2. Shao, J. (2003). Mathematical Statistics. Springer.
  3. Casella, G., & Berger, R. L. (2002). Statistical Inference, 2nd ed. Duxbury.
  4. Aevum Encyclopedia. (2025). "Cumulative Distribution Functions" & "Stochastic Processes".