Statistics [04]: Some Common Continuous Distributions

3 minute read

Published: January 04, 2021

This post will summarize some of the commonly used continuous distributions, including

Uniform distribution
Exponential distribution
Weibull distribution
Normal distribution
$\chi^2$ distribution
Student’s t-distribution
F-distribution
Gamma distribution
Beta Distribution

Uniform Distribution

A uniform distribution, sometimes also known as a rectangular distribution, is a distribution that has constant probability. The probability density function and cumulative distribution function for a continuous uniform distribution on the interval $[a, b]$ are

uniform

Exponential Distribution

Poisson distribution describes the the number of events occurring over a period of time, where the non-overlapping intervals are independent. Exponential distribution describes the distribution of waiting times between successive changes.

Given a Possion distribution with parameter $\nu$ , the distribution of waiting time would be

$D(x) = P(X\leq x) = 1 - P(X>x) = 1 - \dfrac{(\nu x)^0e^{-\nu x}}{0!} = 1 - e^{-\nu x}\ \ (x > 0)$

and the probability distribution function would be

$P(x) = D\text{'}(x) = \nu e^{-\nu x}\ \ (x > 0)$

Like geometric distribution, exponential distribution is memoryless.

Weibull Distribution

Expotential distribution can be seen as a special form of Weibull distribution, whose cumulative distribution function has the following form:

$D(x) = 1 - e^{-\left({x}\text{/}{\beta}\right)^\alpha}\ \ (x > 0)$

and the probability distribution function would be

$P(x) = \alpha\beta^{-\alpha}x^{\alpha-1}e^{-\left({x}\text{/}{\beta}\right)^\alpha}\ \ (x > 0)$

where $\beta$ is called scale parameter, and $\alpha$ is called shape parameter. Depending on its value, Weibull distribution can approximate various, even very different shapes. It is suitable for the characterization of time to failure as well as strength or load.

Normal Distribution

Normal distribution is the most commonly used distribution in our life. It has the following probability density function.

$P(x) = \dfrac{1}{\sigma\sqrt{2\pi}}e^{-(x-\mu)^2\text{/}(2\sigma^2)}$

where $\mu$ and $\sigma^2$ are mean and variance of $X$ respectively.

The so-called standard normal distribution is given by $\mu = 0$ and $\sigma^2 = 1$ in the general normal distribution. An arbitrary normal distribution can be converted to a standard normal distribution by changing variables to $z = (X-\mu)\text{/}\sigma$ , yielding

$P(x)dx = \dfrac{1}{\sqrt{2\pi}}e^{-z^2\text{/}2}dz$

Based on the standard normal distribution, the cumulative distribution can then be found utilizing the lookup table.

Chi-Squared Distribution

If $X_i$ have normal independent distributions with mean 0 and variance 1, then

$Y = {\displaystyle \sum_{i=1}^nX_i^2 \sim \chi_n^2\text{/}\chi^2(n)}$

is a $\chi^2$ distribution with $n$ degrees of freedom.

More generally, if $\chi_i^2$ are independently distributed according to a $\chi^2$ distribution with $n_1, n_2, ..., n_k$ degrees of freedom, then

$\sum_{i=1}^{k}\chi_i^2$

is distributed according to $\chi^2$ with $n = \sum_{i=1}^kn_i$ degrees of freedom.

Student’s t-Distribution

Let $x_1, x_2, ..., x_n$ be independently and identically drawn from the distribution $N(\mu, \sigma^2)$ . Let

$\bar{X} = \dfrac{1}{n}{\displaystyle \sum_{i=1}^nX_i}$

be the sample mean and

$S^2 = \dfrac{1}{n-1}{\displaystyle \sum_{i=1}^n(X_i-\bar{X})^2}$

be the sample variance. The the random variable

$t = \dfrac{\bar{X} - \mu}{S\text{/}\sqrt{n}}$

has a Student’s t-distribution with $n-1$ degrees of freedom.

The only difference of Student’s t-distribution with standard normal distribution is that the $\sigma$ in standard normal distribution is replaced with $S$ . As $n$ increases, Student’s t-distribution approaches the normal distribution.

F-Distribution

F-distribution arises to test whether two observed samples has the same variance. Let $\chi_m^2$ and $\chi_n^2$ be independent $\chi^2$ distribution with $m$ and $n$ degrees of freedom respectively.

The ratio of the two distributions

$F_{m\text{,}n} = \dfrac{\chi_m^2\text{/}m}{\chi_n^2\text{/}n}$

has an F-distribution with degrees of freedom $m$ and $n$ .

Gamma Distribution

Exponential distribution describes the distribution of waiting times between successive changes, whereas Gamma distribution describes the distribution of waiting times until the $h\text{th}$ Possion event happens. Therefore,

$D(x) = 1 - P(X > x) = 1 - {\displaystyle \sum_{k=0}^{h-1}\dfrac{(\nu x)^ke^{-\nu x}}{k!}} = 1 - e^{-\nu x}{\displaystyle \sum_{k=0}^{h-1}\dfrac{(\nu x)^k}{k!}} = 1 - \dfrac{\Gamma(h,\nu x)}{\Gamma(h)}\ \ (x \geq 0)$