Statistics [05]: Statistical Quantities

1 minute read

Published: January 05, 2021

Expectation, Variance and Covariance (Correlation).

Expectation

For discrete variable $X$ , the expectation is

$E(x) = {\displaystyle \sum_{i=1}^\infty x_ip(X=x_i)}$

For continuous variable $X$ , the expectation is

$E(x) = {\displaystyle \int_{-\infty}^\infty xp(x)dx}$

Variance

For variable $X$ , the variance is

$var(X) = E\left((X - E(X))^2\right) = E(X^2) - E^2(X)$

The square root of it is called standard deviation, expressed as

$\sigma(X) = \sqrt{var(X)}$

or $\sigma_X$ .

Covariance

Covariance provides a measure of the strength of the correlation between two or more sets of random variates.

The covariance for two random variates $X$ and $Y$ , each with sample size $N$ , is defined by

$cov(X, Y) = E\left((X-E(X))(Y-E(Y))\right) = E(XY)-E(X)E(Y)$

For unrelated variables,

$cov(X, Y) = E(XY)-E(X)E(Y) = E(X)E(Y) - E(X)E(Y) = 0$

For $X=Y$ ,

$cov(X, Y) = cov(X, X) = E(X^2)-E^2(X) = \sigma_X^2$

which reduces to the usual variance. Therefore, the covariance can also be denoted as

$\sigma_{XY} = cov(X, Y)$

Carrelation is the standardized version of covariance, which is defined as

$cor(X, Y) = \dfrac{cov(X, Y)}{\sigma_X\sigma_Y} = \dfrac{\sigma_{XY}}{\sqrt{\sigma_{XX}\sigma_{YY}}}$

Sample Mean and Variance

Sample Mean

$\bar{x} = \mu = {\displaystyle \dfrac{1}{n}\sum_{i=1}^n x_i}$

Sample Variance

$s^2 = \dfrac{1}{n-1}{\displaystyle \sum_{i=1}^n(x_i-\bar{x})^2}$

Example

Prove that $E(\bar{x}) = \mu, var(\bar{x}) = \dfrac{\sigma^2}{n}, E(s^2) = \sigma^2$ .

Proof.

$E(\bar{x}) = {\displaystyle E\left(\dfrac{1}{n}\sum_{i=1}^n x_i\right) = \dfrac{1}{n}E\left(\sum_{i=1}^n x_i\right) = \dfrac{1}{n}\cdot n\cdot \mu = \mu}$

$var(\bar{x}) = {\displaystyle var\left(\dfrac{1}{n}\sum_{i=1}^n x_i\right) = \dfrac{1}{n^2}var\left(\sum_{i=1}^n x_i\right) = \dfrac{1}{n^2}\cdot n\cdot \sigma^2 = \dfrac{\sigma^2}{n}}$

$E\left(\sum_{i=1}^n (x_i-\bar{x})^2\right) = E\left(\sum_{i=1}^n [(x_i-E(x_i))-(\bar{x}-E(\bar{x}))]^2\right)$

$= \sum_{i=1}^nE\left((x_i-E(x_i))^2\right) + \sum_{i=1}^n\left((\bar{x}-E(\bar{x}))^2\right) - 2\cdot E\left(\sum_{i=1}^n[(x_i-E(x_i))(\bar{x}-E(\bar{x}))]\right)$