Central Limit Theorem

Topics: Statistics - Probability - Standard Normal Distribution

---

(theorem)

Let $X_1,X_2,\dots ,X_n$ be a sequence of iid random variables (random sample) such that $\forall X_i$:

$$\begin{array}{rl}\mathbb{E}[X_i]& =\mu \\ \mathrm{Var}(X_i)& ={\sigma }^2\ne 0\end{array}$$

Let $S_n$ be the sum of all of these random variables:

$S_n=X_1+X_2+\cdots +X_n$

From the properties of the expected value and those of variance, it follows that $\mathbb{E}[S_n]=n\mu$ and $\mathrm{Var}(S_n)={\sigma }^{2}n$. Now, let:

$$Z_n=\frac{S_n-n\mu }{\sigma \sqrt{n}}$$

This is standarisation, so $\mathbb{E}[Z_n]=0$ and $\mathrm{Var}(Z_n)=1$.

The central limit theorem tells us that when $n\to \infty$, the sequence $Z_1,Z_2,\dots ,Z_n$ converges (in probability distribution) towards a standard normal distribution $Z\sim \mathcal{N}(0,1)$.

Usefulness §

(observation)

The usefulness of the central limit theorem becomes evident when, given $Z\sim \mathcal{N}(0,1)$, we use it to affirm that:

$$\underset{n\to \infty }{\lim }\mathbb{P}\left(\frac{S_n-n\mu }{\sigma \sqrt{n}}\le z\right)=\mathbb{P}(Z\le z)$$

That is, the (standardised) sum of $n$ iid random variables with any distribution approaches a standard normal distribution when $n\to \infty$.

This allows us to use a normal standard distribution to approximate such a sum when $n$ is large enough.

(observation)

If we set ${\overline{X}}_n=\frac{S_n}{n}$, then we can write:

$$\underset{n\to \infty }{\lim }\mathbb{P}\left(\frac{{\overline{X}}_n-\mu }{\frac{\sigma }{\sqrt{n}}}\le z\right)=\mathbb{P}(Z\le z)$$