Maximum Likelihood Method for Obtaining Estimators

Topics: Estimator - Statistics

---

When trying to obtain point estimators, we can use the maximum likelihood method, which, given a random sample whose distribution depends on a parameter $\theta$, consists of finding the value of $\theta$ that maximises the sample’s likelihood function.

This value of $\theta$ is called the maximum likelihood estimator and we can use it to build an estimator for $\theta$.

Procedure §

Let $X_1,X_2,\dots ,X_n$ be a random sample whose probability distribution depends on a parameter $\theta$. The maximum likelihood method consists of maximising:

$$\mathrm{L}(\theta \mid x)=\prod _{i=1}^{n}f(x_i\mid \theta )$$

In other words, the method consists of finding the value of $\theta$ that maximises $\mathrm{L}(\theta \mid x)$. According to the theory behind local extrema, this is just finding the $\theta$ that satisfies:

$$\begin{array}{rl}\frac{d}{d\theta }\left[\mathrm{L}(\theta \mid x)\right]& =0\\ \frac{d^2}{d{\theta }^2}\left[\mathrm{L}(\theta \mid x)\right]& <0\end{array}$$

Natural Logarithm §

Sometimes, it might be a good idea to instead maximise:

$$\ln \left[\mathrm{L}(\theta \mid x)\right]$$