Moments Method for Obtaining Estimators

Topics: Estimator - Statistics

---

When trying to obtain point estimators, we can use the moments method, which consists of (1) equating population moments with their corresponding sample moments, and then (2) solving the resulting equation(s).

Procedure §

Let $X$ be a random variable whose probability distribution is characterised by a parameter that we want to estimate. To find an estimator for this parameter, we can use the moments method, which consists of solving:

$$\begin{array}{rl}\mathbb{E}[X]& \simeq \frac{1}{n}\sum _{i=1}^n{x}_i=\overline{X}\\ \mathbb{E}[X^2]& \simeq \frac{1}{n}\sum _{i=1}^n{x}_i^2\\ & ⋮\\ \mathbb{E}[X^n]& \simeq \frac{1}{n}\sum _{i=1}^n{x}_i^n\end{array}$$

Continuous Random Variable §

If $X$ is a continuous random variable, then, for every relevant $j$, we’re just solving:

$$\mathbb{E}[X^j]={\int }_{-\infty }^{\infty }{x}^{j}f(x)dx$$

Example §

Let $X$ be a random variable with density function:

$$f(x)=\{\begin{array}{ll}\theta x^{\theta -1}& \text{if }0\le x\le 1\\ 0& \text{otherwise}\end{array}$$

Notice that it depends on a parameter $\theta$. We can use the moments method to find an estimator $\hat{\theta }$ for $\theta$. We begin by finding $X$’s first sample moment (its expected value):

$$\begin{array}{rl}\mathbb{E}[X]& ={\int }_{-\infty }^{\infty }xf(x)dx\\ & ={\int }_0^{1}x\theta x^{\theta -1}dx\\ & ={\int }_0^1\theta x^{\theta }dx\\ & ={\left(\frac{\theta }{\theta +1}{x}^{\theta +1}\right)|}_0^1\\ & =\frac{\theta }{\theta +1}\end{array}$$

Now, we equate it to $X$’s first population moment (its mean). At this point, we are now working with $\hat{\theta }$, the estimator:

$$\begin{array}{rl}\frac{\hat{\theta }}{\hat{\theta }+1}& =\overline{X}\\ \hat{\theta }& =\overline{X}(\hat{\theta }+1)\\ \hat{\theta }& =\overline{X}\hat{\theta }+\overline{X}\\ \hat{\theta }-\overline{X}\hat{\theta }& =\overline{X}\\ \hat{\theta }(1-\overline{X})& =\overline{X}\\ \hat{\theta }& =\frac{\overline{X}}{1-\overline{X}}\end{array}$$

That’s all. We have found that $\hat{\theta }=\left(\frac{\overline{X}}{1-\overline{X}}\right)$ is an estimator for $\theta$, the parameter that $X$’s distribution depends on.