Statistics/Point Estimates

From Wikibooks, open books for an open world
< Statistics
Jump to: navigation, search

Point Estimates[edit]

Definition: Suppose a random variable X\, follows a statistical distribution or a law \mathcal{L}_\theta indexed by parameter \theta\,. Then a function g:X\rightarrow\hat{\theta} from the sample space to the parameter space is called a point estimator of \theta\,.

In general, let f(\theta)\, be any function of \theta\,. Then any function from the sample space to the domain of f\, will be called a point estimator of f(\theta)\,.

Definition: If h\, is a point estimator for \theta\,, then for a realization x\, of the random variable X\,, the quantity h(x)\, is called a point estimate of \theta\, and is denoted as \hat\theta\,.

Notice that the estimate \hat\theta\, is a random variable (unlike the true parameter \theta\,), since it depends on X\,.


  1. Suppose X_1,X_2,...,X_n\, follow independent Normal(μ,σ2). Then an estimator for the mean μ is the sample mean \sum X_i \over n\,.
  2. Suppose X_1,X_2,...,X_n\, follow Uniform[θ,θ+1]. Then an estimator for θ is \min X_i\,. Another is \max X_i-1\,. Yet another is \frac{\sum X_i}{n} - \tfrac{1}{2}\,

Notice that the above definition does not restrict the point estimator to only the "good" ones. For example, according to the definition it is perfectly fine to estimate the mean \mu\, in the above example as something absurd, like 10\sum X_i^2+\exp(X_1)\,. This freedom is in the definition is deliberate. In general, however, when we form point estimators we take some measure of goodness into account. It should be kept in mind that the point estimators will always be targeted to be close to the parameter it estimates, intuitively and if possible, formally.

A variety of methods are used to evaluate the effectiveness of a particular estimator. One such measure of goodness is called the bias. Bias is defined as in terms of the closeness of the expectation for the estimator to the actual parameter value. For example if estimating a parameter,  \theta with an estimator,  W(x) then bias(W(x))=E[W(x)-\theta] . When bias is zero, the estimator is called unbiased.

As an example we can show that the sample mean is unbiased for distribution mean. E [\frac{1}{n}\sum X_i ]=\frac{1}{n}E[\sum X_i]= \frac{1}{n}\sum {E[ X_i]}=\frac{n}{n}E[X]=\mu .

Unbiasedness alone means the positive and negative errors balance out, but it is not the only measure of quality. This can be thought of as accuracy, but we might also be concerned about precision. Precision is often measured by the mean squared error, or MSE for short. MSE=E[(W(x)-\theta)^2]