# James–Stein estimator

The James–Stein estimator is a biased estimator of the mean of Gaussian random vectors. It can be shown that the James–Stein estimator dominates the "ordinary" least squares approach, i.e., it has lower mean squared error on average. It is the best-known example of Stein's phenomenon.

## Improvements

The basic James–Stein estimator has the peculiar property that for small values of $\|{\mathbf {y} }-{\boldsymbol {\nu }}\|,$ the multiplier on ${\mathbf {y} }-{\boldsymbol {\nu }}$ is actually negative. This can be easily remedied by replacing this multiplier by zero when it is negative. The resulting estimator is called the positive-part James–Stein estimator and is given by

${\widehat {\boldsymbol {\theta }}}_{JS+}=\left(1-{\frac {(m-2)\sigma ^{2}}{\|{\mathbf {y} }-{\boldsymbol {\nu }}\|^{2}}}\right)^{+}({\mathbf {y} }-{\boldsymbol {\nu }})+{\boldsymbol {\nu }}.$ This estimator has a smaller risk than the basic James–Stein estimator. It follows that the basic James–Stein estimator is itself inadmissible.

It turns out, however, that the positive-part estimator is also inadmissible. This follows from a general result which requires admissible estimators to be smooth.

## Extensions

The James–Stein estimator may seem at first sight to be a result of some peculiarity of the problem setting. In fact, the estimator exemplifies a very wide-ranging effect, namely, the fact that the "ordinary" or least squares estimator is often inadmissible for simultaneous estimation of several parameters.{{ safesubst:#invoke:Unsubst||date=__DATE__ |\$B= {{#invoke:Category handler|main}}{{#invoke:Category handler|main}}[citation needed] }} This effect has been called Stein's phenomenon, and has been demonstrated for several different problem settings, some of which are briefly outlined below.

{{#invoke:Category handler|main}}{{#invoke:Category handler|main}}[citation needed] }}

${\widehat {\boldsymbol {\theta }}}_{JS}=\left(1-{\frac {(m-2){\frac {\sigma ^{2}}{n}}}{\|{\overline {\mathbf {y} }}\|^{2}}}\right){\overline {\mathbf {y} }},$ where ${\overline {\mathbf {y} }}$ is the $m$ -length average of the $n$ observations.
• The work of James and Stein has been extended to the case of a general measurement covariance matrix, i.e., where measurements may be statistically dependent and may have differing variances. A similar dominating estimator can be constructed, with a suitably generalized dominance condition. This can be used to construct a linear regression technique which outperforms the standard application of the LS estimator.
• Stein's result has been extended to a wide class of distributions and loss functions. However, this theory provides only an existence result, in that explicit dominating estimators were not actually exhibited. It is quite difficult to obtain explicit estimators improving upon the usual estimator without specific restrictions on the underlying distributions.