[R] Fw: MLE

[(Ted Harding)]

On 08-Oct-08 11:14:39, Ron Michael wrote:
> I made one typo in my previous mail.
> May I ask one statistics related question please? I have one query
> on MLE itself. It's property says that, for large sample size it is
> normally distributed [i.e. asymptotically normal]. On the other hand
> it is "Consistent" as well. My doubt is, how this two asymptotic
> properties exist simultaneously? If it is consistent then
> asymptotically it should collapse to "truth" i.e. for large sample
> size, variance of MLE should be zero. However asymptotic normality
> says, MLE have some distribution and hence variance.
> 
> Can anyone please clarify me? Your help will be highly appreciated.

The false step in your argument is in the following:

 "If it is consistent then asymptotically it should collapse to "truth"
  i.e. for large sample size, variance of MLE should be zero."

The first part would better expressed as:

  If it is consistent then asymptotically it should collapse
  *towards* "truth"

and indeed that is pretty well the definition of "consistent".
More precisely:

1. Decide how close you want the MLE to be to the true value.
   (Say, for example, that this is 0.0001). You're not allowed
   to choose "spot on" (i.e. zero).

2. Decide how sure you want to be that it is that close
   (Say, for example, that you want to be 99.999% sure).
   You're not allowed to choose 100%.

3. Then you can find a sample size N (which may be very large,
   but you are being asymptotic so you can take as much as you need)
   such that, if the sample size it as least N, then

   Probability(|MLE - Truth| < 0.0001) > 0.99999

N, of course, depends on the numbers you chose in (1) and (2).

Not that this does NOT say, anywhere, that the distribution
of the MLE has, for such an N, collapsed strictly to "truth",
i.e. that the variance is zero. All that is implied is that the
variance is very small, sufficiently small for (3) to be true.

And that is all that "consistency" is saying: That, for large
enough N, you can be as sure as you wish (via variance as small
as you need) that the MLE is at least as close as you wish to
the true value. "Consistency" is not saying more than that.

Therefore the second part of your statement:

 "i.e. for large sample size, variance of MLE should be zero."

is not true: you don't attain zero for any large sample size;
you can only get very close. (Except in certain very special
cases -- e.g. sampling 100 different items out of a population
of 100 items, i.e. without replacement, will give you exactly
the value of some quantity calculated on that population).

Hoping that helps!
Ted.

--------------------------------------------------------------------
E-Mail: (Ted Harding) <Ted.Harding at manchester.ac.uk>
Fax-to-email: +44 (0)870 094 0861
Date: 08-Oct-08                                       Time: 13:14:27
------------------------------ XFMail ------------------------------