[R] (Off topic.) Observed Fisher information.

[Duncan Murdoch]

Rolf Turner wrote:
> I have been building an R function to calculate the ***observed***
> (as opposed to expected) Fisher information matrix for parameter
> estimates in a rather complicated setting.  I thought I had it
> working, but I am getting a result which is not positive definite.
> (One negative eigenvalue.  Out of 10.)
> 
> Is it the case that the observed Fisher information must be positive
> definite --- thereby indicating for certain that there are errors in
> my code --- or is it possible for such a matrix not to be pos. def.?

If you are at the maximum, it should be at least positive indefinite (or 
nonnegative definite).  Numerical errors could make zero (or small 
positive) eigenvalues look negative.  It's also possible that your 
optimization has missed the maximum by a bit, and then it could have 
truly negative eigenvalues.

In either case I'd expect the negative eigenvalues to be small.
> 
> It seems to me that if the log likelihood surface is ***not*** well
> approximated by a quadratic in a neighbourhood of the maximum, then
> it might well be that case that the observed information could fail
> to be positive definite.  Is this known/understood?  Can anyone point
> me to appropriate places in the literature?

If there is a true negative eigenvalue, then moving along that 
eigenvector should increase the likelihood, so I don't think even 
irregular problems could have true negative eigenvalues at the MLE. The 
problem there would be that a zero score and a positive definite 
observed information matrix don't necessarily imply you're at even a 
local maximum.

Duncan Murdoch