[R] fitdistr, mle's and gamma distribution

[Ben Bolker]

  PS.  11 MILLION entries??

On Tue, 30 Sep 2003, Ben Bolker wrote:

> 
>   Spencer Graves's suggestion of using shape and scale parameters on a log 
> scale is a good one.
> 
>   To do specifically what you want (check values for which the objective 
> function is called and see what happens) you can do the following 
> (untested!), which makes a local copy of dgamma that you can mess with:
> 
> dgamma.old <- dgamma
> dgamma <- function(x,shape,rate,...) {
>    d <- dgamma.old(x,shape,rate,...)
>    cat(shape,rate,d,"\n")
>    return(d)
> }
> 
> 
> On Tue, 30 Sep 2003, Lourens Olivier Walters wrote:
> 
> > Dear R Users, 
> > 
> > I am trying to obtain a best-fit analytic distribution for a dataset
> > with 11535459 entries. The data range in value from 1 to 300000000. I
> > use: fitdistr(data, "gamma") to obtain mle's for the parameters.
> >  
> > I get the following error:
> > 
> > Error in optim(start, mylogfn, x = x, hessian = TRUE, ...) :
> >         non-finite finite-difference value [1]
> > 
> > And the following warnings:
> > 
> > NaNs produced in: dgamma(x, shape, scale, log)
> > 
> > I have the same problem with the exponential distribution, but the
> > lognormal and weibull distributions don't have this problem. 
> > 
> > I suspect the problem has to do with:
> > 
> > "It means that in computing derivatives by finite differencing, one 
> > of the values is NA, +Inf or -Inf. Put some print values in 
> > your objective function and find out why it is giving a non-finite
> > value." - quote by Prof Ripley from r-help
> > 
> > as from the warnings it is clear that some of the values obtained during
> > maximum likelihood computations are NaN. I cannot however print values
> > of the objective function, as in my case it is dgamma which is called by
> > fitdistr, over which I don't have control.  
> > 
> > Can anyone please point me in the right direction?
> > 
> > Lourens
> > 
> > 
> 
> 

-- 
620B Bartram Hall                            bolker at zoo.ufl.edu
Zoology Department, University of Florida    http://www.zoo.ufl.edu/bolker
Box 118525                                   (ph)  352-392-5697
Gainesville, FL 32611-8525                   (fax) 352-392-3704