[R-sig-ME] Is there a way to deal with errors such as this?

[Ben Bolker]

PS that output was obtained by setting verbose=1000 when running glmer ...

On Sat, Dec 14, 2019 at 9:11 PM Ben Bolker <bbolker using gmail.com> wrote:
>
>   I would sadly agree with this assessment as well.  It would be good
> to collect a set of models that fail in this way and use them to
> debug/figure out what the problem is: the PIRLS loop is implemented
> entirely in C++, which makes it relatively painful to debug. (It might
> also be possible to make some progress by using the "pureR"
> implementation that Steve Walker wrote a few years ago ...)
>
>  One little clue is that (as I vaguely remembered/suspected) is that
> the proximal problem is that iterations that result in NaN are not
> dealt with well. Next step: figuring out why the NaNs are generated in
> the first place ...
>
> *** pwrssUpdate step 1
> pdev=182574; delu_min: -158.148; delu_max: 310.47; delb_min: -7.05005;
> delb_max: 60.6557
>
> pwrssUpdate: Entering step halving loop
> step-halving iteration 0:  pdev=45331.6; delu_min: -79.2949; delu_max:
> 155.422; delb_min: -3.50962; delb_max: 30.2776
> step-halving iteration 1:  pdev=11437.5; delu_min: -39.8683; delu_max:
> 77.8977; delb_min: -1.73941; delb_max: 15.0885
> *** pwrssUpdate step 2
> pdev=nan; delu_min: nan; delu_max: nan; delb_min: nan; delb_max: nan
>
> pwrssUpdate: Entering step halving loop
> step-halving iteration 0:  pdev=nan; delu_min: nan; delu_max: nan;
> delb_min: nan; delb_max: nan
> step-halving iteration 1:  pdev=nan; delu_min: nan; delu_max: nan;
> delb_min: nan; delb_max: nan
> step-halving iteration 2:  pdev=nan; delu_min: nan; delu_max: nan;
> delb_min: nan; delb_max: nan
> step-halving iteration 3:  pdev=nan; delu_min: nan; delu_max: nan;
> delb_min: nan; delb_max: nan
> step-halving iteration 4:  pdev=nan; delu_min: nan; delu_max: nan;
> delb_min: nan; delb_max: nan
> step-halving iteration 5:  pdev=nan; delu_min: nan; delu_max: nan;
> delb_min: nan; delb_max: nan
> step-halving iteration 6:  pdev=nan; delu_min: nan; delu_max: nan;
> delb_min: nan; delb_max: nan
> step-halving iteration 7:  pdev=nan; delu_min: nan; delu_max: nan;
> delb_min: nan; delb_max: nan
> step-halving iteration 8:  pdev=nan; delu_min: nan; delu_max: nan;
> delb_min: nan; delb_max: nan
> step-halving iteration 9:  pdev=nan; delu_min: nan; delu_max: nan;
> delb_min: nan; delb_max: nan
> Error in pwrssUpdate(pp, resp, tol = tolPwrss, GQmat = GHrule(0L),
> compDev = compDev,  :
>   (maxstephalfit) PIRLS step-halvings failed to reduce deviance in pwrssUpdate
>
> On Sat, Dec 14, 2019 at 8:50 PM Jonathan Judge <bachlaw01 using outlook.com> wrote:
> >
> > Rolf:
> >
> > Sadly, that error has always been the kiss of death for me no matter what I tried. As parameterized, the model probably just won’t optimize in lme4. If you want to stick with pseudo-frequentist inference, you could try glmmTMB, which is developed by some of the same folks (well, one in particular) and it may work using the same notation you are used to. But that error message usually means you need to try a different package.
> >
> > Jonathan
> >
> > Sent from my iPhone
> >
> > > On Dec 14, 2019, at 7:21 PM, Rolf Turner <r.turner using auckland.ac.nz> wrote:
> > >
> > > 
> > > By "this" I mean as demonstrated in the following code.  The file testData.txt is attached.
> > >
> > > X <- dget("testData.txt")
> > > library(lme4)
> > > fit <- glmer(cbind(Dead,Alive) ~ (0+Trt)/Dose + (Dose | Rep),
> > >             data=X,family=binomial(link="probit"))
> > >
> > > The foregoing falls over with the (rather complex) error message:
> > >
> > >> Error in pwrssUpdate(pp, resp, tol = tolPwrss, GQmat = GHrule(0L), compDev = compDev,  :   (maxstephalfit) PIRLS step-halvings failed to reduce deviance in pwrssUpdate
> > >
> > > I note that mixed_model() from GLMMadaptive seems to be able to deal with these data and this model:
> > >
> > > library(GLMMadaptive)
> > > fit <- mixed_model(fixed=cbind(Dead,Alive) ~ (0+Trt)/Dose,
> > >                   random=~Dose | Rep,
> > >                   data=X,family=binomial(link="probit"))
> > >
> > > The foregoing runs without complaint.
> > >
> > > I am applying the glmer() model in the context of doing some fairly elaborate simulations (in which "X" gets randomly generated) and the error causes the simulations to crash unpleasantly.  So I would *like* a magic incantation that I can apply in an automated way to prevent the
> > > error from occurring.
> > >
> > > I can of course wrap function calls up in try() and if there is an error
> > > generate a new data set and go again.  However I'm a little apprehensive
> > > that this might bias the results of the simulations in some way.
> > >
> > > I could also switch to using mixed_model(), but would prefer to stick with the devil I know (i.e. glmer()) for the sake of consistency with other work that I have done.  (And who knows?  Maybe in the course of the simulations mixed_model() might fall over too, from time to time.)
> > >
> > > I'd appreciate any avuncular (or materteral) advice that anyone might be inclined to offer.
> > >
> > > cheers,
> > >
> > > Rolf
> > >
> > > --
> > > Honorary Research Fellow
> > > Department of Statistics
> > > University of Auckland
> > > Phone: +64-9-373-7599 ext. 88276
> > > <testData.txt>
> > > _______________________________________________
> > > R-sig-mixed-models using r-project.org mailing list
> > > https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-sig-mixed-models&data=02%7C01%7C%7C014e497b7e884f35d78708d780fd0a94%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637119696600482811&sdata=m%2Binqw1ft8rZ1w37Q7McwqOXJ4ws1%2BtcstsRZKts8Lc%3D&reserved=0
> > _______________________________________________
> > R-sig-mixed-models using r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models