[R-sig-ME] prevent for-loop from stopping after an lmer warning/error
Ben Bolker
bbolker at gmail.com
Sat Mar 24 02:56:54 CET 2018
It's possible that lmerTest is doing slightly weird things with trying
to catch the errors itself.
Rune Haubo is working on a revamped version of lmerTest
(https://rdrr.io/github/runehaubo/lmerTestR/); it *might* be worth
taking a try at installing that to see if it behaves better?
On Fri, Mar 23, 2018 at 5:57 PM, Aleksander Adam Glowka
<aglowka at stanford.edu> wrote:
> Hi Ben,
>
>
> Thank you for pointing this out. Indeed the loops only stops after errors, not warnings.
>
>
> By the way, I am still stuck with my issue. I've made some progress with using tryCatch() as suggested, but still have not resolved my issue. As far as I can tell, currently there are two issues with my code. First, when I query the returned exception in `tryCatch()`, e.g. `poss.err.mod1` I get the following error:
>
> <simpleError in tryCatchList(expr, classes, parentenv, handlers): argument "expr" is missing, with no default>
>
> Still, the warnings are correctly written to file, which is puzzling. Second, when no error is detected, the code seems to be ignored (i.e. no model summaries are written to file). So the loop is running vacuously, skipping errors but apparently handling them and doing nothing else. If anyone has any advice on how to debug this, please let me know!
>
> # divert console stream to file
> options(warn=1)
> wngs=file("warnings_log.txt",open="w+",blocking=TRUE)
> sink(wngs,type="message")
>
> for(iter in 1:1000){
>
> data = read.csv(paste("sample", iter,".csv", sep=""), header=TRUE)
>
> #error handling case #1
> poss.err.mod1 = tryCatch(
>
> lmer(y ~ x1 + (1|ranef), data = data, REML = FALSE), #try part
> error=function(e) e #catch part
> )
>
> #error handling case #2
> poss.err.mod2 = tryCatch(
>
> lmer(y ~ x1 + x2 + (1|ranef), data = data, REML = FALSE), #try part
> error=function(e) e #catch part
> )
>
> #real work #1
> if(!inherits(poss.err.mod1, "error")){
>
> mod1 = lmer(y ~ x1 + (1|ranef1), data = data, REML = FALSE)
> write.csv(summmary$mod2, paste("mod2.sum_", iter, ".csv", sep="")
>
> }else{
>
> write(unlist(mod1 at optinfo$conv$lme4$messages), paste("mod1.errors_", iter, ".txt", sep=""))
> }
>
> #real work #2
> if(!inherits(poss.err.mod2, "error")){
>
> mod2 = lmer(y ~ x1 + x2 + (1|ranef1), data = data, REML = FALSE)
> write.csv(summmary$mod2, paste("mod2.sum_", iter, ".csv",)
>
> }else{
>
> write(unlist(mod2 at optinfo$conv$lme4$messages), paste("mod1.errors_", iter, ".txt", sep=""))
> }
>
> cat("Iteration", iter, "completed!\n")
> }
>
> #close log file & restore warnings stream to console
> closeAllConnections()
>
> ________________________________
> From: Ben Bolker <bbolker at gmail.com>
> Sent: Friday, March 23, 2018 10:42:47 AM
> To: Andrew Robinson
> Cc: Aleksander Adam Glowka; Help Mixed Models
> Subject: Re: [R-sig-ME] prevent for-loop from stopping after an lmer warning/error
>
> Is your loop really stopping after warnings (that would be
> surprising), not just after errors?
>
> On Thu, Mar 22, 2018 at 3:42 PM, Andrew Robinson <apro at unimelb.edu.au> wrote:
>> ?try
>>
>> ?tryCatch
>>
>> Cheers,
>>
>> Andrew
>>
>>
>> On 23 March 2018 at 06:18, Aleksander Adam Glowka <aglowka at stanford.edu>
>> wrote:
>>
>>> Hi all,
>>>
>>> I'm fitting mixed-effects regression models to bootstrap samples in a
>>> for-loop and writing a subset of model results to file. I diverted the
>>> stream from the console to a text file so I can keep track any warnings or
>>> errors. For some samples the model does not converge and a warning is
>>> issued. For other samples there is an error in calculation of the
>>> Satterthwaite's approximation. The problem is that my loop iteration is
>>> aborted for some reason after a warning is issued. Do you know why this
>>> might be happening and how I can make the loops continue to the next
>>> iteration after an error or a warning?
>>>
>>> Below I've included an abridged version of my script and the warnings and
>>> errors I get. I'd be grateful for any advice you may have!
>>>
>>> Thank you,
>>>
>>> Aleksander Glowka
>>> PhD Candidate
>>> Department of Linguistics
>>> Stanford University
>>>
>>> #packages
>>> require(lme4)
>>> require(lmerTest)
>>>
>>> setwd(path0)
>>> source("lmer-data-extract-boot-fnc.R") #selected lmer results extractor
>>>
>>> # divert messages stream to file, so you can log warnings and errors
>>> options(warn=1)
>>> wngs=file("warnings_log.txt",open="w+",blocking=TRUE)
>>> sink(wngs,type="message")
>>>
>>> for(iter in 1:2000){
>>>
>>> setwd(path1)
>>>
>>> data = read.csv(paste("sample", iter,".csv", sep=""), header=TRUE)
>>>
>>> #log model number to file so that any potential warnings/errors appear
>>> underneath
>>> message(paste("holistic model #", iter, sep=""))
>>>
>>> mod = lmer(y ~ x1 +
>>> x2 +
>>> x3 +
>>> (1|ranef1) +
>>> (x1|ranef2),
>>> data = data,
>>> REML = FALSE)
>>>
>>> #write model results to file
>>> setwd(path2)
>>> write.csv(lmer.data.extract.boot(mod, iter), paste("mod.fixef_", iter,
>>> ".csv", sep=""), row.names=TRUE) #fixed effects
>>> write.csv(lmer.ranef.data.extract.boot(holistic.mod, iter),
>>> paste("mod.ranef_", iter, ".csv", sep=""), row.names=FALSE)
>>> write.csv(lmer.optim.data.extract.boot(holistic.mod, iter),
>>> paste("mod.optim_", iter, ".csv", sep=""), row.names=TRUE)
>>> write.csv(as.data.frame(holistic.mod at optinfo$derivs$Hessian),
>>> paste("mod.hessian_", iter, ".csv", sep=""))
>>> write.csv(summary(holistic.mod)$resid, paste("mod.resid_", iter,
>>> ".csv", sep=""), row.names=FALSE)
>>> write(unlist(holistic.mod at optinfo$conv$lme4$messages),
>>> paste("mod.warnings_", iter, ".txt", sep=""))
>>>
>>> cat("Iteration", iter, "completed!\n")
>>>
>>> }
>>>
>>> #close log file & restore warnings stream to console
>>> closeAllConnections()
>>>
>>> Errors:
>>>
>>> Warning in checkConv(attr(opt, "derivs"), opt$par, ctrl =
>>> control$checkConv, :
>>> unable to evaluate scaled gradient
>>> Warning in checkConv(attr(opt, "derivs"), opt$par, ctrl =
>>> control$checkConv, :
>>> Model failed to converge: degenerate Hessian with 1 negative eigenvalues
>>>
>>> Error in calculation of the Satterthwaite's approximation. The output of
>>> lme4 package is returned
>>> summary from lme4 is returned
>>> some computational error has occurred in lmerTest
>>>
>>>
>>>
>>>
>>> [[alternative HTML version deleted]]
>>>
>>> _______________________________________________
>>> R-sig-mixed-models at r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>>>
>>>
>>
>>
>> --
>> Andrew Robinson
>> Director, CEBRA, School of BioSciences
>> Reader & Associate Professor in Applied Statistics Tel: (+61) 0403 138 955
>> School of Mathematics and Statistics Fax: (+61) 03
>> 8344 4599
>> University of Melbourne, VIC 3010 Australia
>> Email: apro at unimelb.edu.au
>> Website: http://www.ms.unimelb.edu.au/~andrewpr
>>
>> [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> R-sig-mixed-models at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>
> [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-mixed-models at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
More information about the R-sig-mixed-models
mailing list