[R] R crashing inconsistently within for loops

[Ben Bolker]

Steve Powers <powers_s <at> nd.edu> writes:

> 
> Hello,
> 
> This one has been bugging me for a long time and I have never found a
> solution. I am using R version 2.15.1 but it has come 
> up in older versions of R I have used over the past 2-3 years.
> 
> Q: Am I wrong to expect that R should handle hundreds of iterations of the
> base model or statistical functions, embedded within for loops, in one
> script run? I have found that when I write scripts that do this, sometimes
> they have a tendency to crash, seemingly unpredictably.
> 
> For example, one problem script of mine employs glm and gls about a hundred
> different times, and output files are being written at the end of each
> iteration. I have used my output files to determine that the crash cause is
> not consistent (R never fails at the same iteration). Note that the data are
> fixed here (no data generation or randomization steps, so that is not the
> issue). But it is clear that scripts with larger numbers of iterations are
> more likely to produce a crash.
> 
> And a year or two ago, I had a seemingly stable R script again with for
> looped model fits, but discovered this script was prone to crashing when I
> ran it on a newer PC. Because the new PC also seemed to be blazing through R
> code absurdly fast, I tried adding a short "fluff" procedure at the end of
> each iteration that required a few seconds of processing time. Low and
> behold, when I added that, the script stopped crashing (and each iteration
> of course took longer). I still don't understand why that fixed things.

  All of the advice given so far is useful (I think), but I thought I would
chime in and answer one of your questions, which is that this is indeed
surprising, especially if you're using only base packages.  I and many
other people routinely run thousands of iterations of these types of
analyses with no problem.  If (as Brian Ripley suggested) "crash" just
means that some of your scripts stop with errors in some cases, then
that's *not* surprising -- there are lots of ways to get glm() and gls()
to give errors with slightly weird data sets.  However, having R actually
crash (i.e. the whole R session 'terminates abnormally', in the words
of the posting guide) is quite a bit more unusual, and (if you are only
using base R, not any contributed packages that may call compiled code
in bad ways [or calling your own compiled code]) *always* constitutes
a bug.  Nondeterministic behavior in a deterministic function (i.e.
no random number generation) is also unusual/surprising.

 The key here is finding a reproducible example, which can be tough
for these kinds of problems.  But it sounds like you have one, so
if you can trim it down to a manageable size
 (see http://tinyurl.com/reproducible-000 for tips on creating
reproducible examples), and give full information about your system
(as suggested by another poster), this would be of great interest,
especially to the people who hang out on r-devel at r-project.org .
My prior probabilities are fairly strongly on the problem being
something flaky about your system, but helping to identify these
kinds of bugs in R is a community service ...