[R] R seems to "stall" after several hours on a long series o f analyses... where to start?

[Thomas Lumley]

On Mon, 7 Nov 2005, Fowler, Mark wrote:

> You can test if the problem is accumulation in memory registers, which is
> certainly what this sounds like. Just do a loop over a reasonably small
> number of iterations and store or print the time between each iteration.

If you are worried about memory accumulation it would make more sense to 
print memory use statistics after each iteration rather than the time (eg 
gc(,reset=TRUE))

 	-thomas



> If
> memory accumulation it will run optimally for the first few iterations,
> after which the time will increase noticeably (essentially exponentially,
> hence ultimately freezes up). If this is the problem you may need to switch
> to a For loop approach, or effect the loop as a DOS script to do each
> iteration as a batch job (open/close R each iteration, old bytes can't
> clutter the memory).
>
>
>> 	Mark Fowler
> 		Population Ecology Division
>> 	Bedford Inst of Oceanography
>> 	Dept Fisheries & Oceans
>> 	Dartmouth NS Canada
> 		B2Y 4A2
> 		Tel. (902) 426-3529
> 		Fax (902) 426-9710
> 		Email fowlerm at mar.dfo-mpo.gc.ca
> 		Home Tel. (902) 461-0708
> 		Home Email mark.fowler at ns.sympatico.ca
>
>
> -----Original Message-----
> From: David L. Van Brunt, Ph.D. [mailto:dlvanbrunt at gmail.com]
> Sent: November 7, 2005 7:16 AM
> To: r-help at stat.math.ethz.ch
> Subject: [R] R seems to "stall" after several hours on a long series of
> analyses... where to start?
>
> Not sure where to even start on this.... I'm hoping there's some debugging I
> can do...
>
> I have a loop that cycles through several different data sets (same
> structure, different info), performing randomForest growth and
> predictions... saving out the predictions for later study...
>
> I get about 5 hours in (9%... of the planned iterations.. yikes!) and R just
> freezes.
>
> This happens in interactive and batch mode execution (I can see from the
> ".Rout" file that it gets about 9% through in Batch mode, and about 6% if in
> interactive mode... does that suggest memory problems?)
>
> I'm thinking of re-executing this same code on a different platform to see
> if that's the issue (currently using OS X)... any other suggestions on where
> to look, or what to try to get more information?
>
> Sorry so vague... it's a LOT of code, runs fine without error for many
> iterations, so I didn't think the problem was syntax...
>
> --
> ---------------------------------------
> David L. Van Brunt, Ph.D.
> mailto:dlvanbrunt at gmail.com
>
> 	[[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide!
> http://www.R-project.org/posting-guide.html
>
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
>

Thomas Lumley			Assoc. Professor, Biostatistics
tlumley at u.washington.edu	University of Washington, Seattle