Great suggestions, all.

I do have a timer in there, and it looks like the time to complete a loop is
not increasing as it goes. From your comments, I take it that suggests there
is not a memory leak. I could try scripting the loop from the shell, rather
than R, to see if that works, but will do that as a last resort as it will
require a good deal of re-writing (the loop follows some "setup" code that
builds a pretty large data set... the loop then slaps several new columns on
a copy of that data set, and analyses that...)

I'll still try the other platform as well, see if the same problem occurs
there.

On 11/7/05, jim holtman <jholtman@gmail.com> wrote:
>
> Here is some code that I use to track the progress of my scripts. This
> will print out the total cpu time and the memory that is being used. You
> call it with 'my.stats("message")' to print out "message" on the console.
>  Also, have you profiled your code to see where the time is being spent?
> Can you break it up into multiple runs so that you can start with a "fresh"
> version of memory?
>  ======script===========
> "my.stats" <- local({
> # local variables to hold the last times
> # first two variables are the elasped and CPU times from the last report
> lastTime <- lastCPU <- 0
> function(text = "stats", reset=F)
> {
> procTime <- proc.time()[1:3] # get current metrics
> if (reset){ # setup to mark timing from this point
> lastTime <<- procTime[3]
> lastCPU <<- procTime[1] + procTime[2]
> } else {
> cat(text, "-",sys.call(sys.parent())[[1]], ": <",
> round((procTime[1] + procTime[2]) - lastCPU,1),
> round(procTime[3] - lastTime,1), ">", procTime,
> " : ", round(memory.size()/2.^20., 1.), "MB\n")
> invisible(flush.console()) # force a write to the console
> }
> }
> })
>  ========= here is some sample output=========
> > my.stats(reset=TRUE) # reset counters
> > x <- runif(1e6) # generate 1M random numbers
> > my.stats('random')
> random - my.stats : < 0.3 31.8 > 96.17 11.7 230474.9 : 69.5 MB
> > y <- x*x+sqrt(x) # just come calculation
> > my.stats('calc')
> calc - my.stats : < 0.7 71.2 > 96.52 11.74 230514.3 : 92.4 MB
> >
>  You can see that memory is growing. The first number is the CPU time and
> the second (in <>) is the elapsed time.
>  HTH
>
>
>  On 11/7/05, David L. Van Brunt, Ph.D. <dlvanbrunt@gmail.com> wrote:
>
> > Not sure where to even start on this.... I'm hoping there's some
> > debugging I
> > can do...
> >
> > I have a loop that cycles through several different data sets (same
> > structure, different info), performing randomForest growth and
> > predictions... saving out the predictions for later study...
> >
> > I get about 5 hours in (9%... of the planned iterations.. yikes!) and R
> > just
> > freezes.
> >
> > This happens in interactive and batch mode execution (I can see from the
> > ".Rout" file that it gets about 9% through in Batch mode, and about 6%
> > if in
> > interactive mode... does that suggest memory problems?)
> >
> > I'm thinking of re-executing this same code on a different platform to
> > see
> > if that's the issue (currently using OS X)... any other suggestions on
> > where
> > to look, or what to try to get more information?
> >
> > Sorry so vague... it's a LOT of code, runs fine without error for many
> > iterations, so I didn't think the problem was syntax...
> >
> > --
> > ---------------------------------------
> > David L. Van Brunt, Ph.D.
> > mailto: dlvanbrunt@gmail.com
> >
> > [[alternative HTML version deleted]]
> >
> > ______________________________________________
> > R-help@stat.math.ethz.ch mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
> >
> >
>
>
>
> --
> Jim Holtman
> Cincinnati, OH
> +1 513 247 0281
>
> What the problem you are trying to solve?




--
---------------------------------------
David L. Van Brunt, Ph.D.
mailto:dlvanbrunt@gmail.com

	[[alternative HTML version deleted]]


