[R] for loop performance

[Barth B. Riley]

Thanks Martin,  this is very helpful.
Barth

-----Original Message-----
From: Martin Morgan [mailto:mtmorgan at fhcrc.org]
Sent: Thursday, April 14, 2011 11:04 AM
To: Barth B. Riley
Subject: Re: [R] for loop performance

On 04/14/2011 07:12 AM, Barth B. Riley wrote:
> Hi Martin
>
> Question--when variables are defined within a function (and not
> returned by the function) the memory they used is deallocated when the
> function returns, correct? That would seem to make sense but since my
> R code is not compiled, I'm not sure how R handles local variables
> within a function.

The answer depends on what the function returns, and perhaps other things. For instance

f = function() { a = 1; b = 2; a }
x = f()

'a' and 'b' created inside f() will be garbage collected; there is no way to access their value after f() returns. But in

g = function() { a = 1; function() {} }
y = g()

(i.e., g a function that returns a function) y is now a function, functions have environments (the one in which they were created), and the environment can accessed, so 'a' is still accessible

 > environment(y)[["a"]]
[1] 1

and is not available for garbage collection until y is removed.

Martin

>
> Barth
>
>
> -----Original Message----- From: Martin Morgan
> [mailto:mtmorgan at fhcrc.org] Sent: Thursday, April 14, 2011 8:57 AM
> To: Barth B. Riley Cc: r-help at r-project.org Subject: Re: [R] for loop
> performance
>
> On 04/13/2011 02:55 PM, Barth B. Riley wrote:
>> Dear list
>>
>> I am running some simulations in R involving reading in several
>> hundred datasets, performing some statistics and outputting those
>> statistics to file. I have noticed that it seems that the time it
>> takes to process of a dataset (or, say, a set of 100 datasets) seems
>> to take longer as the simulation progresses. Has anyone else noticed
>> this? I am curious to know if this has to do with how R processes
>> code in loops or if it might be due to memory usage issues (e.g.,
>> repeatedly reading data into the same matrix).
>
> Hi Barth
>
> The 'it gets slower' symptom is often due to repeatedly 'growing by 1'
> a list or  other data structure, e.g.,
>
> m = matrix(100000, 100) n = 20000 result = list() system.time(for (i
> in seq_len(n)) result[[i]] = m)
>
> versus 'pre-allocate and fill'
>
> result = vector("list", n) system.time(for (i in seq_len(n))
> result[[i]] = m)
>
> The former causes 'result' to be copied on each new assignment, and
> the size of the copy gets larger each time.
>
>>
>> Thanks in advance
>>
>> Barth
>>
>> PRIVILEGED AND CONFIDENTIAL INFORMATION This transmittal and any
>> attachments may contain PRIVILEGED AND CONFIDENTIAL information and
>> is intended only for the use of the addressee. If you are not the
>> designated recipient, or an employee or agent authorized to deliver
>> such transmittals to the designated recipient, you are hereby
>> notified that any dissemination, copying or publication of this
>> transmittal is strictly prohibited. If you have received this
>> transmittal in error, please notify us immediately by replying to the
>> sender and delete this copy from your system. You may also call us at
>> (309) 827-6026 for assistance.
>>
>> ______________________________________________
>> R-help at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the
>> posting guide http://www.R-project.org/posting-guide.html and provide
>> commented, minimal, self-contained, reproducible code.
>
>
> -- Computational Biology Fred Hutchinson Cancer Research Center 1100
> Fairview Ave. N. PO Box 19024 Seattle, WA 98109
>
> Location: M1-B861 Telephone: 206 667-2793
>
> PRIVILEGED AND CONFIDENTIAL INFORMATION This transmittal and any
> attachments may contain PRIVILEGED AND CONFIDENTIAL information and is
> intended only for the use of the addressee. If you are not the
> designated recipient, or an employee or agent authorized to deliver
> such transmittals to the designated recipient, you are hereby notified
> that any dissemination, copying or publication of this transmittal is
> strictly prohibited. If you have received this transmittal in error,
> please notify us immediately by replying to the sender and delete this
> copy from your system. You may also call us at
> (309) 827-6026 for assistance.


--
Computational Biology
Fred Hutchinson Cancer Research Center
1100 Fairview Ave. N. PO Box 19024 Seattle, WA 98109

Location: M1-B861
Telephone: 206 667-2793

PRIVILEGED AND CONFIDENTIAL INFORMATION
This transmittal and any attachments may contain PRIVILEGED AND
CONFIDENTIAL information and is intended only for the use of the
addressee. If you are not the designated recipient, or an employee
or agent authorized to deliver such transmittals to the designated
recipient, you are hereby notified that any dissemination,
copying or publication of this transmittal is strictly prohibited. If
you have received this transmittal in error, please notify us
immediately by replying to the sender and delete this copy from your
system. You may also call us at (309) 827-6026 for assistance.