[R-sig-hpc] [R] Determining the maximum memory usage of a function
jgrn at illinois.edu
Thu Jun 20 17:36:21 CEST 2013
Thanks, but I'm looking for something that can be used somewhat
automatically -- the function in question would be user-provided and
passed to my "chunking" algorithm, so in this case it would be the
end-user (not me) who would have to embed these -- would
# my function
... and then taking the max of the tseries output be a reasonable
approach? If so, which of the three outputs (vsize.small vsize.large
nodes) would be best compared against the available memory?
On Thu, Jun 20, 2013 at 10:07 AM, jim holtman <jholtman at gmail.com> wrote:
> What I would do is to use "memory.size()" to get the amount of memory being
> used. Do a call at the beginning of the function to determine the base, and
> then at other points in the code to see what the difference from the base is
> and keep track of the maximum difference. I am not sure if just getting the
> memory usage at the end would be sufficient since there may be some garbage
> collection in between, or you might be creating some large objects and then
> deleting/reusing them. So keep track after large chunks of code to see what
> is happening.
> On Thu, Jun 20, 2013 at 10:45 AM, Jonathan Greenberg <jgrn at illinois.edu>
>> I apologize for the cross-posting between r-help and r-sig-hpc, but I
>> figured this question was relevant to both lists. I'm writing a
>> function to be applied to an input dataset that will be broken up into
>> chunks for memory management reasons and for parallel execution. I am
>> trying to determine, for a given function, what the *maximum* memory
>> usage during its execution is (which may not be the beginning or the
>> end of the function, but somewhere in the middle), so I can "plan" for
>> the chunk size (e.g. have a table of chunk size vs. max memory usage).
>> Is there a trick for determining this?
>> Jonathan A. Greenberg, PhD
>> Assistant Professor
>> Global Environmental Analysis and Remote Sensing (GEARS) Laboratory
>> Department of Geography and Geographic Information Science
>> University of Illinois at Urbana-Champaign
>> 607 South Mathews Avenue, MC 150
>> Urbana, IL 61801
>> Phone: 217-300-1924
>> AIM: jgrn307, MSN: jgrn307 at hotmail.com, Gchat: jgrn307, Skype: jgrn3007
>> R-help at r-project.org mailing list
>> PLEASE do read the posting guide
>> and provide commented, minimal, self-contained, reproducible code.
> Jim Holtman
> Data Munger Guru
> What is the problem that you are trying to solve?
> Tell me what you want to do, not how you want to do it.
Jonathan A. Greenberg, PhD
Global Environmental Analysis and Remote Sensing (GEARS) Laboratory
Department of Geography and Geographic Information Science
University of Illinois at Urbana-Champaign
607 South Mathews Avenue, MC 150
Urbana, IL 61801
AIM: jgrn307, MSN: jgrn307 at hotmail.com, Gchat: jgrn307, Skype: jgrn3007
More information about the R-sig-hpc