[Rd] .Call and to reclaim the memory by allocVector

[Yongchao Ge]

Hi Seth,

Thank you for the suggestion. Because of using .Call (which does not copy 
the value) for both parts of my program, there is no extra copy shown by 
tracemem(). Anyway, the information shown by gc() is very misleading as 
stated by Prof. Ripley, especially after creating and removing a 
couple of large R datasets and applying the function gc() a couple of times.

As shown by "ps aux", there is no "memory leak" from .Call. It's a big 
relief to me. Mysteriously, my program works now for storing the 
intermediate results as a 660M R object. I can run the same function  as 
often as I want. The maximum space taken by the program has never 
exceeded 1.8G as I expected. The disappearance of taking too much memory 
from .Call may be due to a recompile of my C code or a restart of the 
linux or a fresh mind after the weekend.

Thank you and Prof. Ripley for the suggestions. It helps me to stay 
focused.


Yongchao




On Sat, 25 Aug 2007, Seth Falcon wrote:

> Hi Yongchao,
>
> Yongchao Ge <Yongchao.Ge at mssm.edu> writes:
>> Why am I storing a large dataset in the R? My program consist of two
>> parts. The first part is to get the intermediate results, the computation
>> of which takes a lot of time. The second part contains many
>> different functions to manipulate the the intermediate
>> results.
>>
>> My current solution is to save intermediate result in a temporary file,
>> but my final goal is to to save it as an R object. The "memory leak" in
>> .Call stops me from doing this and I'd like to know if I can have a clean
>> solution for the R package I am writing.
>
> There are many examples of packages that use .Call to create large
> objects.  I don't think there is a "memory leak".
>
> One thing that may be catching you up is that because of R's
> pass-by-value semantics, you may be ending up with multiple copies of
> the object on the R side during some of your operations.  I would
> recommend recompiling with --enable-memory-profiling and using
> tracemem() to see if you can identify places where copies of your
> large object are occurring.  You can also take a look at
> Rprof(memory.profile=TRUE).
>
> + seth
>
>
> -- 
> Seth Falcon | Computational Biology | Fred Hutchinson Cancer Research Center
> BioC: http://bioconductor.org/
> Blog: http://userprimary.net/user/