[Bioc-devel] IntAE buffers needs to be freed or not??

[Hervé Pagès]

Hi Ge,

On 07/22/2013 02:59 AM, Ge Tan wrote:
> Hi Hervé,
>
> Thank you very much!
> So how about Calloc? Does it run much slower than the direct malloc() too?

Calloc() is just a wrapper to calloc() so it's also user-controlled
memory (like malloc()). The difference with malloc() is that it sets the
memory to zero so I expect it to be almost as fast as malloc(), and
therefore, also much faster than R_alloc().

Note that the auto-extending buffers (like IntAE) in IRanges don't need
to be set to zero so the choice is really between malloc/realloc (user-
controlled memory) and R_alloc/S_realloc (transient memory).

Finally I should be a little bit more precise. In addition to malloc()
being much faster than R_alloc(), realloc() is also much faster than
S_realloc(). What I observed (last time I checked) was that using
malloc/realloc was about 10x-20x faster (on my Linux system) than using
R_alloc/S_realloc for growing a buffer. The reason S_realloc() is much
slower than realloc() can easily be understood by looking at its
implementation (<R_home>/src/main/memory.c):

   char *S_realloc(char *p, long new, long old, int size)
   {
     size_t nold;
     char *q;
     /* shrinking is a no-op */
     if(new <= old) return p;
     q = R_alloc((size_t)new, size);
     nold = (size_t)old * size;
     memcpy(q, p, nold);
     memset(q + nold, 0, (size_t)new*size - nold);
     return q;
   }

It calls R_alloc(), then copies the data from the old place to the
new place, and then sets the remaining memory to zero.

OTOH realloc() is system-dependent and my understanding is that,
on a descent OS, it will try to do an in-place reallocation, thus
avoiding the need to move the data around.

Cheers,
H.

>
> Best,
> Ge
>
> ----------------------------------------
>> Date: Fri, 19 Jul 2013 17:05:30 -0700
>> From: hpages at fhcrc.org
>> To: ge_tan at live.com
>> CC: bioc-devel at r-project.org
>> Subject: Re: [Bioc-devel] IntAE buffers needs to be freed or not??
>>
>> Hi Ge,
>>
>> No you don't. At least not currently, because those buffers use
>> transient memory which will be freed automatically when .Call()
>> returns. However, I might change the implementation of the IntAE
>> buffers to use user-controlled memory at some point (malloc() is
>> so much faster than R_alloc(), about 10x-20x for me), so when this
>> happens you will need to do something like
>>
>> .Call("AEbufs_free", PACKAGE="IRanges")
>>
>> unless you call your .Call entry point with .Call2 (defined in the
>> IRanges package), which will take care of doing that for you. I highly
>> recommend you do that if you use the IntAE buffers in your C code or
>> if you call C functions that use the IntAE buffers (and a lot of C
>> functions in IRanges and Biostrings use them).
>>
>> Cheers,
>> H.
>>
>> On 07/19/2013 03:35 AM, Ge Tan wrote:
>>> Hi all,
>>>
>>> I am using the IntAE buffers (taken from IRanges packages) in my .Call() code.
>>> Sample code in C:
>>>
>>> IntAE width_buf;
>>> width_buf = new_IntAE(0, 0, 0);
>>> for(…){
>>> IntAE_insert_at(&width_buf, IntAE_get_nelt(&width_buf), width);
>>> }
>>> PROTECT(width = new_INTEGER_from_IntAE(&width_buf));
>>> UNPROTECT(1);
>>> return(width);
>>>
>>> So after using .Call(), do I need to run something like ".Call("AEbufs_free", PACKAGE="IRanges")" in R?
>>> I got this from the IRanges/src/AEbufs.c.
>>>
>>> Thanks!
>>> Ge
>>> _______________________________________________
>>> Bioc-devel at r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>>>
>>
>> --
>> Hervé Pagès
>>
>> Program in Computational Biology
>> Division of Public Health Sciences
>> Fred Hutchinson Cancer Research Center
>> 1100 Fairview Ave. N, M1-B514
>> P.O. Box 19024
>> Seattle, WA 98109-1024
>>
>> E-mail: hpages at fhcrc.org
>> Phone: (206) 667-5791
>> Fax: (206) 667-1319 		 	   		

-- 
Hervé Pagès

Program in Computational Biology
Division of Public Health Sciences
Fred Hutchinson Cancer Research Center
1100 Fairview Ave. N, M1-B514
P.O. Box 19024
Seattle, WA 98109-1024

E-mail: hpages at fhcrc.org
Phone:  (206) 667-5791
Fax:    (206) 667-1319