[Rd] C - R integration: Memory Issues

[Dirk Eddelbuettel]

On 18 April 2012 at 17:40, Nikolaos Bezirgiannidis wrote:
|  Hi all,
| 
|  I am a PhD student and I am working on a C project that involves some 
|  statistical calculations. So, I tried to embed R into C, in order to 
|  call R functions from a C program. My program seems to get the correct 
|  results from R. However, it appears to have a lot of memory allocation 
|  issues, in contrast to the small amounts of memory that my code 
|  allocates. Some additional info that might be useful: I have build R 
|  from source with shared libraries enabled and the compiler I use is gcc 
|  version 4.61 in a Ubuntu 11.10 linux machine.

[ Well I suspect "sudo apt-get install r-base" would have given you the same;
  see the README at $CRAN/src/bin/linux/ubuntu ]
 
|  This is my function:
| 
|  static int prediction(double *berHistory, int berValues, double *ber)
|  {
|       SEXP e;
|       SEXP bers;
|       SEXP mean;
|       int     i;
|       int     errorOccurred;
|       static int init = 0;
|       char *argv[] = {"REmbeddedPostgres", "--gui=none", "--silent"};
|       int argc = sizeof(argv)/sizeof(argv[0]);
| 
|       // Initialize Embedded R
|       if (init == 0)
|       {
|            Rf_initEmbeddedR(argc, argv);
|       }
|       init = 1;
| 
|       // Allocate bers and copy values
|       PROTECT(bers = allocVector(REALSXP, berValues));
|  
|       for (i = 0; i < berValues; i++)
|       {
|            REAL(bers)[i] = berHistory[i];
|       }
| 
|       PROTECT(mean = allocVector(REALSXP, 1));
|       PROTECT(e = lang2(install("mean"), bers));
|       mean = R_tryEval(e, R_GlobalEnv, &errorOccurred);
|       if (errorOccurred)
|       {
|            printf("error occurred in mean\n");
|       }
| 
|       for (i = 0; i < berValues; i++)
|       {
|            REAL(bers)[i] = REAL(bers)[i] / REAL(mean)[0];
|       }
| 
|       *ber = REAL(mean)[0];
| 
|       Rf_endEmbeddedR(0);
|       UNPROTECT(3);
|       return 0;
|  }
|
|  And these are the errors from Valgrind output:
| 
|  HEAP SUMMARY:
|  ==2909==     in use at exit: 18,832,260 bytes in 6,791 blocks
|  ==2909==   total heap usage: 21,758 allocs, 14,967 frees, 30,803,476 
|  bytes allocated
|  ==2909==
|  ==2909== 160 (40 direct, 120 indirect) bytes in 1 blocks are definitely 
|  lost in loss record 179 of 1,398
|  ==2909==    at 0x4028876: malloc (vg_replace_malloc.c:236)
|  ==2909==    by 0x41B364C: nss_parse_service_list (nsswitch.c:626)
|  ==2909==    by 0x41B3C59: __nss_database_lookup (nsswitch.c:167)
|  ==2909==    by 0x59272F8: ???
|  ==2909==    by 0x5928CCC: ???
|  ==2909==    by 0x416ABA6: getpwuid_r@@GLIBC_2.1.2 (getXXbyYY_r.c:256)
|  ==2909==    by 0x416A4ED: getpwuid (getXXbyYY.c:117)
|  ==2909==    by 0x439CCB9: do_fileinfo (platform.c:944)
|  ==2909==    by 0x43289ED: bcEval (eval.c:4430)
|  ==2909==    by 0x4332CA4: Rf_eval (eval.c:397)
|  ==2909==    by 0x43377E0: Rf_applyClosure (eval.c:855)
|  ==2909==    by 0x432F17E: bcEval (eval.c:4410)
|  ==2909==
|  ==2909== 160 (40 direct, 120 indirect) bytes in 1 blocks are definitely 
|  lost in loss record 180 of 1,398
|  ==2909==    at 0x4028876: malloc (vg_replace_malloc.c:236)
|  ==2909==    by 0x41B364C: nss_parse_service_list (nsswitch.c:626)
|  ==2909==    by 0x41B3C59: __nss_database_lookup (nsswitch.c:167)
|  ==2909==    by 0x5926148: ???
|  ==2909==    by 0x5926F3C: ???
|  ==2909==    by 0x41694A6: getgrgid_r@@GLIBC_2.1.2 (getXXbyYY_r.c:256)
|  ==2909==    by 0x4168CAD: getgrgid (getXXbyYY.c:117)
|  ==2909==    by 0x439CCEB: do_fileinfo (platform.c:947)
|  ==2909==    by 0x43289ED: bcEval (eval.c:4430)
|  ==2909==    by 0x4332CA4: Rf_eval (eval.c:397)
|  ==2909==    by 0x43377E0: Rf_applyClosure (eval.c:855)
|  ==2909==    by 0x432F17E: bcEval (eval.c:4410)
|  ==2909==
|  ==2909== LEAK SUMMARY:
|  ==2909==    definitely lost: 80 bytes in 2 blocks
|  ==2909==    indirectly lost: 240 bytes in 20 blocks
|  ==2909==      possibly lost: 0 bytes in 0 blocks
|  ==2909==    still reachable: 18,831,940 bytes in 6,769 blocks

Well I think there is no real bad error here. You lost 80 and 240 bytes. That
is nothing to worry about, and somewhat normal. R is a dynamic system,
valgrind measures "with some error".

You can compare the result to doing an allocation of a longer vector and not
freeing it.

You could see if freeing your 'mean' variable at the end makes a difference,
or using a pointer to a single (stack) instance instead of an allocation
makes a difference.  Likewise, you could returns bers as well. Or free
it. Right now I am not entirely what it is that your 'prediction' function is
trying to do.

|  Reachable error summary is far too long to include in this mail. The 
|  interesting thing is that reachable errors are all caused by this small 
|  function.
| 
|  Any ideas? I would also appreciate any suggestions on how to improve 
|  the R-C integration in my code.

Have you considered using R and C++ instead, and looked at Rcpp and RInside?  

     http://dirk.eddelbuettel.com/code/rcpp.html 

     http://dirk.eddelbuettel.com/code/rinside.html 

RInside in particular is a lot simpler, at least to me. But some people
really want plain C in which case you know which route to take

Dirk

-- 
R/Finance 2012 Conference on May 11 and 12, 2012 at UIC in Chicago, IL
See agenda, registration details and more at http://www.RinFinance.com