[Rd] C - R integration: Memory Issues
Nikolaos Bezirgiannidis
nbezirgi at ee.duth.gr
Fri Apr 20 11:06:52 CEST 2012
On Thu, 19 Apr 2012 07:21:12 -0500, Dirk Eddelbuettel wrote:
> On 18 April 2012 at 17:40, Nikolaos Bezirgiannidis wrote:
> | Hi all,
> |
> | I am a PhD student and I am working on a C project that involves
> some
> | statistical calculations. So, I tried to embed R into C, in order
> to
> | call R functions from a C program. My program seems to get the
> correct
> | results from R. However, it appears to have a lot of memory
> allocation
> | issues, in contrast to the small amounts of memory that my code
> | allocates. Some additional info that might be useful: I have build
> R
> | from source with shared libraries enabled and the compiler I use
> is gcc
> | version 4.61 in a Ubuntu 11.10 linux machine.
>
> [ Well I suspect "sudo apt-get install r-base" would have given you
> the same;
> see the README at $CRAN/src/bin/linux/ubuntu ]
>
> | This is my function:
> |
> | static int prediction(double *berHistory, int berValues, double
> *ber)
> | {
> | SEXP e;
> | SEXP bers;
> | SEXP mean;
> | int i;
> | int errorOccurred;
> | static int init = 0;
> | char *argv[] = {"REmbeddedPostgres", "--gui=none",
> "--silent"};
> | int argc = sizeof(argv)/sizeof(argv[0]);
> |
> | // Initialize Embedded R
> | if (init == 0)
> | {
> | Rf_initEmbeddedR(argc, argv);
> | }
> | init = 1;
> |
> | // Allocate bers and copy values
> | PROTECT(bers = allocVector(REALSXP, berValues));
> |
> | for (i = 0; i < berValues; i++)
> | {
> | REAL(bers)[i] = berHistory[i];
> | }
> |
> | PROTECT(mean = allocVector(REALSXP, 1));
> | PROTECT(e = lang2(install("mean"), bers));
> | mean = R_tryEval(e, R_GlobalEnv, &errorOccurred);
> | if (errorOccurred)
> | {
> | printf("error occurred in mean\n");
> | }
> |
> | for (i = 0; i < berValues; i++)
> | {
> | REAL(bers)[i] = REAL(bers)[i] / REAL(mean)[0];
> | }
> |
> | *ber = REAL(mean)[0];
> |
> | Rf_endEmbeddedR(0);
> | UNPROTECT(3);
> | return 0;
> | }
> |
> | And these are the errors from Valgrind output:
> |
> | HEAP SUMMARY:
> | ==2909== in use at exit: 18,832,260 bytes in 6,791 blocks
> | ==2909== total heap usage: 21,758 allocs, 14,967 frees,
> 30,803,476
> | bytes allocated
> | ==2909==
> | ==2909== 160 (40 direct, 120 indirect) bytes in 1 blocks are
> definitely
> | lost in loss record 179 of 1,398
> | ==2909== at 0x4028876: malloc (vg_replace_malloc.c:236)
> | ==2909== by 0x41B364C: nss_parse_service_list (nsswitch.c:626)
> | ==2909== by 0x41B3C59: __nss_database_lookup (nsswitch.c:167)
> | ==2909== by 0x59272F8: ???
> | ==2909== by 0x5928CCC: ???
> | ==2909== by 0x416ABA6: getpwuid_r@@GLIBC_2.1.2
> (getXXbyYY_r.c:256)
> | ==2909== by 0x416A4ED: getpwuid (getXXbyYY.c:117)
> | ==2909== by 0x439CCB9: do_fileinfo (platform.c:944)
> | ==2909== by 0x43289ED: bcEval (eval.c:4430)
> | ==2909== by 0x4332CA4: Rf_eval (eval.c:397)
> | ==2909== by 0x43377E0: Rf_applyClosure (eval.c:855)
> | ==2909== by 0x432F17E: bcEval (eval.c:4410)
> | ==2909==
> | ==2909== 160 (40 direct, 120 indirect) bytes in 1 blocks are
> definitely
> | lost in loss record 180 of 1,398
> | ==2909== at 0x4028876: malloc (vg_replace_malloc.c:236)
> | ==2909== by 0x41B364C: nss_parse_service_list (nsswitch.c:626)
> | ==2909== by 0x41B3C59: __nss_database_lookup (nsswitch.c:167)
> | ==2909== by 0x5926148: ???
> | ==2909== by 0x5926F3C: ???
> | ==2909== by 0x41694A6: getgrgid_r@@GLIBC_2.1.2
> (getXXbyYY_r.c:256)
> | ==2909== by 0x4168CAD: getgrgid (getXXbyYY.c:117)
> | ==2909== by 0x439CCEB: do_fileinfo (platform.c:947)
> | ==2909== by 0x43289ED: bcEval (eval.c:4430)
> | ==2909== by 0x4332CA4: Rf_eval (eval.c:397)
> | ==2909== by 0x43377E0: Rf_applyClosure (eval.c:855)
> | ==2909== by 0x432F17E: bcEval (eval.c:4410)
> | ==2909==
> | ==2909== LEAK SUMMARY:
> | ==2909== definitely lost: 80 bytes in 2 blocks
> | ==2909== indirectly lost: 240 bytes in 20 blocks
> | ==2909== possibly lost: 0 bytes in 0 blocks
> | ==2909== still reachable: 18,831,940 bytes in 6,769 blocks
>
> Well I think there is no real bad error here. You lost 80 and 240
> bytes. That
> is nothing to worry about, and somewhat normal. R is a dynamic
> system,
> valgrind measures "with some error".
>
> You can compare the result to doing an allocation of a longer vector
> and not
> freeing it.
>
> You could see if freeing your 'mean' variable at the end makes a
> difference,
> or using a pointer to a single (stack) instance instead of an
> allocation
> makes a difference. Likewise, you could returns bers as well. Or
> free
> it. Right now I am not entirely what it is that your 'prediction'
> function is
> trying to do.
It is actually a more complex function with many more allocations and
R_tryEval calls; the simplified one that I included leads to exactly the
same lost bytes, which means that it is not caused by allocVector, but
probably by the R_tryEval call. Is there any other, more proper way to
evaluate R expressions from C?
In my function I only need to return ber, so I could easily free all R
objects (bers and mean here). But C's free() has no result and I am not
familiar with any R function that frees objects allocated by
allocVector. I have also seen in some R tutorials and examples that
objects allocated by allocVector are not freed in any way.
>
> | Reachable error summary is far too long to include in this mail.
> The
> | interesting thing is that reachable errors are all caused by this
> small
> | function.
> |
> | Any ideas? I would also appreciate any suggestions on how to
> improve
> | the R-C integration in my code.
>
> Have you considered using R and C++ instead, and looked at Rcpp and
> RInside?
>
> http://dirk.eddelbuettel.com/code/rcpp.html
>
> http://dirk.eddelbuettel.com/code/rinside.html
>
> RInside in particular is a lot simpler, at least to me. But some
> people
> really want plain C in which case you know which route to take
>
> Dirk
No, unfortunately I cannot switch to C++, because my program is a part
of a bigger C project.
Thanks for your reply and suggestions,
Nikos
More information about the R-devel
mailing list