[R-pkg-devel] potential memory leak using openMP

[Marcin Jurek]

Hello everyone, so my package was rejected from CRAN because of the
following error:

==29416== 640 bytes in 2 blocks are possibly lost in loss record 162 of
2,089
==29416==    at 0x4837B65: calloc
(coregrind/m_replacemalloc/vg_replace_malloc.c:762)
==29416==    by 0x40118B1: allocate_dtv
(/build/glibc-suXNNi/glibc-2.29/elf/../elf/dl-tls.c:286)
==29416==    by 0x40121FD: _dl_allocate_tls
(/build/glibc-suXNNi/glibc-2.29/elf/../elf/dl-tls.c:532)
==29416==    by 0x5684BDD: allocate_stack
(/build/glibc-suXNNi/glibc-2.29/nptl/allocatestack.c:621)
==29416==    by 0x5684BDD: pthread_create@@GLIBC_2.2.5
(/build/glibc-suXNNi/glibc-2.29/nptl/pthread_create.c:669)
==29416==    by 0x565DF65: ??? (in
/usr/lib/x86_64-linux-gnu/libgomp.so.1.0.0)
==29416==    by 0x5654A3C: GOMP_parallel (in
/usr/lib/x86_64-linux-gnu/libgomp.so.1.0.0)
==29416==    by 0x1DBACEBA: U_NZentries(int, int, arma::Mat<double> const&,
arma::Mat<unsigned int> const&, arma::Mat<double> const&, arma::Col<double>
const&, arma::Col<double> const&, std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> >,  arma::Col<double>)
(/home/Hornik/tmp/CRAN/GPvecchia.Rcheck/00_pkg_src/GPvecchia/src/U_NZentries.cpp:225)
==29416==    by 0x1DBA3B90: _GPvecchia_U_NZentries
(/home/Hornik/tmp/CRAN/GPvecchia.Rcheck/00_pkg_src/GPvecchia/src/RcppExports.cpp:78)

First, I don't know how to reproduce it because running
R CMD check <package_name> --use-valgrind
does not produce any errors on my machine.

Second, see below the relevant part of the U_NZentries function (with the
offending line marked with (***).

*** #pragma omp parallel for
shared(locs,revNNarray,revCondOnLatent,nuggets, nnp,m,Lentries,COV)
private(k,dist,onevec,covmat,nug,n0,inds,revCon_row,inds00)
 schedule(static)

  for (k = 0; k < nnp; k++) {
    inds=revNNarray.row(k).t();
    revCon_row=revCondOnLatent.row(k).t();
    n0 = get_nonzero_count_general(inds); // for general case
    inds00 = get_idx_vals_general(n0, inds);
    nug=nuggets.elem(inds00) % (ones(n0)-revCon_row(span(m+1-n0,m))); //
vec is vec, cannot convert to mat
    dist = calcPWD(locs.rows(inds00));
    #pragma omp critical
    {
      if( COV=="matern"){
        covmat= MaternFun(dist,covparms) + diagmat(nug) ; // summation from
arma
      } else if(COV=="esqe") {
        covmat= EsqeFun(dist,covparms) + diagmat(nug) ; // summation from
arma
      }
    }
    onevec.resize(n0);
    onevec = zeros(n0);
    onevec[n0-1] = 1;
    arma::vec M=solve(chol(covmat,"upper"),onevec);
    // save the entries to matrix
    Lentries(k,span(0,n0-1)) = M.t();
  }

I have googled around and some people are saying that what valgrind reports
is an unavoidable consequence of using openMP (for example here:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=36298 and here:
https://stackoverflow.com/questions/6973489/valgrind-and-openmp-still-reachable-and-possibly-lost-is-that-bad
).

Three questions: 1) what should I do to reproduce the error? 2) is it worth
trying to do so? 3) why does it come up?

Thanks a lot everyone!

Marcin

	[[alternative HTML version deleted]]