[R-pkg-devel] potential memory leak using openMP

[Zhian Kamvar]

I've had a problem like this before and I also thought that it was due to
some weird magic in OpenMP, but it turned out to be a fence-post error on
my part [1]. Unfortunately, the valgrind output is not as helpful as
something like ASAN. My suggestion would be to go through that section of
the code with a debugger and manually check for any weird behaviors (full
disclosure: I haven't done this with parallel code). My blog [1] has a
short version of Kevin Ushey's post on debugging with lldb [2], so those
might be good starting points.

[1]: https://zkamvar.netlify.com/post/2018-01-14-i-c-bugs/i-c-bugs/
[2]: http://kevinushey.github.io/blog/2015/04/13/debugging-with-lldb/

Hope that helps.

Best,
Zhian

On Fri, Nov 15, 2019 at 4:50 PM Marcin Jurek <marcinjurek1988 using gmail.com>
wrote:

> Hello everyone, so my package was rejected from CRAN because of the
> following error:
>
> ==29416== 640 bytes in 2 blocks are possibly lost in loss record 162 of
> 2,089
> ==29416==    at 0x4837B65: calloc
> (coregrind/m_replacemalloc/vg_replace_malloc.c:762)
> ==29416==    by 0x40118B1: allocate_dtv
> (/build/glibc-suXNNi/glibc-2.29/elf/../elf/dl-tls.c:286)
> ==29416==    by 0x40121FD: _dl_allocate_tls
> (/build/glibc-suXNNi/glibc-2.29/elf/../elf/dl-tls.c:532)
> ==29416==    by 0x5684BDD: allocate_stack
> (/build/glibc-suXNNi/glibc-2.29/nptl/allocatestack.c:621)
> ==29416==    by 0x5684BDD: pthread_create@@GLIBC_2.2.5
> (/build/glibc-suXNNi/glibc-2.29/nptl/pthread_create.c:669)
> ==29416==    by 0x565DF65: ??? (in
> /usr/lib/x86_64-linux-gnu/libgomp.so.1.0.0)
> ==29416==    by 0x5654A3C: GOMP_parallel (in
> /usr/lib/x86_64-linux-gnu/libgomp.so.1.0.0)
> ==29416==    by 0x1DBACEBA: U_NZentries(int, int, arma::Mat<double> const&,
> arma::Mat<unsigned int> const&, arma::Mat<double> const&, arma::Col<double>
> const&, arma::Col<double> const&, std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> >,  arma::Col<double>)
>
> (/home/Hornik/tmp/CRAN/GPvecchia.Rcheck/00_pkg_src/GPvecchia/src/U_NZentries.cpp:225)
> ==29416==    by 0x1DBA3B90: _GPvecchia_U_NZentries
>
> (/home/Hornik/tmp/CRAN/GPvecchia.Rcheck/00_pkg_src/GPvecchia/src/RcppExports.cpp:78)
>
> First, I don't know how to reproduce it because running
> R CMD check <package_name> --use-valgrind
> does not produce any errors on my machine.
>
> Second, see below the relevant part of the U_NZentries function (with the
> offending line marked with (***).
>
> *** #pragma omp parallel for
> shared(locs,revNNarray,revCondOnLatent,nuggets, nnp,m,Lentries,COV)
> private(k,dist,onevec,covmat,nug,n0,inds,revCon_row,inds00)
>  schedule(static)
>
>   for (k = 0; k < nnp; k++) {
>     inds=revNNarray.row(k).t();
>     revCon_row=revCondOnLatent.row(k).t();
>     n0 = get_nonzero_count_general(inds); // for general case
>     inds00 = get_idx_vals_general(n0, inds);
>     nug=nuggets.elem(inds00) % (ones(n0)-revCon_row(span(m+1-n0,m))); //
> vec is vec, cannot convert to mat
>     dist = calcPWD(locs.rows(inds00));
>     #pragma omp critical
>     {
>       if( COV=="matern"){
>         covmat= MaternFun(dist,covparms) + diagmat(nug) ; // summation from
> arma
>       } else if(COV=="esqe") {
>         covmat= EsqeFun(dist,covparms) + diagmat(nug) ; // summation from
> arma
>       }
>     }
>     onevec.resize(n0);
>     onevec = zeros(n0);
>     onevec[n0-1] = 1;
>     arma::vec M=solve(chol(covmat,"upper"),onevec);
>     // save the entries to matrix
>     Lentries(k,span(0,n0-1)) = M.t();
>   }
>
> I have googled around and some people are saying that what valgrind reports
> is an unavoidable consequence of using openMP (for example here:
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=36298 and here:
>
> https://stackoverflow.com/questions/6973489/valgrind-and-openmp-still-reachable-and-possibly-lost-is-that-bad
> ).
>
> Three questions: 1) what should I do to reproduce the error? 2) is it worth
> trying to do so? 3) why does it come up?
>
> Thanks a lot everyone!
>
> Marcin
>
>         [[alternative HTML version deleted]]
>
> ______________________________________________
> R-package-devel using r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-package-devel
>

	[[alternative HTML version deleted]]