[Rd] load/unload segfault puzzle

[Ben Bolker]

  Dear r-devel readers,

  I have a pretty deep problem with package loading and unloading in
the development version of the lme4 package
<https://github.com/lme4/lme4>; it's not boiled down to a properly
minimal example yet (this has been difficult), but I am posting anyway
in the hopes that someone has ideas about how to proceed farther,
since I'm nearly stumped. Apologies in advance for the long post.

   EXECUTIVE SUMMARY: after one cycle of loading, testing (e.g. by
running example(lmer)) and unloading lme4, then loading and unloading
the nlme package, re-loading and exercising lme4 becomes very
unstable, leading eventually to a segmentation fault.  More detail is
available at <https://github.com/lme4/lme4/issues/35> .  Because it's
a segmentation fault, exactly _where_ the crash happens varies a bit
according to platform and precise incantation, but it seems I can
always get a segfault eventually.

  We had previously forgotten to write a .onUnload() function to call
library.dynam.unload() [it's not entirely clear to me why R handles
dynamic loading automatically when there is a useDynLib() in the
NAMESPACE, but not unloading: does anyone know if this is
intentional?]; before we had the appropriate unloading hook, instead
of segfaults we got errors about objects from the DLL not being found.

  The package has many complexities that might make it an especially
tough case: it uses reference classes as well as making use of the
Rcpp and RcppEigen packages ...

  I have tried running under valgrind and have seen what appear to be
memory leaks but no memory access errors. The beginning and end of the
valgrind output are appended below -- I can send the full output if
anyone wants to see it ...

  I plan to continue to try to isolate the problem by (1) minimizing
the amount of package-exercising code run between loading and
unloading; (2) trying to create a minimal version of the package. In
the meantime if anyone can think of any shortcuts or clever
diagnostics I would love to hear about it ...

  thanks
    Ben Bolker


--24640-- Valgrind options:
--24640--    --suppressions=/usr/lib/valgrind/debian-libc6-dbg.supp
--24640--    -v
--24640--    --leak-check=full

==24640== LEAK SUMMARY:
==24640==    definitely lost: 80 bytes in 2 blocks
==24640==    indirectly lost: 240 bytes in 20 blocks
==24640==      possibly lost: 56,014,306 bytes in 32,200 blocks
==24640==    still reachable: 11,032,257 bytes in 993 blocks
==24640==         suppressed: 0 bytes in 0 blocks
==24640== Reachable blocks (those to which a pointer was found) are
not shown.
==24640== To see them, rerun with: --leak-check=full --show-reachable=yes
==24640==
==24640== ERROR SUMMARY: 2493 errors from 2493 contexts (suppressed:
151 from 11)
--24640--
--24640-- used_suppression:    151 dl-hack3-cond-1
==24640==
==24640== ERROR SUMMARY: 2493 errors from 2493 contexts (suppressed:
151 from 11)