[R] Maximum # of DLLs reached, or, how to clean up after yourself?

[Alexander Shenkin]

Hi Henrik,

Thanks for your reply.  I didn't realize that floating DLLs were an 
issue (good to know).  My query is actually a bit more basic.  I'm 
actually wondering how folks manage their loading and unloading of 
packages when calling scripts within scripts.

Quick example:
Script1:
	library(package1)
	source("script2.r")
	# do stuff reliant on package1
	detach("package:package1", unload=TRUE)

Script2:
	library(package1)
	library(package2)
	# do stuff reliant on package1 and package2
	detach("package:package1", unload=TRUE)
	detach("package:package2", unload=TRUE)

Script2 breaks Script1 by unloading package1 (though unloading package2 
is ok).  I will have to test whether each package is loaded in Script2 
before loading it, and use that list when unloading at the end of the 
Script2.  *Unless there's a better way to do it* (which is my question - 
is there?).  I'm possibly just pushing the whole procedural scripting 
thing too far, but I also think that this likely isn't uncommon in R.

Any thoughts greatly appreciated!

Thanks,
Allie

On 9/13/2016 7:23 PM, Henrik Bengtsson wrote:
> In R.utils (>= 2.4.0), which I hope to submitted to CRAN today or
> tomorrow, you can simply call:
>
>    R.utils::gcDLLs()
>
> It will look at base::getLoadedDLLs() and its content and compare to
> loadedNamespaces() and unregister any "stray" DLLs that remain after
> corresponding packages have been unloaded.
>
> Until the new version is on CRAN, you can install it via
>
>     source("http://callr.org/install#HenrikBengtsson/R.utils@develop")
>
> or alternatively just source() the source file:
>
>     source("https://raw.githubusercontent.com/HenrikBengtsson/R.utils/develop/R/gcDLLs.R")
>
>
> DISCUSSION:
> (this might be better suited for R-devel; feel free to move it there)
>
> As far as I understand the problem, running into this error / limit is
> _not_ the fault of the user.  Instead, I'd argue that it is the
> responsibility of package developers to make sure to unregister any
> registered DLLs of theirs when the package is unloaded.  A developer
> can do this by adding the following to their package:
>
> .onUnload <- function(libpath) {
>     library.dynam.unload(utils::packageName(), libpath)
>  }
>
> That should be all - then the DLL will be unloaded as soon as the
> package is unloaded.
>
> I would like to suggest that 'R CMD check' would include a check that
> asserts when a package is unloaded it does not leave any registered
> DLLs behind, e.g.
>
> * checking whether the namespace can be unloaded cleanly ... WARNING
>   Unloading the namespace does not unload DLL
> * checking loading without being on the library search path ... OK
>
> For further details on my thoughts on this, see
> https://github.com/HenrikBengtsson/Wishlist-for-R/issues/29.
>
> Hope this helps
>
> Henrik
>
> On Tue, Sep 13, 2016 at 6:05 AM, Alexander Shenkin <ashenkin at ufl.edu> wrote:
>> Hello all,
>>
>> I have a number of analyses that call bunches of sub-scripts, and in the
>> end, I get the "maximal number of DLLs reached" error.  This has been asked
>> before (e.g.
>> http://stackoverflow.com/questions/36974206/r-maximal-number-of-dlls-reached),
>> and the general answer is, "just clean up after yourself".
>>
>> Assuming there are no plans to raise this 100-DLL limit in the near future,
>> my question becomes, what is best practice for cleaning up (detaching)
>> loaded packages in scripts, when those scripts are sometimes called from
>> other scripts?  One can detach all packages at the end of a script that were
>> loaded at the beginning of the script.  However, if a package is required in
>> a calling script, one should really make sure it hadn't been loaded prior to
>> sub-script invocation before detaching it.
>>
>> I could write a custom function that pushes and pops package names from a
>> global list, in order to keep track, but maybe there's a better way out
>> there...
>>
>> Thanks for any thoughts.
>>
>> Allie
>>
>> ______________________________________________
>> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.