[R-pkg-devel] install.R running out of memory

[Viktor Gal]

Hi Dirk,

so the project is open source, you can reproduce the error yourself (but note it’ll take a long time to actually compile it). steps for reproducing:
git clone https://github.com/shogun-toolbox/shogun.git
cd shogun
git checkout feature/shared_ptr
mkdir build
cd build
cmake -DINTERFACE_R=ON ..
make
make install

(it requires tons of dependencies… if you have docker you can docker pull shogun/shogun-dev and run things inside the container)

the make install part runs the R CMD INSTALL so that’ll cause the problem.

but i’ve just uploaded the generated R code that causes the problem here, note the script is 7Mb i.e. 175k LoC, so you better wget/curl it:
http://maeth.com/shogun.R

cheers,
viktor

> On 2 Nov 2019, at 13:52, Dirk Eddelbuettel <edd using debian.org> wrote:
> 
> 
> Hi Viktor,
> 
> On 2 November 2019 at 13:09, Viktor Gal wrote:
> | I’m developing an ML library that has R bindings… when installing the library with R CMD INSTALL the R process is running out of memory (50G+ ram) when doing:
> | ** byte-compile and prepare package for lazy loading
> | 
> | any ideas how i could debug this part of code, to figure out what is actually happening and why is there a memory leak?
> 
> Easiest for us to help if we can see code -- so if you have a public repo
> somewhere please the link. 
> 
> I suspect you have some sort of recursion or circular dependency
> somewhere. It would be very hard for R to run out of 50gb. But we cannot say
> more.
> 
> So maybe triage. In a situation like this when a (supposedly complete)
> package draft of mine fails "top-down" I often re-validate the toolchain
> "bottom-up" with a minimal package. If that works, keep adding pieces step by
> step from the 'not-working large package' to the 'small working' package
> while continuously ensuring that it still builds.
> 
> Hope this helps, Dirk
> 
> -- 
> http://dirk.eddelbuettel.com | @eddelbuettel | edd using debian.org