[Rd] [External] memory consumption of nested (un)serialize of sys.frames()

[Andreas Kersting]

Hi,

For (hopefully) full reproducibility:

docker run rocker/tidyverse:4.0.5 Rscript -e 'devtools::install_github("akersting/dumpTest", INSTALL_opts = "--with-keep.source"); library(dumpTest); for (i in 1:100) {print(i); print(system.time(f()))}'

Regards,
Andreas

2021-04-07 17:09 GMT+02:00 "Andreas Kersting" <r-devel using akersting.de>:
> Hi Dirk, hi Luke,
> 
> Thanks for checking!
> 
> I could narrow it down further. I have the issue only if I install --with-keep.source, i.e.
> 
> R CMD INSTALL --with-keep.source dumpTest
> 
> Since this is the default in RStudio when clicking "Install and Restart", I was always having the issue - also from base R. If I install using e.g. devtools::install_github() directly it is also fine for me.
> 
> Could you please confirm? Thanks!
> 
> Regards,
> Andreas
> 
> 2021-04-07 16:20 GMT+02:00 "Dirk Eddelbuettel" <edd using debian.org>:
>> 
>> On 7 April 2021 at 16:06, Andreas Kersting wrote:
>> | Hi Luke,
>> | 
>> | Please see https://github.com/akersting/dumpTest for the package.
>> | 
>> | Here a session showing my issue:
>> | 
>> | > library(dumpTest)
>> | > sessionInfo()
>> | R version 4.0.5 (2021-03-31)
>> | Platform: x86_64-pc-linux-gnu (64-bit)
>> | Running under: Debian GNU/Linux 10 (buster)
>> | 
>> | Matrix products: default
>> | BLAS:   /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.8.0
>> | LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.8.0
>> | 
>> | locale:
>> |  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
>> |  [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
>> |  [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
>> |  [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
>> |  [9] LC_ADDRESS=C               LC_TELEPHONE=C            
>> | [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       
>> | 
>> | attached base packages:
>> | [1] stats     graphics  grDevices utils     datasets  methods   base     
>> | 
>> | other attached packages:
>> | [1] dumpTest_0.1.0
>> | 
>> | loaded via a namespace (and not attached):
>> | [1] compiler_4.0.5
>> | > for (i in 1:100) {
>> | +   print(i)
>> | +   print(system.time(f()))
>> | + }
>> | [1] 1
>> |    user  system elapsed 
>> |   0.028   0.004   0.034 
>> | [1] 2
>> |    user  system elapsed 
>> |   0.067   0.008   0.075 
>> | [1] 3
>> |    user  system elapsed 
>> |   0.176   0.000   0.176 
>> | [1] 4
>> |    user  system elapsed 
>> |   0.335   0.012   0.349 
>> | [1] 5
>> |    user  system elapsed 
>> |   0.745   0.023   0.770 
>> | [1] 6
>> |    user  system elapsed 
>> |   1.495   0.060   1.572 
>> | [1] 7
>> |    user  system elapsed 
>> |   2.902   0.136   3.040 
>> | [1] 8
>> |    user  system elapsed 
>> |   5.753   0.272   6.034 
>> | [1] 9
>> |    user  system elapsed 
>> |  11.807   0.708  12.597 
>> | [1] 10
>> | ^C
>> | Timing stopped at: 6.638 0.549 7.214
>> | 
>> | I had to interrupt in iteration 10 because I was running low on RAM.
>> 
>> No issue here.  Ubuntu 20.10, R 4.0.5 'from CRAN' i.e. Michael's PPA build
>> off my Debian package, hence instrumentation as in the Debian package.
>> 
>> edd using rob:~$ installGithub.r akersting/dumpTest
>> Using github PAT from envvar GITHUB_PAT
>> Downloading GitHub repo akersting/dumpTest using HEAD
>> ✔  checking for file ‘/tmp/remotes3f9af733166ccd/akersting-dumpTest-3bed8e2/DESCRIPTION’ ...
>> ─  preparing ‘dumpTest’:
>> ✔  checking DESCRIPTION meta-information ...
>> ─  checking for LF line-endings in source and make files and shell scripts
>> ─  checking for empty or unneeded directories
>> ─  building ‘dumpTest_0.1.0.tar.gz’
>>    
>> Installing package into ‘/usr/local/lib/R/site-library’
>> (as ‘lib’ is unspecified)
>> * installing *source* package ‘dumpTest’ ...
>> ** using staged installation
>> ** R
>> ** byte-compile and prepare package for lazy loading
>> ** help
>> No man pages found in package  ‘dumpTest’ 
>> *** installing help indices
>> ** building package indices
>> ** testing if installed package can be loaded from temporary location
>> ** testing if installed package can be loaded from final location
>> ** testing if installed package keeps a record of temporary installation path
>> * DONE (dumpTest)
>> edd using rob:~$ Rscript -e 'system.time({for (i in 1:100) dumpTest::f()})'
>>    user  system elapsed 
>>   0.481   0.019   0.500 
>> edd using rob:~$
>> 
>> (I also ran the variant you showed with the dual print statements, it just
>> consumes more screen real estate and ends on
>> 
>> [...]
>> [1] 97                                      
>>    user  system elapsed     
>>   0.004   0.000   0.005 
>> [1] 98                                                                                                   
>>    user  system elapsed 
>>   0.004   0.000   0.005                   
>> [1] 99                     
>>    user  system elapsed    
>>   0.004   0.000   0.004                                                                                  
>> [1] 100                                                                                                  
>>    user  system elapsed                                                                                  
>>   0.005   0.000   0.005 
>> edd using rob:~$ )
>> 
>> Dirk
>> 
>> -- 
>> https://dirk.eddelbuettel.com | @eddelbuettel | edd using debian.org
>> 
> ______________________________________________
> R-devel using r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>