I report here that, in RStudio in Data Scientist Workbench,
lapply(raw(2^31), function(x) NULL)
failed after not so long time.
> res <- lapply(raw(2^31), function(x) NULL)
Error in FUN(X[[i]], ...) : long vectors not supported yet: memory.c:1652
> sessionInfo()
R version 3.3.1 (2016-06-21)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Debian GNU/Linux stretch/sid
locale:
[1] LC_CTYPE=en_US.UTF-8
[2] LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8
[4] LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8
[6] LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8
[8] LC_NAME=C
[9] LC_ADDRESS=C
[10] LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8
[12] LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets
[6] methods base
other attached packages:
[1] SparkR_1.6.1
loaded via a namespace (and not attached):
[1] tools_3.3.1
However, the code that implements 'lapply', function 'do_lapply' in apply.c, seems to support long vectors.
The error message points to memory.c:1652. I don't understand the code there.
A different case:
gc()
after
system.time(vector("list", 2^31))
gave an error with message pointing to memory.c at different line. Subsequent
gc()
didn't give error.
> system.time(vector("list", 2^31))
user system elapsed
3.104 15.608 18.711
> gc()
Error in gc() : long vectors not supported yet: memory.c:1121
> gc()
used (Mb) gc trigger (Mb) max used (Mb)
Ncells 445496 23.8 750400 40.1 592000 31.7
Vcells 667725 5.1 2062440028 15735.2 2148153146 16389.2