[R] parallel number of cores according to memory?
|vowe| @end|ng |rom gm@||@com
Wed Jul 8 06:20:39 CEST 2020
if I understand correctly, R makes a copy of the full environment for each
process. thus, even if I have 32 processors, if I only have 64GB of RAM
and my R process holds about 10GB, I should probably not spawn 32 processes.
has anyone written a function that sets the number of cores for use (in
mclapply) to be guessed at by appropriate memory requirements (e.g.,
"amount-of-RAM"/"RAM held by R")?
(it would be even nicer if I could declare my 8GB data frame to be
read-only and to be shared among my processes, but this is presumably
technically very difficult.)
[[alternative HTML version deleted]]
More information about the R-help