[R] parallel number of cores according to memory?

[ivo welch]

no, I'm not.   mostly conventional use afaik.  if this should not be
happening, I can trace it down to a small reproducible example to
figure it out.

--
Ivo Welch (ivo.welch using ucla.edu)

--
Ivo Welch (ivo.welch using ucla.edu)
http://www.ivo-welch.info/
J. Fred Weston Distinguished Professor of Finance, UCLA Anderson



On Tue, Jul 7, 2020 at 10:24 PM Jeff Newmiller <jdnewmil using dcn.davis.ca.us> wrote:
>
> The list does strip html, but the quality of what remains varies greatly.
>
> Are you using tidyverse functions in your workers? Sounds to me like you are doing something in the workers that is triggering making copies of the input data frame.
>
> On July 7, 2020 10:15:15 PM PDT, ivo welch <ivowel using gmail.com> wrote:
> >ugghhh---apologies.  although in 2020, it would be nice if the mailing
> >list had an automatic html filter (or even bouncer!)
> >
> >I am using macos.  alas, my experiments suggest that `mclapply()` on a
> >32-core intel system with 64GB of RAM, where the input data frame is
> >8GB and the output is about 500MB per core (to be stitched together
> >into about 16GB), the system starts swapping like crazy, comes to a
> >halt, and then usually crashes.
> >
> >/iaw
>
> --
> Sent from my phone. Please excuse my brevity.