[R] Parallel processes collapse into one
kry|ov@r00t @end|ng |rom gm@||@com
Thu Jun 6 13:20:51 CEST 2019
On Mon, 3 Jun 2019 06:37:46 +0200
Nicola Lunardon <nicola.lunardon.84 using gmail.com> wrote:
> R seems to be running, but simulations do not progress further.
Have you tried message() (or REprintf() in C code) to narrow down the
specific part of the code where simulations stop in their progress?
It's less convenient than a good debugger, but with parallel code it is
sometimes the only way to reproduce the problem and get some
information about it.
> If I run simulations on 16 cores I end up having an R instance with
> CPU usage about 1600%, never experienced such a behaviour.
> BLAS: /usr/lib/openblas-base/libblas.so.3
> LAPACK: /usr/lib/libopenblasp-r0.2.18.so
OpenBLAS can, indeed, use multiple threads on its own, inside a single
process. Combined with mclapply, this might create a situation when
there are far more threads competing for CPU time than CPU cores
available. Does it help if you set an environment variable such as
OPENBLAS_NUM_THREADS [*] to a number less than or equal to (number of
CPU cores / mc.cores mclapply argument)?
More information about the R-help