[R-pkg-devel] parallel computing slower than sequential computing

[Wang, Zhu]

-----Original Message-----
From: Ivan Krylov <krylov.r00t using gmail.com> 
Sent: Tuesday, April 30, 2019 10:17 AM
To: Wang, Zhu <wangz1 using uthscsa.edu>
Cc: R-package-devel using r-project.org
Subject: Re: [R-pkg-devel] parallel computing slower than sequential computing

On Mon, 29 Apr 2019 23:44:42 +0000
"Wang, Zhu" <wangz1 using uthscsa.edu> wrote:

> sessionInfo()
> R version 3.5.2 (2018-12-20)
> Platform: x86_64-pc-linux-gnu (64-bit) Running under: Ubuntu 18.04.2 
> LTS

Which BLAS implementation do you use? One popular implementation, OpenBLAS, spawns multiple threads to do some operations faster; the threads can compete against each other for CPU resources if resulting number of processes * threads per process is more than what CPU can handle.

>>BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.7.1
>>LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.7.1


How many CPU cores does your system have? Does this include SMT (also known as hyper-threading on Intel processors)? While some problems benefit from processor pipeline being able to fetch from multiple threads at the same time, for others it's more of a bottleneck.

>>8 CPU cores and 16 logical processors. The Linux system is on a Virtualbox. I realized this might be a factor.

It may help to decrease the n.cores parameter.
>> n.cores=3, 4, 5 would have similar user/elapsed time compared with sequential computing.  Thanks Ivan.
--
Best regards,
Ivan