[R-pkg-devel] multithreading in packages

[Vladimir Dergachev]

On Sat, 9 Oct 2021, Erin Hodgess wrote:

> Have you thought about using C or c++, please? 

Yes, indeed, the core of the package is written in C, with some C++ for 
sorting (which turned out to be rather interesting).

Beyound writing optimized C there are two ways to speed up execution on a 
single computer - multithreading and vector instructions.

Multithreading is easier here, because only one or two libraries are 
needed (libgomp or pthread) and because it is often hard to vectorize 
operations like sorting, hashing and the like.

Also, to use vector instructions to full potential one typically needs a 
fair bit of black magic which is unlikely to pass CRAN tests. I am having 
enough trouble as it is getting a simple flexible array past address 
sanitizers.

> Also, there are packages called pbdDMAT from Drew Schmidt at U of Tenn which might help.

Great, thanks for pointing this out ! Looks like pbdDMAT uses mpi.

Also, it appears this package was removed from CRAN for failing to compile 
on macs, which seems rather unfair - I don't know of any clusters running 
mac os.

Vladimir Dergachev