[Rd] accelerating matrix multiply

[Radford Neal]

> From: "Cohn, Robert S" <robert.s.cohn at intel.com>
>
> I am using R to multiply some large (30k x 30k double) matrices on a
>  64 core machine (xeon phi).  I added some timers to
>  src/main/array.c to see where the time is going. All of the time is
>  being spent in the matprod function, most of that time is spent in
>  dgemm. 15 seconds is in matprod in some code that is checking if
>  there are NaNs.
>
> The NaN checking code is not being vectorized...

This can be a problem with big matrices when lots of cores are used
for the actual multiply, but is even more of a problem when at least
one of the matrices is small (eg, a vector-matrix multiply), in which
case the NaN check can dominate, slowing the operation by up to a
factor of about ten.

I pointed this problem out over six years ago, and provided a 
patch that greatly speeds up many matrix multiplies (see
http://www.cs.utoronto.ca/~radford/R-mods.html).  But this
improvement has not been incorporated into R Core versions of R.

Since then, a more elaborate solution to the problem of NaN checks has
been incorporated into my pqR version of R (see pqR-project.org).  The
documentation on this approach can be found with help("%*%") if you're
running pqR, or you can just look at the source for this help file in
the pqR source code repository, at

https://github.com/radfordneal/pqR/blob/Release-2016-10-24/src/library/base/man/matmult.Rd

    Radford