[R-sig-hpc] multicore performance question

[Rainer M Krug]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 09/07/12 12:25, Mauricio Zambrano-Bigiarini wrote:
> Thanks Rainer for your feedback.
> 
> What I got for the second case (also almost the same for a few trials):
> 
> nr <- 5000; X <- matrix(rnorm(1000), ncol=30, nrow=nr); set.seed(100)
> 
> system.time(unlist(lapply(1:nr, FUN=fn1, x=X))) #   user  system elapsed #  0.139   0.000
> 0.140
> 
> 
> set.seed(100); system.time(unlist(mclapply(1:nr, FUN=fn1, x=X)))
> 
> #   user  system elapsed #  0.083   0.091   0.118
> 
> 
> In that case mclapply is slightly faster than lapply. Do you have any idea why?

Please keep this on-list - somebody else might have any ideas.

You have the actual computation time and communication time. If the computation time is small, you
won't see an increase in speed when using several cores. But if the computations take long, you
might / will see a speed increase. So in this case I guess, that the computation is fast, and the
communication between the threads / data transfer takes up the chunk of the time. You can check
this by running the code on one core, 2 cores, ... up to eight cores and plot the time needed (you
can set the number of threads in multicore).


Cheers,

Rainer

> 
> I'm running on an 8 cores machine with RHEL6 (64).
> 
> Thanks in advance,
> 
> Mauricio
> 
> 
> On 09/07/12 12:03, Rainer M Krug wrote:
>> 
>> 
> On 09/07/12 10:49, Mauricio Zambrano-Bigiarini wrote:
>>>> Dear list,
>>>> 
>>>> I need to evaluate a vectorial function (~30 dimensions), and I would like to know if it
>>>> is possible to take advantage of a multicore machine.
>>>> 
>>>> So far, I'm using the multicore package, and I'm not getting any time saving. I guess
>>>> the larger the computation time the larger the gain in performance, and probably that is
>>>> the reason why I'm not speeding up my computations by using multicore.
>>>> 
>>>> However, I would like to ask you if the reason for not getting any time saving by using 
>>>> 'multicore'
> 
> 
> Well - I see a difference:
> 
> Without multicore: user  system elapsed 0.020   0.000   0.023
> 
> With multicore: user  system elapsed 0.012   0.024   0.048
> 
> Approximately the same for a few tries, and the two times factor one would expect (I use a
> dual core).
> 
> Or even more pronounced:
> 
>>>> nr<- 5000; X<- matrix(rnorm(1000), ncol=30, nrow=nr) set.seed(100); 
>>>> system.time(unlist(lapply(1:nr, FUN=fn1, x=X)))
> user  system elapsed 0.224   0.004   0.229
>>>> set.seed(100); system.time(unlist(mclapply(1:nr, FUN=fn1, x=X)))
> user  system elapsed 0.120   0.044   0.174
>>>> 
> 
> Cheers,
> 
> Rainer
> 
> 
>>>> is the short time of my computations OR  the way in which I defined the function to be
>>>> run (fn and fn1):
>>>> 
>>>> ----------- START -------------- library(multicore)
>>>> 
>>>> fn<- function(x) { n<- length(x) return(1 + (1/4000) * sum(x^2) - 
>>>> prod(cos(x/sqrt(seq(1:n))))) }
>>>> 
>>>> fn1<- function(i, x) fn(x[i,])
>>>> 
>>>> nr<- 500 X<- matrix(rnorm(1000), ncol=30, nrow=nr)
>>>> 
>>>> # No multicore set.seed(100) system.time( unlist(lapply(1:nr, FUN=fn1, x=X)) )
>>>> 
>>>> # multicore set.seed(100) system.time( unlist(mclapply(1:nr, FUN=fn1, x=X, mc.cores=6))
>>>> )
>>>> 
>>>> sessionInfo() R version 2.15.0 (2012-03-30) Platform: x86_64-redhat-linux-gnu (64-bit)
>>>> 
>>>> locale: [1] LC_CTYPE=en_GB.utf8       LC_NUMERIC=C [3] LC_TIME=en_GB.utf8 
>>>> LC_COLLATE=en_GB.utf8 [5] LC_MONETARY=en_GB.utf8    LC_MESSAGES=en_GB.utf8 [7]
>>>> LC_PAPER=C LC_NAME=C [9] LC_ADDRESS=C              LC_TELEPHONE=C [11]
>>>> LC_MEASUREMENT=en_GB.utf8 LC_IDENTIFICATION=C
>>>> 
>>>> attached base packages: [1] stats     graphics  grDevices utils     datasets  methods
>>>> base
>>>> 
>>>> other attached packages: [1] multicore_0.1-7  hydroPSO_0.1-57  hydroTSM_0.3-5-2 xts_0.8-6
>>>> [5] zoo_1.7-7        sp_0.9-99
>>>> 
>>>> loaded via a namespace (and not attached): [1] automap_1.0-12 class_7.3-4
>>>> cluster_1.14.2 e1071_1.6 grid_2.15.0 [6] gstat_1.0-12   Hmisc_3.9-3    lattice_0.20-6
>>>> tools_2.15.0
>>>> 
>>>> ----------- END --------------
>>>> 
>>>> 
>>>> any comment will be very much appreciated.
>>>> 
>>>> 
>>>> Thanks in advance,
>>>> 
>>>> 
>>>> Mauricio Zambrano-Bigiarini
>>>> 
> 
> 
>> 
>> 
> 
> 
> k)

- -- 
Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation Biology, UCT), Dipl. Phys.
(Germany)

Centre of Excellence for Invasion Biology
Stellenbosch University
South Africa

Tel :       +33 - (0)9 53 10 27 44
Cell:       +33 - (0)6 85 62 59 98
Fax :       +33 - (0)9 58 10 27 44

Fax (D):    +49 - (0)3 21 21 25 22 44

email:      Rainer at krugs.de

Skype:      RMkrug


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk/6x0UACgkQoYgNqgF2egoaagCeIFbusF1owz8DYqvAjwe/teUb
YpQAmgP6t3/cbj2/kLZaiXS/LeJaOVhI
=aTAK
-----END PGP SIGNATURE-----