[R-sig-hpc] How to check if %dopar% really run parallel?
Mario Valle
mvalle at cscs.ch
Tue May 4 15:31:00 CEST 2010
*BIG RED FACE*
I'm ashamed of myself, that's was the error!
A small, stupid pair of parenthesis missing.
Now the parallel version is faster than the serial one as it should.
(serial: 57.41, parallel 2 cores: 39.31)
Thanks to Stephen and all.
mario
Stephen Weston wrote:
> There is a mistake. Rather than:
>
> times(10000) %dopar% fun
>
> you should write:
>
> times(10000) %dopar% fun()
>
> On my machine, "fun" executes in about 0.4 seconds, so executing
> it 10,000 times should take over an hour to execute. Your error turned
> a real program into a toy program. The error also resulted in more
> communication, since now the function itself is being returned by the
> workers.
>
> When I ran your benchmark on my machine with 100, rather than 10,000
> tasks, I got the following results:
>
> user system elapsed
> 43.573 0.191 43.823
> user system elapsed
> 0.093 0.007 24.890
>
> That's not so bad.
>
> - Steve
>
>
> On Tue, May 4, 2010 at 12:22 AM, Mario Valle <mvalle at cscs.ch> wrote:
>> Is there any way to check that %dopar% really runs parallel?
>> The following code (on a dual core laptop running windows+R 2.11.0pat and on
>> Linux+R2.11.0) runs %dopar% more slowly than the same %do% code.
>> BTW, if you see any obvious mistake in the code...
>> Thanks!
>> mario
>>
>>
>> library(doSNOW)
>> library(foreach)
>>
>> fun <- function() for(q in 1:1000000) sqrt(3)
>>
>> system.time(times(10000) %do% fun, gcFirst = TRUE)
>> # user system elapsed
>> # 5.74 0.01 6.24
>>
>> cl <- makeCluster(2, type = "SOCK")
>> registerDoSNOW(cl)
>>
>> system.time(times(10000) %dopar% fun, gcFirst = TRUE)
>> # user system elapsed
>> # 7.89 0.19 9.01
>>
>> stopCluster(cl)
>>
>> --
>> Ing. Mario Valle
>> Data Analysis and Visualization Group |
>> http://www.cscs.ch/~mvalle
>> Swiss National Supercomputing Centre (CSCS) | Tel: +41 (91) 610.82.60
>> v. Cantonale Galleria 2, 6928 Manno, Switzerland | Fax: +41 (91) 610.82.82
>>
>> _______________________________________________
>> R-sig-hpc mailing list
>> R-sig-hpc at r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-hpc
>>
--
Ing. Mario Valle
Data Analysis and Visualization Group | http://www.cscs.ch/~mvalle
Swiss National Supercomputing Centre (CSCS) | Tel: +41 (91) 610.82.60
v. Cantonale Galleria 2, 6928 Manno, Switzerland | Fax: +41 (91) 610.82.82
More information about the R-sig-hpc
mailing list