Hyperthreading can be part of the problem here. Hyperthreading allows 'simultaneous' thread execution at half the speed.  We tend to turn it off on our CPU-intensive boxes.

As romanov has pointed out, forking, interprocess communications, and collecting and combining all the results are not free.  

Parallel computing makes sense when each individual worker computation is non-trivial, and you can benefit from all the cores running at once for some time.  Your example is still a pretty computationally trivial operation, with the time spent in memory allocation, not CPU.

To get back to my first point, your hyperthreaded CPU has only one memory controller, so the malloc operation will still happen serially.  You need to understand your compute load.

To further extend my point, on server machines where we do a significant amount of parallel processing, we use optimized single threaded BLAS. On desktop machines, we may use a multithreaded BLAS such as the MKL.

Regards,

Brian
Regards,

Brian

Xuening Zhu <puddingnnn529@gmail.com> wrote:
>Thanks for your reply.
>Do you mean mc_t also take consideration of overhead costs? BTW, I
>don't
>point to the whole cost of parallel task. I just want to know why the
>pure
>computation time of each parallel process takes longer than the serial
>one.
>
>
>2014-02-13 15:32 GMT+08:00 romunov <romunov@gmail.com>:
>
>> When doing calculations in parallel, there's also some overhead
>costs. If
>> computation time per core is short, the overhead costs may exceed the
>> computation time, time-wise raising the cost of parallel task.
>>
>> Cheers,
>> Roman
>>
>>
>> On Thu, Feb 13, 2014 at 5:26 AM, Xuening Zhu
><puddingnnn529@gmail.com>wrote:
>>
>>> I am learning about parallel computing in R , and I found this
>happening
>>> in
>>> my experiments.
>>>
>>> Briefly, in the following example, why are most values of user time
>in
>>> t smaller
>>> than that in mc_t ? My machine has 32GB memory, 2 cpus with 4 cores
>and 8
>>> hyper threads in total. Tools such as BLAS to enhance performance
>aren't
>>> installed as well.
>>>
>>> system.time({t = lapply(1:4,function(i) {
>>>     m = matrix(1:10^6,ncol=100)
>>>     t = system.time({
>>>         m%*%t(m)
>>>     })
>>>     return(t)})})
>>>
>>>
>>> library(multicore)
>>> system.time({
>>>     mc_t = mclapply(1:4,function(m){
>>>         m = matrix(1:10^6,ncol=100)
>>>         t = system.time({
>>>             m%*%t(m)
>>>         })
>>>         return(t)
>>>     },mc.cores=4)})
>>>
>>> > t[[1]]
>>> user  system elapsed
>>>
>>> 11.136   0.548  11.703
>>>
>>> [[2]]
>>> user  system elapsed
>>>
>>> 11.533   0.548  12.098
>>>
>>> [[3]]
>>> user  system elapsed
>>>
>>> 11.665   0.432  12.115
>>>
>>> [[4]]
>>> user  system elapsed
>>>
>>> 11.580   0.512  12.115
>>>
>>> > mc_t[[1]]
>>>
>>> user  system elapsed
>>>
>>> 16.677   0.496  17.199
>>>
>>> [[2]]
>>> user  system elapsed
>>>
>>> 16.741   0.428  17.198
>>>
>>> [[3]]
>>> user  system elapsed
>>>
>>> 16.653   0.520  17.198
>>>
>>> [[4]]
>>> user  system elapsed
>>>
>>> 11.056   0.444  11.520
>>>
>>> mc_t and t measures pure computation time according to my
>comprehension.
>>>  Such things happens to parLapply in parallel package as well. The
>memory
>>> in my machine is enough for that computation. (It just takes a few
>percent
>>> of that).
>>>
>>> Also I try to run 4 similar Rscript as below by hand using command
>>> 'Rscript' at the same time on the same machine and save the results.
>The
>>> elapsed time for each of them is about 12s as well. So I think it
>may not
>>> be the contention of cores.
>>>
>>> system.time({t = lapply(1,function(i) {
>>>     m = matrix(1:10^6,ncol=100)
>>>     t = system.time({
>>>         m%*%t(m)
>>>     })
>>>     return(t)})})
>>>
>>> So what happened during the parallel? Does mc_t really measure the
>pure
>>> computation time? Can someone explain the whole process by step in
>detail?
>>>
>>> Thanks.
>>>
>>> --
>>>
>>> Xuening Zhu
>>>
>>>         [[alternative HTML version deleted]]
>>>
>>> _______________________________________________
>>> R-sig-hpc mailing list
>>> R-sig-hpc@r-project.org
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-hpc
>>>
>>
>>
>>
>> --
>> In God we trust, all others bring data.
>>
>
>
>
>-- 
>������
>--------------------------------------------------------
>�������� ������������
>����������2013��������
>������15600608462
>
>Xuening Zhu
>Guanghua School of Management, Peking University
>Tel:+86 15600608462
>
>	[[alternative HTML version deleted]]
>
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>R-sig-hpc mailing list
>R-sig-hpc@r-project.org
>https://stat.ethz.ch/mailman/listinfo/r-sig-hpc

-- 
Brian G. Peterson
Ph: 773.459.4973
IM: bgpbraverock
http://braverock.com/brian/
Sent from my phone. Please excuse my brevity.
	[[alternative HTML version deleted]]

