Hi Drew:
Thanks for your explanation. I'll try to exploit the serial version with
MKL later. Actually I have tried openblas earlier, but it seems that the
parallel function such as mclapply can't work under such condition. And I
want to include something like Newton-Raphson algorithm in each core
(process), so I wonder if it can be boosted with pbdDMAT or mclapply.
The server has 8 cores and 32G memory.


2014-02-18 0:30 GMT+08:00 Drew Schmidt <schmidt@math.utk.edu>:

>  On 02/17/2014 04:13 AM, Xuening Zhu wrote:
>
>  So I want to include the data distribution, communication and
> computation time as a whole for comparation.
>
>
> What do you think your group and mm lines are doing?
>
>
>  And I find it better to use mclapply in a small machine with shared
> memory. Also I think I need to increase the data size in this experiment.
>
>
> For a tiny problem such as the one you have provided, sure.  You'd be
> better off still to link with a good parallel BLAS.  In fact, you're better
> off just doing it in serial at your problem size.  On my machine, I get
> 0.017s in serial, and with 2 cores, 0.053s using your parallel algorithm
> as-is, and 0.087s when I include the group and mm lines in the timings.
> All of these were run in batch with Rscript.
>
> If you want to exploit (fast) parallel BLAS on a small machine with a
> small problem, you would be best served by something like MKL.  The story
> is slightly different with matrix factorizations, however.  For example,
> you might be interested in computing the condition number of a matrix.
> Consider:
>
> ### Serial
> m <- 1e4
> n <- 1e4
> mat <- matrix(rnorm(m*n), m, n)
> system.time(rcond(mat))
>
> ### Parallel
> library(pbdDMAT,quiet=TRUE)
> init.grid()
>
> m <- 1e4
> n <- 1e4
>
> mat = matrix(rnorm(m*n), m, n)
> bldim <- 64
>
>
> tt = system.time({
>   if(comm.rank()==0){
>     mat = mat
>   } else {
>     mat=NULL
>   }
>   dx <- as.ddmatrix(x=mat,bldim=bldim)
>   cond <- rcond(dx)
> })
>
> comm.print(tt)
> finalize()
>
>
> In this case, I get 69.050s for the serial script, and 43.908s for the
> parallel one (again with 2 ranks).
>
> Brian's email in this thread is spot on.
>
>
> --
> Drew Schmidt
> National Institute for Computational Sciences
> University of Tennessee, USA
> http://r-pbd.org/
>
>  2014-02-17 3:19 GMT+08:00 Drew Schmidt <schmidt@math.utk.edu>:
>
>> There are a few things going on here. Most notably, the script you
>> provided is comparing two completely different operations.  t(dx) %*% dx
>> produces a matrix of dimension 10000x10000 (762 GiB), and mat %*% t(mat)
>> produces a matrix of dimension 100x100 (78 KiB).  Of course the second
>> one will be faster.  You also include the data generation within the
>> parallel, but not serial timings, which doesn't fairly compare the
>> computation time (especially for such small data, where hundredths of a
>> second count).  Making only these changes, on my machine the timings
>> with 2 ranks are 0.033 for the serial operation, and 0.153 for the
>> parallel one.
>>
>> Also, as noted above, this matrix is actually quite small, only about 8
>> MiB in size for the double precision storage it will get converted to
>> for LAPACK/ScaLAPACK operations.  For small matrices, the communication
>> overhead in ScaLAPACK can eat you alive.  You can see this by making the
>> bldim large enough to encompass the entire matrix; even then, when the
>> "parallel" product is done on one rank, there is some communication
>> overhead.  On my machine, again with 2 ranks, in this case the timings
>> are 0.033 and 0.043, for serial and parallel respectively.  You can use
>> pbdDMAT effectively on a small shared memory machine, but it really
>> begins to shine for larger, distributed platforms (servers, clusters,
>> supercomputers).
>>
>> As a final side note, you can improve the performance of both the serial
>> and parallel operations by using crossprod()/tcrossprod().
>>
>> --
>> Drew Schmidt
>> National Institute for Computational Sciences
>> University of Tennessee, USA
>> http://r-pbd.org/
>>
>> On 02/16/2014 08:16 AM, Xuening Zhu wrote:
>> > Hi George:
>> >   I wonder whether pbdR is better for multiple machines (computer
>> cluster)
>> > for speeding up matrix computation? Since I have only a single machine,
>> I
>> > didn't see much better performance here. I tried to compare 'parallel'
>> and
>> > 'pbdDMAT' package in the experiment for matrix multiplication in
>> parallel
>> > below:
>> >
>> > ############################################
>> >
>> > library(parallel)
>> >
>> > mat = matrix(1:1e6,ncol=1e4)
>> > group = sample(rep(1:4,length.out=ncol(mat)))
>> > mm = lapply(split(seq(ncol(mat)),group),function(i) mat[,i])
>> >
>> > system.time({
>> >    a = mclapply(mm,function(m){
>> >      m%*%t(m)
>> >    },mc.cores=2)
>> >    b = Reduce("+",a)
>> > })
>> >
>> > ##############################################
>> >
>> > library(pbdDMAT,quiet=TRUE)
>> > init.grid()
>> > tt = system.time({
>> >    #ScaLAPACK blocking dimension
>> >    bldim<-c(4,4)
>> >    #Generate data on process0, then distribute to theothers
>> >    if(comm.rank()==0){
>> >      mat = matrix(1:1e6,ncol=1e4)
>> >    }else{
>> >      mat=NULL
>> >    }
>> >    dx<-as.ddmatrix(x=mat,bldim=bldim)
>> >
>> >    #Computations in parallel
>> >    ddx<-t(dx)%*%dx
>> > })
>> > mm = as.matrix(ddx)
>> >
>> > if(comm.rank()==0){
>> >    print(system.time({
>> >      MM = mat%*%t(mat)
>> >    }))
>> >    print(all.equal(MM,mm))
>> > }
>> >
>> > comm.print(tt)
>> > finalize()
>> >
>> > ###############################################
>> >
>> > The second one takes about 4.561 seconds while the first one takes only
>> > 0.104 seconds.
>> >
>> >
>> >
>> > 2014-02-14 1:21 GMT+08:00 George Ostrouchov <georgeost@gmail.com>:
>> >
>> >> Consider using pbdR. It puts PBLAS and ScaLAPACK at your disposal for
>> >> FORTRAN speed matrix parallelism without the need to learn their API.
>> While
>> >> built for truly big machines, you will see a lot of benefit with your
>> size
>> >> machine already. Start with pbdDEMO to learn the basics. It is batch
>> >> computing (because that's what's done on big machines) with Rscript
>> but the
>> >> speed and simplicity are worth it!
>> >>
>> >> Cheers,
>> >> George
>> >>
>> >>
>> >> On 2/13/14 2:32 AM, romunov wrote:
>> >>
>> >>> When doing calculations in parallel, there's also some overhead
>> costs. If
>> >>> computation time per core is short, the overhead costs may exceed the
>> >>> computation time, time-wise raising the cost of parallel task.
>> >>>
>> >>> Cheers,
>> >>> Roman
>> >>>
>> >>>
>> >>> On Thu, Feb 13, 2014 at 5:26 AM, Xuening Zhu <puddingnnn529@gmail.com
>> >
>> >>> wrote:
>> >>>
>> >>>   I am learning about parallel computing in R , and I found this
>> happening
>> >>>> in
>> >>>> my experiments.
>> >>>>
>> >>>> Briefly, in the following example, why are most values of user time
>> in
>> >>>> t smaller
>> >>>> than that in mc_t ? My machine has 32GB memory, 2 cpus with 4 cores
>> and 8
>> >>>> hyper threads in total. Tools such as BLAS to enhance performance
>> aren't
>> >>>> installed as well.
>> >>>>
>> >>>> system.time({t = lapply(1:4,function(i) {
>> >>>>       m = matrix(1:10^6,ncol=100)
>> >>>>       t = system.time({
>> >>>>           m%*%t(m)
>> >>>>       })
>> >>>>       return(t)})})
>> >>>>
>> >>>>
>> >>>> library(multicore)
>> >>>> system.time({
>> >>>>       mc_t = mclapply(1:4,function(m){
>> >>>>           m = matrix(1:10^6,ncol=100)
>> >>>>           t = system.time({
>> >>>>               m%*%t(m)
>> >>>>           })
>> >>>>           return(t)
>> >>>>       },mc.cores=4)})
>> >>>>
>> >>>>   t[[1]]
>> >>>> user  system elapsed
>> >>>>
>> >>>> 11.136   0.548  11.703
>> >>>>
>> >>>> [[2]]
>> >>>> user  system elapsed
>> >>>>
>> >>>> 11.533   0.548  12.098
>> >>>>
>> >>>> [[3]]
>> >>>> user  system elapsed
>> >>>>
>> >>>> 11.665   0.432  12.115
>> >>>>
>> >>>> [[4]]
>> >>>> user  system elapsed
>> >>>>
>> >>>> 11.580   0.512  12.115
>> >>>>
>> >>>>   mc_t[[1]]
>> >>>> user  system elapsed
>> >>>>
>> >>>> 16.677   0.496  17.199
>> >>>>
>> >>>> [[2]]
>> >>>> user  system elapsed
>> >>>>
>> >>>> 16.741   0.428  17.198
>> >>>>
>> >>>> [[3]]
>> >>>> user  system elapsed
>> >>>>
>> >>>> 16.653   0.520  17.198
>> >>>>
>> >>>> [[4]]
>> >>>> user  system elapsed
>> >>>>
>> >>>> 11.056   0.444  11.520
>> >>>>
>> >>>> mc_t and t measures pure computation time according to my
>> comprehension.
>> >>>>    Such things happens to parLapply in parallel package as well. The
>> >>>> memory
>> >>>> in my machine is enough for that computation. (It just takes a few
>> >>>> percent
>> >>>> of that).
>> >>>>
>> >>>> Also I try to run 4 similar Rscript as below by hand using command
>> >>>> 'Rscript' at the same time on the same machine and save the results.
>> The
>> >>>> elapsed time for each of them is about 12s as well. So I think it
>> may not
>> >>>> be the contention of cores.
>> >>>>
>> >>>> system.time({t = lapply(1,function(i) {
>> >>>>       m = matrix(1:10^6,ncol=100)
>> >>>>       t = system.time({
>> >>>>           m%*%t(m)
>> >>>>       })
>> >>>>       return(t)})})
>> >>>>
>> >>>> So what happened during the parallel? Does mc_t really measure the
>> pure
>> >>>> computation time? Can someone explain the whole process by step in
>> >>>> detail?
>> >>>>
>> >>>> Thanks.
>> >>>>
>> >>>> --
>> >>>>
>> >>>> Xuening Zhu
>> >>>>
>> >>>>           [[alternative HTML version deleted]]
>> >>>>
>> >>>> _______________________________________________
>> >>>> R-sig-hpc mailing list
>> >>>> R-sig-hpc@r-project.org
>> >>>> https://stat.ethz.ch/mailman/listinfo/r-sig-hpc
>> >>>>
>> >>>>
>> >>>
>> >> _______________________________________________
>> >> R-sig-hpc mailing list
>> >> R-sig-hpc@r-project.org
>> >> https://stat.ethz.ch/mailman/listinfo/r-sig-hpc
>> >>
>> >
>> >
>> >
>> >
>> > _______________________________________________
>> > R-sig-hpc mailing list
>> > R-sig-hpc@r-project.org
>> > https://stat.ethz.ch/mailman/listinfo/r-sig-hpc
>>
>>
>>         [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> R-sig-hpc mailing list
>> R-sig-hpc@r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-hpc
>>
>
>
>
>  --
>    朱雪宁
> --------------------------------------------------------
>  北京大学 光华管理学院
> 商务统计（2013级硕士）
> 手 机：15600608462
>
>  Xuening Zhu
> Guanghua School of Management, Peking University
> Tel:+86 15600608462
>
>
>


-- 
朱雪宁
--------------------------------------------------------
北京大学 光华管理学院
商务统计（2013级硕士）
手机：15600608462

Xuening Zhu
Guanghua School of Management, Peking University
Tel:+86 15600608462

	[[alternative HTML version deleted]]

