[R-sig-hpc] parallelising: nodes and cores (Martin Ivanov) [ R-sig-hpc Digest, Vol 55, Issue 6 ]

[Pragnesh Patel]

Hi Martin,

Since you are already running in batch, you can consider
pbdMPI(http://cran.r-project.org/web/packages/pbdMPI/index.html) and
get started with the pbdDEMO
vignette(http://cran.r-project.org/web/packages/pbdDEMO/index.html).
The default is parallel for pbdMPI, there is no master, but workers
can be singled out for specific tasks like print or plot.

To request 3 nodes, each with 2 cores (Assuming you use pbdMPI and no
master involve)
#PBS -l nodes=3:ppn=2 (Put this in your job script)

To run on 3 nodes each with 2 cores
mpirun -npernode 2 -n 6 Rscript test.R (This says 2 processes per
node, Total 6 processes. since I requested 3 nodes) (I am not sure
whether "-npernode" option available on your system or not)

If available try with "-display-map" option as well to view processes
mapping for your job.
mpirun -display-map -npernode 2 -n 6 Rscript test.R

Let me know, if you have any question.

Thanks
Pragnesh




On 4/21/13, r-sig-hpc-request at r-project.org
<r-sig-hpc-request at r-project.org> wrote:
> Send R-sig-hpc mailing list submissions to
> 	r-sig-hpc at r-project.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> 	https://stat.ethz.ch/mailman/listinfo/r-sig-hpc
> or, via email, send a message with subject or body 'help' to
> 	r-sig-hpc-request at r-project.org
>
> You can reach the person managing the list at
> 	r-sig-hpc-owner at r-project.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of R-sig-hpc digest..."
>
>
> Today's Topics:
>
>    1. parallelising: nodes and cores (Martin Ivanov)
>    2. Break command if it is taking too long (Francis Smart)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Sat, 20 Apr 2013 19:19:29 +0200
> From: Martin Ivanov <martin.ivanov at ifg.uni-tuebingen.de>
> To: "R-sig-hpc at r-project.org" <r-sig-hpc at r-project.org>
> Subject: [R-sig-hpc] parallelising: nodes and cores
> Message-ID: <5172CE21.2090600 at ifg.uni-tuebingen.de>
> Content-Type: text/plain; charset=UTF-8; format=flowed
>
> Dear all,
>
> I have a task, which I would like to run in parallel on 3 nodes, each
> with 2 cores.
> So I create an MPI cluster, use clusterSplit to split the task into 3
> subtasks, one for each node.
> My question is: Will the function in the fun argument to clusterApply be
> run in parallel on the 2 available cores on each node?
> Do I need to explicitly parallelise within fun along the 2 available
> cores on the current node?
>
> To be more specific, this is my current setup (testNodes.R):
>
> library(parallel);
> library(snow);
> library(Rmpi);
> cl <- makeCluster(spec=3L, type="MPI", outfile=""); # 3 slave nodes are
> created
> x <- seq_len(20L);
> xseq <- clusterSplit(cl=cl, seq=x);
> y <- clusterApply(cl=cl, x=xseq, fun=function(x)
> list(sysInfo=Sys.info()[c("nodename","machine")], x=x));
> save(x,y, xseq, file="/home-link/epaiv01/test.RData");
> stopCluster(cl=cl);
> mpi.quit()
>
> I submit this script (testNodes.R) to torque via:
>
> #!/bin/bash
> #PBS -l nodes=1:ppn=1+3:ppn=2
> #PBS -l walltime=00:01:00
> #PBS -l pmem=100kb
> . /$HOME/.bashrc
> cd $PBS_O_WORKDIR
> mpirun -np 1 --hostfile ${PBS_NODEFILE}
> /home-link/epaiv01/system/usr/bin/Rscript testNodes.R
>
> I have deliberately chosen 1 node with 1 core for the master process,
> and 3 nodes with 2 cpus for the slave processes, spawned in R.
> The question is, will clusterApply use the 2 cores on each node? How can
> I make sure that the task is not actually
> enaging only 1 core on the current node? Is there a way to actually
> check that?
>
> Any comments will be appreciated.
>
> Best regards,
>
> Martin
>
>
>
>
> --
> Dr. Martin Ivanov
> Eberhard-Karls-Universit?t T?bingen
> Mathematisch-Naturwissenschaftliche Fakult?t
> Fachbereich Geowissenschaften
> Water & Earth System Science (WESS)
> H?lderlinstra?e 12, 72074 T?bingen, Deutschland
> Tel. +4970712974213
>
>
>
> ------------------------------
>
> Message: 2
> Date: Sun, 21 Apr 2013 09:59:12 +0100
> From: Francis Smart <smartfra at msu.edu>
> To: r-sig-hpc at r-project.org
> Subject: [R-sig-hpc] Break command if it is taking too long
> Message-ID:
> 	<CAMHTUxvVPr3FzxupiAy2o-enW3Wzih1UC20SNNaQatQ+FF9wcw at mail.gmail.com>
> Content-Type: text/plain
>
> Dear Users,
>
> I am not sure if this is possible.  I would like a command that is capable
> of running another command that is within it but stops running if the
> command within it is taking too long.
>
> For example it might work like this:
>
> slow.function <- function() {
>   a <<- 0
>   for (i in 1:10^9) {
>    a <<- a+i
>   }
> }
>
> # Calling it might result in the following:
>> flow.control(eval="slow.function", sec.max = 5)
> Warning: function slow.function stopped after 5 seconds.
> $ seconds = 5
> $ break = T
>
>> a
> 345113
>
> fast.function <- function() {
>   a <<- 0
>   for (i in 1:10) {
>    a <<- a+i
>   }
> }
>
>> flow.control(eval="fast.function", sec.max = 5)
> $ seconds = .000001
> $ break = F
>
>> a
> 55
>
> I know I can design a stopping command to kill an individual loop after a
> certain amount of time.  However, I want a catch all function that I can
> plug any function into.
>
> Thank you very much for your consideration,
> Francis
>
> --
> Francis Smart
> --------------------
> PhD Student - Michigan State University
> Measurement and Quantitative Methods (MQM)
>
> Blog: www.EconometricsBySimulation.com
>
> Cell: (563) 223-8108
>
> 	[[alternative HTML version deleted]]
>
>
>
> ------------------------------
>
> _______________________________________________
> R-sig-hpc mailing list
> R-sig-hpc at r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-hpc
>
>
> End of R-sig-hpc Digest, Vol 55, Issue 6
> ****************************************