[R-sig-hpc] R on a Beowulf Cluster

[Liaw, Andy]

Actually "beowulf" is not really all that generic, I think.  We happen to have a Scyld Beowulf cluster (running ClusterWare on top of CentOS).  Beowulf is very different from other types of cluster in that it has a single master node (in most setup) and there are no login access to slave nodes.  If the master node goes down, the entire cluster is down.  The advantages are that it is single system image (there's one process table for the entire cluster), and no OS/software installation is needed on the slaves.

We've used Rmpi on the cluster.  NWS can work, but need some setup. 

Best,
Andy

> -----Original Message-----
> From: r-sig-hpc-bounces at r-project.org 
> [mailto:r-sig-hpc-bounces at r-project.org] On Behalf Of Paul Johnson
> Sent: Monday, December 20, 2010 3:04 PM
> To: Michael D; R SIG High Performance Computing
> Subject: Re: [R-sig-hpc] R on a Beowulf Cluster
> 
> "Beowulf" is a generic term, it does not mean in particular what kind
> of cluster framework your school has. For example, we run "Rocks"
> cluster linux. You might contact your folks, find out what they are
> doing. Or, if you can log in, you will be able to tell by snooping
> around a bit in /etc, or by running
> 
> $ uname -a
> 
> You need to find out what scheduling framework your administrators
> have set up and which system is your "head node" where you can log in
> and submit jobs to the queue.
> 
> For work with R, you need the ordinary stuff for compiling R programs
> (as you would in any workstation) and for the communications between
> systems, get  OpenMPI.  Rmpi compiles against that, so don't bother
> trying to make R go parallel until you have a working OpenMPI.  It is
> possible to install that in a user account, but usually a sysadmin
> will put it somewhere and tell you where the lib dir is for compiling
> apps.  it will supply an executable "mpirun" or such that is named in
> your scripts.
> 
> We have a new HPC system here and I've been trying to build user
> documentaiton from the ground up because most of the students I'm
> trying to help are not programmers either.
> 
> I have some online "working examples" I'm cultivating.  I work on this
> pretty hard for a few weeks and then I get so frustrated I do other
> things for a month or two. Parallel programming is very frustrating,
> compared with the ordinary kind.
> 
> This URL should be all on one line:
> 
> /http://pj.freefaculty.org/cgi-bin/mw/index.php?title=Main_Pag
> e#2._High_Performance_Computing_on_a_Linux_Cluster
> 
> On Mon, Dec 20, 2010 at 1:54 PM, Michael D <mike409 at gmail.com> wrote:
> > Greetings HPC SIG,
> >
> > I recently found out my school has a HPC project where they 
> use Beowulf
> > clusters and I was really hoping to wield this computing 
> power before I
> > finish my last semester; mostly to be acquainted with the 
> workflow but also
> > to run some large simulations for the course I'll be in.
> > Suggestions for any interesting topics I might try are welcome.
> >
> > What I really need is some idea of what libraries I would 
> need to have in
> > order to run some R jobs on the system. It will take a bit 
> of negotiation
> > with the admins to get the packages I need so I'd like to 
> have as complete a
> > list as possible to start. I can get the details of the 
> setup if that would
> > be necessary.
> >
> > Also, I've seen the list of relevant packages (
> > 
> http://cran.r-project.org/web/views/HighPerformanceComputing.h
> tml) but my
> > knowledge of computing only goes so far (my knowledge of 
> HPC even less so)
> > and my fall-back method of ctrl-f "beowulf" turned up no results.
> >
> > Thanks for the help,
> >
> > Michael DePersio
> > M.S. Candidate
> > Georgetown University Department of Mathematics and Statistics
> >
> >        [[alternative HTML version deleted]]
> >
> > _______________________________________________
> > R-sig-hpc mailing list
> > R-sig-hpc at r-project.org
> > https://stat.ethz.ch/mailman/listinfo/r-sig-hpc
> >
> 
> 
> 
> -- 
> Paul E. Johnson
> Professor, Political Science
> 1541 Lilac Lane, Room 504
> University of Kansas
> 
> _______________________________________________
> R-sig-hpc mailing list
> R-sig-hpc at r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-hpc
> 
Notice:  This e-mail message, together with any attachme...{{dropped:11}}