[R-SIG-Finance] Running R as a server or in a cluster

[Joshua Reich]

Yes - its JBOC (just a bunch of computers). You provide them with a disk
image (of sorts) and they will load it on to as many computers as you
request. Images are loaded and machines are requested via a web services
API. Initially you can request up to 20 machines - but if you email them
you can ask for more. All network bandwidth between machines is free,
but there is a per GB transfer charge for external connectivity - I
can't recall what the rate is, but it is very reasonable. 

Not being a specialized grid environment, all inter-node communication
and scheduling has to be handled by your own application. But for the
price, that's not too bad.

While I was aware of SNOW, I'm not familiar with the other clustering
approaches mentioned earlier in this thread. What special sauce does Sun
provide to make running on a grid easier than running on a JBOC style
setup?

Josh

-----Original Message-----
From: Jeff Ryan [mailto:jeff.a.ryan at gmail.com] 
Sent: Wednesday, September 26, 2007 10:18 AM
To: Joshua Reich
Cc: Brian G. Peterson; r-sig-finance at stat.math.ethz.ch
Subject: Re: [R-SIG-Finance] Running R as a server or in a cluster

I do know the Sun one is using their grid software, and is supposedly
highly secure.  Basically have access to a 2000 node opteron cluster.

The Amazon one seems to be more of using a machine, one at a time.  Is
that correct?


On 9/26/07, Joshua Reich <josh at gghc.com> wrote:
> We recently set up a similar environment using Amazon's EC2 service.
> They charge $0.1 per CPU hour. I can't say what our results have been 
> like yet - still ironing out the kinks in our R code. But I will 
> certainly let you all know how it goes.
>
> Our 'clustering' mechanism is very simple. We have written perl 
> scripts that receive data over HTTP, start R, process the data, and 
> then post the results back via HTTP to a central server.
>
> Josh
>
> -----Original Message-----
> From: r-sig-finance-bounces at stat.math.ethz.ch
> [mailto:r-sig-finance-bounces at stat.math.ethz.ch] On Behalf Of Jeff 
> Ryan
> Sent: Wednesday, September 26, 2007 10:08 AM
> To: Brian G. Peterson
> Cc: r-sig-finance at stat.math.ethz.ch
> Subject: Re: [R-SIG-Finance] Running R as a server or in a cluster
>
> Hi all,
>
> Short of answers, but I do wonder if anyone has used Sun Microsystems 
> www.network.com for grid work with R.  At 1USD a CPU hr, with R 
> already built - and a working example script on the service - it seems

> like a path worth exploring.
>
> Has anyone given it a try.  I set up an account, but have yet to get 
> the opportunity to try it out.
>
> Here is the link:
>
> http://www.network.com/apps/r_project.html
>
> Jeff Ryan
>
> On 9/26/07, Brian G. Peterson <brian at braverock.com> wrote:
> > Adrian Dragulescu wrote:
> > > We have set up a Condor cluster, see 
> > > http://www.cs.wisc.edu/condor/ and we submit R jobs to the 
> > > cluster.  It works well because Condor has very advanced
scheduling capabilities, job monitoring, etc.
> >
> > Adrian,
> >
> > Could you provide more details?  Are you running Rserve on the 
> > cluster, running "R CMD BATCH", or using Parallel-R?
> >
> > I'd like to suggest that we use this thread to continue to develop 
> > the
>
> > collective knowledge of the r-sig-finance community on distributed 
> > or high-throughput R calculations.
> >
> > Regards,
> >
> >     - Brian
> >
> > _______________________________________________
> > R-SIG-Finance at stat.math.ethz.ch mailing list 
> > https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> > -- Subscriber-posting only.
> > -- If you want to post, subscribe first.
> >
>
> _______________________________________________
> R-SIG-Finance at stat.math.ethz.ch mailing list 
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only.
> -- If you want to post, subscribe first.
>