[R-SIG-Finance] Running R as a server or in a cluster

[Jeff Ryan]

Unfortunately I don't really know, I have one Sun personally - so
clustering my one was easy : )

I think the problem is the same regardless of platform - R isn't
multithreaded, so some additional glue is required, be it SNOW or some
other method.

I will try out the service as soon as I can - maybe in the next week
or so.  Maybe with the 'free' time, a few more could give it a shot
and we could get some sort of feedback.  My dream is to have some sort
of R package to handle the whole process within a local package
framework, though clearly a only a dream at the moment.

Again, has anyone given this a try yet?

Jeff

On 9/26/07, Joshua Reich <josh at gghc.com> wrote:
> Yes - its JBOC (just a bunch of computers). You provide them with a disk
> image (of sorts) and they will load it on to as many computers as you
> request. Images are loaded and machines are requested via a web services
> API. Initially you can request up to 20 machines - but if you email them
> you can ask for more. All network bandwidth between machines is free,
> but there is a per GB transfer charge for external connectivity - I
> can't recall what the rate is, but it is very reasonable.
>
> Not being a specialized grid environment, all inter-node communication
> and scheduling has to be handled by your own application. But for the
> price, that's not too bad.
>
> While I was aware of SNOW, I'm not familiar with the other clustering
> approaches mentioned earlier in this thread. What special sauce does Sun
> provide to make running on a grid easier than running on a JBOC style
> setup?
>
> Josh
>
> -----Original Message-----
> From: Jeff Ryan [mailto:jeff.a.ryan at gmail.com]
> Sent: Wednesday, September 26, 2007 10:18 AM
> To: Joshua Reich
> Cc: Brian G. Peterson; r-sig-finance at stat.math.ethz.ch
> Subject: Re: [R-SIG-Finance] Running R as a server or in a cluster
>
> I do know the Sun one is using their grid software, and is supposedly
> highly secure.  Basically have access to a 2000 node opteron cluster.
>
> The Amazon one seems to be more of using a machine, one at a time.  Is
> that correct?
>
>
> On 9/26/07, Joshua Reich <josh at gghc.com> wrote:
> > We recently set up a similar environment using Amazon's EC2 service.
> > They charge $0.1 per CPU hour. I can't say what our results have been
> > like yet - still ironing out the kinks in our R code. But I will
> > certainly let you all know how it goes.
> >
> > Our 'clustering' mechanism is very simple. We have written perl
> > scripts that receive data over HTTP, start R, process the data, and
> > then post the results back via HTTP to a central server.
> >
> > Josh
> >
> > -----Original Message-----
> > From: r-sig-finance-bounces at stat.math.ethz.ch
> > [mailto:r-sig-finance-bounces at stat.math.ethz.ch] On Behalf Of Jeff
> > Ryan
> > Sent: Wednesday, September 26, 2007 10:08 AM
> > To: Brian G. Peterson
> > Cc: r-sig-finance at stat.math.ethz.ch
> > Subject: Re: [R-SIG-Finance] Running R as a server or in a cluster
> >
> > Hi all,
> >
> > Short of answers, but I do wonder if anyone has used Sun Microsystems
> > www.network.com for grid work with R.  At 1USD a CPU hr, with R
> > already built - and a working example script on the service - it seems
>
> > like a path worth exploring.
> >
> > Has anyone given it a try.  I set up an account, but have yet to get
> > the opportunity to try it out.
> >
> > Here is the link:
> >
> > http://www.network.com/apps/r_project.html
> >
> > Jeff Ryan
> >
> > On 9/26/07, Brian G. Peterson <brian at braverock.com> wrote:
> > > Adrian Dragulescu wrote:
> > > > We have set up a Condor cluster, see
> > > > http://www.cs.wisc.edu/condor/ and we submit R jobs to the
> > > > cluster.  It works well because Condor has very advanced
> scheduling capabilities, job monitoring, etc.
> > >
> > > Adrian,
> > >
> > > Could you provide more details?  Are you running Rserve on the
> > > cluster, running "R CMD BATCH", or using Parallel-R?
> > >
> > > I'd like to suggest that we use this thread to continue to develop
> > > the
> >
> > > collective knowledge of the r-sig-finance community on distributed
> > > or high-throughput R calculations.
> > >
> > > Regards,
> > >
> > >     - Brian
> > >
> > > _______________________________________________
> > > R-SIG-Finance at stat.math.ethz.ch mailing list
> > > https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> > > -- Subscriber-posting only.
> > > -- If you want to post, subscribe first.
> > >
> >
> > _______________________________________________
> > R-SIG-Finance at stat.math.ethz.ch mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> > -- Subscriber-posting only.
> > -- If you want to post, subscribe first.
> >
>