[R-sig-dyn-mod] Parallel deSolve on AWS

[Jeremy Chacon]

Hi John,

I haven't used AWS, but I run a lot of simulations on my University's super
computer.  I usually use the foreach command with %dopar% from the
doParallel package.  I tell the super computer how many processors I want,
and match this to the number of processors I specify.

Here's a superficial example of a script:

library(doParallel)

processors_available = 8
cl = makeCluster(processors_available)
registerDoParallel(cl)

simulation_parameters = seq(from = 0, to = 1, by = 0.01)

results = foreach(current_parameter = simulation_parameters,
        .combine = rbind) %dopar% {
          data.frame(x = rnorm(100,0,current_parameter))
        }

On Sat, Mar 25, 2017 at 9:53 AM, Daniel Kaschek <
daniel.kaschek at physik.uni-freiburg.de> wrote:

> Dear John,
>
> I wrote a function for my dMod package to automatize the scp and ssh
> processes. You can have a look at the runbg() function in
>
> https://github.com/dkaschek/dMod/blob/development/R/remoteComputing.R
>
> The approach might not be the most elegant. Furthermore, it is written
> with Linux on both the origin and target machine in mind. The function
> is documented and you might just have a look at it. Perhaps you need to
> modify it to your needs. If you have questions, let me know.
>
> Best regards,
> Daniel
>
> On Mon, 2017-03-20 at 13:02 +0530, John Harrold wrote:
> > Howdy Folks,
> >
> > I'm looking at parallelizing some of my code. Mostly I'm thinking
> > about
> > spreading the individual simulations in a Monte Carlo across a bunch
> > of
> > computers.  I'd like to spread this out across computers in a more
> > automated fashion as opposed to just adding cores or ssh-ing to
> > computers
> > is a somewhat manual fashion.  I was going to get a free account on
> > AWS to
> > play around with, so I started looking through different tutorials
> > online.
> > There seems to be quite a few options, and I thought it was likely
> > someone
> > on this forum has some direct experience with this. And I'd like to
> > hear
> > about what folks are using in terms of packages, instances they are
> > using
> > on AWS (or other services if thats the case), etc.
> >
> > Thanks
> > John
> >
> >       [[alternative HTML version deleted]]
> >
> > _______________________________________________
> > R-sig-dynamic-models mailing list
> > R-sig-dynamic-models at r-project.org
> > https://stat.ethz.ch/mailman/listinfo/r-sig-dynamic-models
>
> _______________________________________________
> R-sig-dynamic-models mailing list
> R-sig-dynamic-models at r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-dynamic-models




-- 

*___________________________________________________________________________Jeremy
M. Chacon, Ph.D.*

*Post-Doctoral Associate, Harcombe Lab*
*University of Minnesota*
*Ecology, Evolution and Behavior*

	[[alternative HTML version deleted]]