[R-sig-hpc] rzmq package

Whit Armstrong armstrong.whit at gmail.com
Thu Sep 29 04:54:15 CEST 2011


Just a quick post on a new package I've been wrapping up, rzmq.

https://github.com/armstrtw/rzmq

Finding inspiration in JD Long's segue package, and frustration with
the config steps involved in dynamically updating the older debian
distribution that Amazon uses for it's emr machines, I decided to try
to hit the ec2 machines directly using my own ami.

The zmq messaging patterns allow one to distribute jobs across many
nodes, but for now a simple example with only 1 micro instance.

the remote server:

ubuntu at ip-10-243-90-36:~$ cat remote.server2.r
#!/usr/bin/env Rscript

library(rzmq)
context = init.context()
in.socket = init.socket(context,"ZMQ_PULL")
bind.socket(in.socket,"tcp://*:5557")

out.socket = init.socket(context,"ZMQ_PUSH")
bind.socket(out.socket,"tcp://*:5558")

while(1) {
    msg = receive.socket(in.socket);
    fun <- msg$fun
    args <- msg$args
    print(args)
    ans <- do.call(fun,args)
    send.socket(out.socket,ans);
}
ubuntu at ip-10-243-90-36:~$


and the locally executed code:

estimatePi <- function(seed) {
    set.seed(seed)
    numDraws <- 1e5

    r <- .5 #radius... in case the unit circle is too boring
    x <- runif(numDraws, min=-r, max=r)
    y <- runif(numDraws, min=-r, max=r)
    inCircle <- ifelse( (x^2 + y^2)^.5 < r , 1, 0)

    sum(inCircle) / length(inCircle) * 4
}


print(system.time(ans <- zmq.lapply(as.list(1:1e2),
                                    estimatePi,

execution.server="tcp://ec2-184-73-102-95.compute-1.amazonaws.com:5557",

sink.server="tcp://ec2-184-73-102-95.compute-1.amazonaws.com:5558")))

print(mean(unlist(ans)))

yields:
warmstrong at krypton:~/dvl/zmq.test/test.ec2$ Rscript lapply.exmaple.r
   user  system elapsed
  0.010   0.010   7.007
[1] 3.140964


Anyway, I'll post up a better example tomorrow that actually uses more
than one machine.

-Whit



More information about the R-sig-hpc mailing list