[R-sig-hpc] rzmq package
Whit Armstrong
armstrong.whit at gmail.com
Thu Sep 29 04:54:15 CEST 2011
Just a quick post on a new package I've been wrapping up, rzmq.
https://github.com/armstrtw/rzmq
Finding inspiration in JD Long's segue package, and frustration with
the config steps involved in dynamically updating the older debian
distribution that Amazon uses for it's emr machines, I decided to try
to hit the ec2 machines directly using my own ami.
The zmq messaging patterns allow one to distribute jobs across many
nodes, but for now a simple example with only 1 micro instance.
the remote server:
ubuntu at ip-10-243-90-36:~$ cat remote.server2.r
#!/usr/bin/env Rscript
library(rzmq)
context = init.context()
in.socket = init.socket(context,"ZMQ_PULL")
bind.socket(in.socket,"tcp://*:5557")
out.socket = init.socket(context,"ZMQ_PUSH")
bind.socket(out.socket,"tcp://*:5558")
while(1) {
msg = receive.socket(in.socket);
fun <- msg$fun
args <- msg$args
print(args)
ans <- do.call(fun,args)
send.socket(out.socket,ans);
}
ubuntu at ip-10-243-90-36:~$
and the locally executed code:
estimatePi <- function(seed) {
set.seed(seed)
numDraws <- 1e5
r <- .5 #radius... in case the unit circle is too boring
x <- runif(numDraws, min=-r, max=r)
y <- runif(numDraws, min=-r, max=r)
inCircle <- ifelse( (x^2 + y^2)^.5 < r , 1, 0)
sum(inCircle) / length(inCircle) * 4
}
print(system.time(ans <- zmq.lapply(as.list(1:1e2),
estimatePi,
execution.server="tcp://ec2-184-73-102-95.compute-1.amazonaws.com:5557",
sink.server="tcp://ec2-184-73-102-95.compute-1.amazonaws.com:5558")))
print(mean(unlist(ans)))
yields:
warmstrong at krypton:~/dvl/zmq.test/test.ec2$ Rscript lapply.exmaple.r
user system elapsed
0.010 0.010 7.007
[1] 3.140964
Anyway, I'll post up a better example tomorrow that actually uses more
than one machine.
-Whit
More information about the R-sig-hpc
mailing list