

--- On Wed, 2/4/09, Feng Li <840116@gmail.com> wrote:
From: Feng Li <840116@gmail.com>
Subject: Re: [R] Passing data among multiple instances
To: "Warren Young" <warren@etr-usa.com>
Cc: r-help@r-project.org
Date: Wednesday, February 4, 2009, 10:19 AM

On Wed, Feb 4, 2009 at 4:02 PM, Warren Young <warren@etr-usa.com> wrote:

> Feng Li wrote:
>
>>
>> I have two R instances running at the same time,
>>
>
> On the same computer, or on different computers?

The first trial is only on my single computer with Quad CPU and more than 2G
mem.


>
>
> Is the number of Rs likely to change, or will it always be just the two?


I am planning to do three tasks at same time. That will be instance A,
instance B and instance C. There tasks are more or less the same. But one
always depends on others results.

>
>
> Is this a simple one-off problem, or are you breaking the problem up into
> pieces so you can throw lots of hardware at it?

This is just for one project. But if this is available, later I will try
more on this!


>
>
>  Is there a simpler way to pass the data in A to B?
>>
>
> Perhaps the simplest option is to write the data structure to a file,
using
> any of the several R ways to do that.  When instance 2 sees that a file is
> available, it slurps its contents in and works on it.  The hard part is
> making the second instance wait until the whole file is written out by the
> first.  You wouldn't want it to read in half the file then hit the end
> because the first process hasn't finished writing out the file.  I
don't see
> any good mechanism in R to fix this.

actually, could this be fixed this way: Instance B waits on instance A for a signal file. Instance A first writes the data file as usual (the file you mentioned above). After instance A finishes the data file, it writes a signal file. Instance B checks on signal file existence but it works on the data file.  
>
> A more robust option is to use sockets.  This is suitable even within a
> single machine.  See ?make.socket.  This solves the "how do I know
when I've
> got the full data structure problem" because the second process can
just
> keep reading until it gets an error indicating that the remote peer closed
> the connection.  Once you have the data structure in string form, you can
> eval() it to get an R object suitable for munching on.  Figuring out how
to
> pass the data might be the hardest part.  deparse() might be the easiest
> way.
>
> If you're hoping to scale this up to lots of processes, look into
Rmpi.
>  This provides a very clean way for an R program on one computer to start
> slaves on other computers and then pass data to them in native R
structures.
>  Setting up MPI itself is not trivial, however.  It's best when you
already
> have a cluster of computers linked with MPI.
>
> ______________________________________________
> R-help@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>



-- 
Feng Li
Department of Statistics
Stockholm University
106 91 Stockholm, Sweden

	[[alternative HTML version deleted]]

______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.



      
	[[alternative HTML version deleted]]


