[R] Threaded R-Java

[Vele Samak]

I was about to write a response until Duncan wrote my exact thoughts. As
he said, I'd be surprised if S-plus is thread-safe or has a threaded
execution capability of S code. This is not something trivial since my
group had similar ideas back in 2000 and tried to find shortcuts, but
basically there aren't any. In fact I corresponded with Duncan on this
for the tcltk package.

Instead, we managed to use a well-developed and highly-scalable
multi-threaded platform (aolserver) to manage access to, running and
synchronization of multiple R processes. The sharing of objects was via
xml and an Oracle database, and the access point was web-based. Lots of
patches and points for breakdown, but it worked very well and was robust
and fast enough to fix itself. 

We tried java as a backup. One way to do this is to reimplement the core
R functionality in java classes, similar to the way jackl is done (tcl
in java). This gives you a major headstart, maybe not in overall
performance, but in scalability and deployability. Or, you can go around
and put a multi-threaded layer over R and its memory management so that
every call to R is a separate thread. The core footprint is replicated
in each thread. Nothing to complicated on paper, but again, time and
multi-threaded experience is a must.

In the end, I'd be surprised if splus can do this since it would compete
with their statserver product even though it isn't multi-threaded.  And
if it's true, I'd love to hear about it.



--
Vele Samak
http://www.velesamak.com 

-----Original Message-----
From: owner-r-help at stat.math.ethz.ch
[mailto:owner-r-help at stat.math.ethz.ch] On Behalf Of Duncan Temple Lang
Sent: Tuesday, September 24, 2002 2:44 PM
To: nathan whitehouse
Cc: r-help at stat.math.ethz.ch
Subject: Re: [R] Threaded R-Java


nathan whitehouse wrote:
> Hello all,
> 
>   We currently have a java web utility that allows multi-threaded data

> analysis using S+.

Is the Java utility multi-threaded and talking to different S-Plus
processes? or are you talking to a single S-Plus process that is itself
multi-threaded?  If it is the latter, I am happily suprised; but I very
much doubt that S-Plus is entirely thread-safe at this point or even
supports threads.  If you are using threads within S-Plus to execute S
code, I would very much like to hear more about it. Typically one just
runs the same code in different S-Plus processes and have them report
back to the multi-threaded daemon that manages the comunication from the
client to the S-Plus server.

> 
>   We want to migrate it to R.  However, the guy who
> preceded me here left some documentation indicating
> that he couldn't implement it because R didn't support threading.  
> Also, he noted that the R-Java
> interface(org.omegahat.R.Java.REvaluator) wasn't
> thread safe even if R was.

Without knowing the details of your system, my guess would be that R and
S-Plus provide the same level of support for a threaded application;
basically they don't. (There are a several ways to get asynchronous
events in either system, but not threading!) So, I would imagine that
your setup will work with R as well as it does with S-Plus but there
will be some practical details to sort out.

The R-Java interface is basically thread safe. One can create different
instances of the REvaluator class and they will act independently.  They
can share objects, but will typically do so via Databases and so will be
synchronized correctly.

However, the guts of the REvaluator class is the collection of 
native routines that allow it to access the R evaluator.
Since the R evaluator is not thread-safe, the REvaluator
class must serialize access to the evaluator. So the REvaluator code is
basically thread-safe, but the underlying system is not.

Threads  and synchronization can be very tricky and require an explicit
model to work correctly.  


> 
>   Can anybody who knows about this comment on it?  I'd
> like to know if this is completed(his documentation
> claimed it was a work in progress?)
> 
>   Thanks a bunch,
> 
>   Nathan Whitehouse
>   Programmer
>   Shaw/Shaulsky Lab
>   Baylor College of Medicine
>   Houston, TX, USA
> 
> 
> __________________________________________________
> 
> New DSL Internet Access from SBC & Yahoo!
> 
> -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.
> -.-.-.-.-
> r-help mailing list -- Read
http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
> Send "info", "help", or "[un]subscribe"
> (in the "body", not the subject !)  To:
r-help-request at stat.math.ethz.ch
>
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._.
_._._._

-- 
_______________________________________________________________

Duncan Temple Lang                duncan at research.bell-labs.com
Bell Labs, Lucent Technologies    office: (908)582-3217
700 Mountain Avenue, Room 2C-259  fax:    (908)582-3340
Murray Hill, NJ  07974-2070       
         http://cm.bell-labs.com/stat/duncan
-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.
-.-.-.-
r-help mailing list -- Read
http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._.
_._._._

-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._