[Rd] How to execute R scripts simultaneously from multiple threads

[Erik van Zijst]

Vladimir Dergachev wrote:
>>Using pipes or shared memory to pass things around to other processes on
>>the same box is very fast indeed, but if we base our design around
>>something like RServe which uses TCP it could be significantly slower.
>>Our R-based system will be running scripts in response to high-volume
>>real-time stock exchange data, so we expect lots of calls to many tiny
>>functions indeed.
> 
> Very interesting :) 
> 
> If you are running RServe on the other box you will need to send data over 
> ethernet anyway (and will probably use TCP). If it is on the same box and you 
> use "localhost" the packets will go over loopback - which would be 
> significantly faster.

I'd like to do as little IO as possible, so I'm looking at using a 
single, very powerfull box and keep everything local.

> At some point (years ago) there was even an argument on some mailiing list 
> (xfree86-devel ?) about whether Xserver should support shared memory as unix 
> socket was "fast enough" - with the other side arguing that when you pass 
> megabyte images around (as in DVD playback) there is non-negligible overhead.

We're currently doing performance tests with the RServe-approach where 
we measure the actual evaluation time of a function. I'm interested in 
the evaluation-time versus overhead ratio. Loopback TCP might work as 
long as this ratio is sufficiently high.

Erik
-- 
Practice is the best of all instructors.
		-- Publilius