[R] computational capacity of Linux network

[Warnes, Gregory R]

Steve Wisdom [swisdom at arbitrade.com] pointed out a typo:  The URL for
ClusterNFS should be http://clusternfs.sourceforge.net.

-Greg



-----Original Message-----
From: Warnes, Gregory R [mailto:gregory_r_warnes at groton.pfizer.com]
Sent: Monday, September 17, 2001 7:44 PM
To: 'Vadim Ogranovich'
Cc: 'r-help at stat.math.ethz.ch'
Subject: RE: [R] computational capacity of Linux network



Hi Vadim,

I have set up just such a system using MOSIX (http://www.mosix.org) and
ClusterNFS (http://ClusterNFS.sourceforege.net). MOSIX automatically
shuffles processes around a network of Linux machines to maximize
performance & make use of memory.  I've successfully used R, C, and Java
programs in this environment and have been very happy. 

<blatant advertisement>
I developed ClusterNFS to simplify the setup and maintenance of a group of
Linux machines by allowing all of the machines to share a common root
filesystem.  This is accomplished by providing 'interpreted file names' so
that configuration files can differ as necessary, while maintaining the
common file system.
</blatant advertisement>

I used them both with Debian, but they can be used with any distribution.  

-Greg


-----Original Message-----
From: Vadim Ogranovich [mailto:vograno at arbitrade.com]
Sent: Monday, September 17, 2001 3:41 PM
Cc: R-help at stat.math.ethz.ch
Subject: [R] computational capacity of Linux network


Hi, This is not an R question per ce, but I feel like this is a right
community to ask it.

As a part of our work we run a lot of non-interactive computational jobs. To
increase the throughput we would like to distribute the load over the entire
network and we are looking at Linux network as a platform. Ideally we would
like to be able to submit a job to the network, rather than to a computer,
and let the "network" to figure out the resource to run the job on.

I'd like to hear from those people who have had an experience of working in
such an environment. Specifically
* what system (Red Hat, etc.) was used? What kind of software was used for
load balancing? 
* How big is the USER overhead to set such a job comparatively to running a
command from shell? How reliable and convenient error reporting is if the
job fails?
* how much of sysadmin time is required to maintain such a network?
* any other comment.

Please reply directly to me and I'll post the summary.

Thanks,
Vadim

-------------------------------------------------- 
DISCLAIMER 
This e-mail, and any attachments thereto, is intended only for use by the
addressee(s) named herein and may contain legally privileged and/or
confidential information.  If you are not the intended recipient of this
e-mail, you are hereby notified that any dissemination, distribution or
copying of this e-mail, and any attachments thereto, is strictly prohibited.
If you have received this e-mail in error, please immediately notify me and
permanently delete the original and any copy of any e-mail and any printout
thereof. 

E-mail transmission cannot be guaranteed to be secure or error-free.  The
sender therefore does not accept liability for any errors or omissions in
the contents of this message which arise as a result of e-mail transmission.

NOTICE REGARDING PRIVACY AND CONFIDENTIALITY 

Knight Trading Group may, at its discretion, monitor and review the content
of all e-mail communications. 

-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.
-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._.
_._


LEGAL NOTICE
Unless expressly stated otherwise, this message is confidential and may be
privileged. It is intended for the addressee(s) only. Access to this E-mail
by anyone else is unauthorized. If you are not an addressee, any disclosure
or copying of the contents of this E-mail or any action taken (or not taken)
in reliance on it is unauthorized and may be unlawful. If you are not an
addressee, please inform the sender immediately.
-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.
-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._.
_._
-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._