[BioC] openMosix and R

[Seth Falcon]

> Claudio Isella <claudio.isella at ircc.it> writes:
>> in my lab we are evaluating the possibility to aquire a linux farm
>> based on openMosix. In order to do this, we have to evaluate if the
>> application we usually run, are going to improve their performance
>> with this solution.

It really depends on your application.  OpenMosix provides automatic
load balancing of separate individual processes among a set of compute
nodes.  This could benefit you in two ways:

1. You have many people sharing a compute resource, all launching
   jobs.  Instead of trying to organize who should use which server,
   all users can launch jobs on the openMosix head node and the jobs
   will migrate to less busy nodes.

2. You have an application that can take advantage of some parallell
   processing and you have written some support scripts to launch many
   processes (which will be load balanced for you).  IMO, openMosix
   isn't providing all that much help in this case.

>> Therefore I want to ask you if R, and in particular BioC packages
>> can fork in this openMosix enviroment, and if so how to do it.

I have some limited experience mixing R and openMosix.  At that time,
I found the migration hit or miss.  Some R codes would migrate fine,
others would segfault upon migration attempt.

I ended up using the snow package along with Rpvm and ran pvm on top
of the openMosix cluster.  This worked very well.

In summary:

 - I think you will get the most bang for your buck by taking
   advantage of pvm or mpi via snow (it is easier than it sounds).
   Whether or not you have openMosix is less important, but having a
   number of reasonably powerful Linux servers is important :-)

 - openMosix (1 year ago) was something of a maintence headache and
   I'm not sure the advantages it brought were worth it.  For use-case
   #1 above, many jobs just didn't migrate as desdired.  For use-case
   ##2, using snow on top of mpi is a better bet, IMO.

HTH,

+ seth