[BioC] High-performance Bioconductor experiments

[Ramon Diaz-Uriarte]

This might be irrelevant or already well known, and if so please disregard. 
But I feel that several different issues are being discussed here. It is my 
cursory understanding that altough paralelization (pvm) and openMosix can 
coexist peacefully paralelization of R might not be a trivial issue; load 
balancing, however, can be achieved using LVS (linux virtual server 
---http://www.linux-vs.org/), so separate R processes could be started on 
different CPUs, and then the result (the .RData?) combined, which might be 
along the lines of what Greg suggested; LVS and openMosix also seem to get 
along fine.

In our cluster, and given the mixed results we have found with the migration 
of R, we will probably use LVS (instead of relying on openMosix migrating R 
processes). For Michael Benjamin's situation a possible kluge (which I have 
not tried) would be to use LVS to run several R processes (i.e., as many 
processes as disjunct subsets of cel files), and then combine the output.

R.


On Friday 12 December 2003 04:13, Michael Benjamin wrote:
> This is the most exciting, and cryptic, part of the message...
>
> > which you can then normalize and scale.  The normalizing and scaling
> > can, of course also be split up across processors.
>
> How?
>
> I was able to use snow to split up the CEL file readings--it's actually
> not that hard.
>
> cl<-makeCluster(2)
> Readaffy<-function(x){
> 	Data<-ReadAffy(x)
> 	Return(Data)
> }
> Data<-clusterApply(cl, filenames,Readaffy)
>
> I find that pvm is not as easy to use as openMosix, because it doesn't
> autodiscover (or does it?!).  My idea is to make a multinode cluster on
> the base computer using RPVM, then have openMosix farm out the processes
> instead of relying on pvm to do that.
>
> In other words, run RPVM on a single pvm node, multi-openMosix node.
> I'll try that experiment tomorrow.
>
> _______________________________________________
> Bioconductor mailing list
> Bioconductor at stat.math.ethz.ch
> https://www.stat.math.ethz.ch/mailman/listinfo/bioconductor

-- 
Ramón Díaz-Uriarte
Bioinformatics Unit
Centro Nacional de Investigaciones Oncológicas (CNIO)
(Spanish National Cancer Center)
Melchor Fernández Almagro, 3
28029 Madrid (Spain)
Fax: +-34-91-224-6972
Phone: +-34-91-224-6900

http://bioinfo.cnio.es/~rdiaz
PGP KeyID: 0xE89B3462
(http://bioinfo.cnio.es/~rdiaz/0xE89B3462.asc)