[BioC] package xps: export.filter issues

[cstrato]

Dear Steven,

Good to hear that my workaround could solve your problem.

RMA normalization with about 2000 microarrays should be no problem, 
since some years ago one user did RMA with all 23000 HGU133_Plus2 arrays 
from GEO. It took about one week and did use about 2.5-3GB RAM.

However, I can give you the same suggestion as to the former user, i.e. 
do RMA stepwise, as shown in my example script "script4xps.R":

# first, load ROOT scheme file and ROOT data file
scheme.test3 <- 
root.scheme(paste(.path.package("xps"),"schemes/SchemeTest3.root",sep="/"))
data.test3 <- root.data(scheme.test3, 
paste(.path.package("xps"),"rootdata/DataTest3_cel.root",sep="/"))

# 1.step: background - rma
data.bg.rma <- bgcorrect.rma(data.test3,"Test3RMABgrd",filedir=datdir)

# 2step: normalization - quantile
data.qu.rma <- 
normalize.quantiles(data.bg.rma,"Test3RMANorm",filedir=datdir)

# 3.step: summarization - medpol
data.mp.rma <- 
summarize.rma(data.qu.rma,"Test3RMAExpr",filedir=datdir,tmpdir="")

If one step fails then you do not need to start from the beginning.

This code from my script is for the Test3 array, so that you have to 
modify it for your HG-U133_Plus_2 arrays.

Please note that if you do stepwise computation you are not allowed to 
define a "tmpdir" for the background step and the normalization step 
since this will result in empty root files due to saving the trees in a 
temporary file. Only for the summarization step it is allowed to define 
a "tmpdir".

Furthermore, in the normalization step I would do:
normalize.quantiles(..., add.data = FALSE)
(This may also be necessary in the summarization step, but hopefully not.)

Finally, I would suggest to test your code first with 6 CEL-files only.

Please let me know of your further progress.

Best regards,
Christian



On 8/21/12 9:35 AM, steven wink wrote:
 > Dear Christian,
 >
 > That does indeed solve it for me, thank you again for your help.
 >
 > In the next few days I plan to use the xps rma function on about 2000
 > microArrays. Do you forsee any problems or have any advice on this? I
 > assume it will take several days? What would be the bottleneck when
 > using xps? If it is pocessor speed, is there a user friendly way to use
 > all 4 of my processors in parrallel?
 >
 > Kind regards
 > Steven