[BioC] Repost: marrayNorm 1.1.3 gets stuck

[Gordon Smyth]

>Hi List,
>
>Please excuse the repost. No one responded to my
>previous post -- and it seems to me to be quite
>important.
>
>The problem is the new marrayNorm 1.1.3
>(installed with bioconductor 1.2) -- which seems to get
>stuck in an endless loop

It isn't in an endless loop, it's just very slow.

The problem is the loess function. If you use family="symmetric" and 
iterations=4 to get a robust loess curve rather than just least squares, 
then the function is quite slow with lots of data and you get heaps of 
warnings associated with memory limits and the use of a k-d tree. If you 
use surface="direct" to avoid the k-d tree and stop the warnings, then the 
function is very slow indeed with lots of points. You see the result below 
when you try to run it on 30,000 data points.

Versions of marrayNorm prior to 1.1.3 used least squares for the loess 
curves - much quicker but not ideal as a normalization tool.

I have used some tricks to avoid this sort of speed degradation in the 
limma package. I believe that Jean is in the process of implementing to 
same sort of thing in the marrayNorm package.

Regards
Gordon

>marrayNorm 1.1.3   (installed with bioconductor 1.2)
> > unix.time(experiment1.norm<-maNorm(experiment1, norm="loess"))
>Timing stopped at: 8874.73 19.65 10289.06 0 0
>
>ie I interrupted the process -
>but with  marrayNorm 1.1.1 reinstalled
>
>  marrayNorm 1.1.1
> > unix.time(experiment1.norm<-maNorm(experiment1, norm="loess"))
>[1] 803.99  29.73 843.40   0.00   0.00
>
>
>
>is there a known problem with this package?
>
>                                         bye
>                                         rob
>
>--
>Rob Dunne         Fax: +61 2 9325 3200     Tel: +61 2 9325 3263
>CSIRO Mathematical and Information Sciences     +61 2 9325 3100
>Locked Bag 17, North Ryde, New South Wales, Australia, 1670
><http://matilda.vu.edu.au/~dunne>http://matilda.vu.edu.au/~dunne  Email: 
><https://www.stat.math.ethz.ch/mailman/listinfo/bioconductor>Rob.Dunne at 
>csiro.au