[Rd] Speed up code, profiling, optimization, lapply vs. loops

[Roger Peng]

My advice would be to use the profiler 'Rprof()' --- you may find that
the loop is not really the problem. In my experience, there's
relatively little difference between 'lapply' and a 'for' loop,
although 'lapply' can be faster at times.

-roger

On Mon, Jul 6, 2009 at 4:26 AM, Thorn Thaler<thothal at sbox.tugraz.at> wrote:
> High everybody,
>
> currently I'm writinig a package that, for a given family of variance
> functions depending on a parameter theta, say, computes the extended quasi
> likelihood (eql) function for different values of theta.
>
> The computation involves a couple of calls of the 'glm' routine. What I'm
> doing now is to call 'lapply' for a list of theta values and a function,
> that constructs a family object for the particular choice of theta, computes
> the glm and uses the results to get the eql. Not surprisingly the function
> is not very fast. Depending on the size of the parameter space under
> consideration it takes a couple of minutes until the function finishes.
> Testing ~1000 Parameters takes about 5 minutes on my machine.
>
> I know that loops in R are slow more often than not. Thus, I thought using
> 'lapply' is a better way. But anyways, it is just another way of a loop.
> Besides, it involves some overhead for the function call and hence i'm not
> sure wheter using 'lapply' is really the better choice.
>
> What I like to know is to figure out, where the bottleneck lies.
> Vectorization would help, but since I don't think that there is vectorized
> 'glm' function, which is able to handle a vector of family objects. I'm not
> aware if there is any choice aside from using a loop.
>
> So my questions:
> - how can I figure out where the bottleneck lies?
> - is 'lapply' always superior to a loop in terms of execution time?
> - are there any 'evil' commands that should be avoided in a loop, for they
> slow down the computation?
> - are there any good books, tutorials about how to profile R code
> efficiently?
>
> TIA 4 ur help,
>
> Thorn
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>



-- 
Roger D. Peng  |  http://www.biostat.jhsph.edu/~rpeng/