Hi Rex,

You're right... I missed the proper route of thought :) I should read in a
set of data that I can handle memory wise, calculate the covariance matrix
and add it to the total... Sorry I assessed your suggestion the wrong way.

Thanks a million,

Tsjerk

On Mar 14, 2011 7:58 PM, <rex.dwyer@syngenta.com> wrote:

 Tsjerk,

It seems to me that memory and not time is your big efficiency problem, and
I’ve showed you how to avoid storing your entire input.

If you want to avoid doing each multiplication twice, you can replace the
“outer” with a function that computes each product only once and accumulate
sums of those products.

Iii = matrix(c(rep(1:200,200),rep(1:200,each=200)), ncol=2)

Iii = iii[ iii[,1]<=iii[,2]]



and at each step



V2 = v2+X[iii[,1]] * X[iii[,2]]



instead of M.



I would imagine that the internal cov does this anyway, as you are not the
first person to notice this symmetry, so I’m not sure of the point of this
exercise.



PS: I actually know how to spell and pronounce Tsjerk, but Tsj is not a very
familiar pattern for my fingers.



*From:* Tsjerk Wassenaar [mailto:tsjerkw@gmail.com]
*Sent:* Monday, March 14, 2011 1:41 PM
*To:* Dwyer Rex USRE
*Subject:* Re: RE: [R] *Building* a covariance matrix efficiently

  Hi Rex, Thanks for the reply. But it doesn't solve the issues of redundant
calculations due to...


 ------------------------------
*This message may contain confidential information. If you are not the
designated recipient, please notify the sender immediately, and delete the
original and any copies. Any use of the message by you is prohibited.*

	[[alternative HTML version deleted]]

