[R] Large loops in R

[Sarah Goslee]

I don't think there's any reason for the calculation you're doing that
you must have the whole matrix in memory, is there?

Unless there's something more than what you've shown us, you're just
taking the sum of elementwise operations. You can read the matrix in
in manageable chunks, take the sum of that chunk and save the single
value. Repeat, then add them all up at the end.

Sarah

On Tue, Dec 4, 2012 at 3:14 PM, Charles Novaes de Santana
<charles.santana at gmail.com> wrote:
> On Tue, Dec 4, 2012 at 8:43 PM, Peter Langfelder
> <peter.langfelder at gmail.com> wrote:
>> On Tue, Dec 4, 2012 at 11:27 AM, Charles Novaes de Santana
>> <charles.santana at gmail.com> wrote:
>>> Dear Michael,
>>>
>>> Thank you for your answer.
>>>
>>> I have 2 matrices. Each position of the matrices is a weight. And I
>>> need to calculate the following sum of differences:
>>>
>>> Considering:
>>> mat1 and mat2 - two matrices (each of them 48000 x 48000).
>>> d1 and d2 - two constant values.
>>>
>>> sum<-0;
>>> for(i in 1:nrows1){
>>>                         for(j in 1:nrows2){
>>>                                         sum<-sum+ ( ( (mat1(i,j)/d1) -
>>> (mat2(i,j)/d2) )^2 )
>>>                                 }
>>>                         }
>>>                 }
>>>
>>> I was wondering if there is a better way to do this sum.
>>
>> sum( (mat1/d1-mat2/d2)^2)
>>
>> Correct me if I'm wrong though - aren't matrices of 48x times 48k
>> larger than what R can handle at present?
>>
>> HTH
>>
>> Peter
>
> hmmm I didn't know that the limitation of R was below this value. I
> found this error message:
>
> "Error in matrix(0, 48000, 48000) : too many elements specified"
>
> but I thought it was a machine limitation (and I was asking for access
> to a better machine in my labs...). Thanks for clarifying it.
>
> Well, when Sarah gave me the answer for my problem, I got a new one :)
> Thank you, Sarah and Peter.
>
> So, is there any other way to "trick R" and allocate such large matrices?
>
> Best,
>
> Charles
> --
> Um axé! :)
>

--
Sarah Goslee
http://www.functionaldiversity.org