[R] loop is going to take 26 hours - needs to be quicker!

[Jenny Barnes]

Dear R-help,

Thank you for the responses off everyone- you'll be please to hear Duncan that 
using: 
> gpcc.array <- array(gpcc.data2[,5], c(144, 72, 46))
was spot-on, worked like a dream. The data is in the correct places as I checked 
with the text file. It took literally 2 seconds - quite an improvement time on 
the predicted 26 hours :-)

I really really appreciate your help, you're all very very kind people.

Merry Christmas,

Jenny Barnes



>Date: Thu, 14 Dec 2006 08:17:24 -0500
>From: Duncan Murdoch <murdoch at stats.uwo.ca>
>User-Agent: Thunderbird 1.5.0.8 (Windows/20061025)
>MIME-Version: 1.0
>To: Jenny Barnes <jmb at mssl.ucl.ac.uk>
>CC: r-help at stat.math.ethz.ch
>Subject: Re: [R] loop is going to take 26 hours - needs to be quicker!
>Content-Transfer-Encoding: 7bit
>X-MSSL-MailScanner-Information: Please contact the ISP for more information
>X-MSSL-MailScanner: No virus found
>X-MSSL-MailScanner-SpamCheck: not spam, SpamAssassin (score=-4.9, required 5, 
BAYES_00 -4.90)
>
>On 12/14/2006 7:56 AM, Jenny Barnes wrote:
>> Dear R-help,
>> 
>> I have a loop, which is set to take about 26 hours to run at the rate it's 
going 
>> - this is ridiculous and I really need your help to find a more efficient way 
of 
>> loading up my array gpcc.array:
>> 
>> #My data is stored in a table format with all the data in one long column 
>> #running though every longitute, for every latitude, for every year. The 
>> #original data is sotred as gpcc.data2 where dim(gpcc.data2) = [476928,5] 
where 
>> #the 5th column is the data:
>> 
>> #make the array in the format I need [longitude,latitude,years]
>> 
>> gpcc.array <- array(NA, c(144,72,46)) 
>> 
>> n=0
>> for(k in 1:46){
>> for(j in 1:72){
>> for(i in 1:144){
>> n <- n+1
>> gpcc.array[i,j,k] <- gpcc.data2[n,5]
>> print(j)
>> }
>> }
>> }
>> 
>> So it runs through all the longs for every lat for every year - which is the 
>> order the data is running down the column in gpcc.data2 so n increses by 1 
each 
>> time and each data point is pulled off....
>> 
>> It needs to be a lot quicker, I'd appreciate any ideas!
>
>I think the loop above is equivalent to
>
>gpcc.array <- array(gpcc.data2[,5], c(144, 72, 46))
>
>which would certainly be a lot quicker.  You should check that the 
>values are loaded in the right order (probably on a smaller example!). 
>If not, you should change the order of indices when you create the 
>array, and use the aperm() function to get them the way you want afterwards.
>
>Duncan Murdoch

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Jennifer Barnes
PhD student - long range drought prediction
Climate Extremes
Department of Space and Climate Physics
University College London
Holmbury St Mary, Dorking
Surrey
RH5 6NT
01483 204149
07916 139187
Web: http://climate.mssl.ucl.ac.uk