[R] Large 3d array manipulation
David Winsemius
dwinsemius at comcast.net
Sat Feb 28 04:37:15 CET 2009
On Feb 27, 2009, at 7:49 PM, Duncan Murdoch wrote:
> On 27/02/2009 6:15 PM, Vemuri, Aparna wrote:
>> I have a large 3 dimensional array of size (243,246,768)
>> The first dimension is Rows, second is columns and the third is
>> Time. So for each row and column, I want to calculate the mean of
>> time steps
>> 1:8, 2:9, 3:10 and so on and assign the values to a new array. For
>> this
>> I am using the following script.
>> for(i in 1:243)
>> {
>> for(j in 1:246)
>> {
>> for(k in 1:768)
>> {
>> newVar[i,j,k] <- mean( myVar[i,j,k:k+7])
>> }
>> }
>> }
>> This works, but needless to mention it take a very long time to loop
>> over all the rows, columns and time periods. I was wondering if
>> there is
>> a simpler way to do this.
>
> Yes, vectorize it. I think this would work:
>
> newVar <- array(NA, c(243, 246, 768))
>
> for (k in 1:768)
> newVar[,,k] <- apply(myVar, 1:2, function(x) mean(x[k:(k+7)]))
That's rather interesting. I had not realized that one could use that
construction with apply. I had earlier tried to substitute rollmean
for the inner loop. For one thing I thought that trying to index 768+7
was going to create some problems with out-of-range indexing. I got
the same error with my effort to insert rollmean in the OP's
construction as I do in this construction:
myVar <- array(1:(243*246*768), dim=c(243,246,768))
> myVar[1,1,1:8]
[1] 1 59779 119557 179335 239113 298891 358669 418447
newVar <-array(,dim=c(243,246,768))
library(zoo)
for (k in 1:768) newVar[,,k] <- apply(myVar, 1:2, function(x)
mean(x[k:(k+7)]))
Error in newVar[, , k] <- apply(myVar, 1:2, function(x) rollmean(x,
8)) :
number of items to replace is not a multiple of replacement length
I am guessing that at some point the assignment function cannot
resolve which index in newVar to use. So I tried redimensioning newVar
to only have 761 as it third dimension and using:
myVar[1,1,1:8]
for (k in 1:761) {newVar[ , , ] <- apply(myVar, 1:2, function(x)
mean(x[k:(k+7)])) ; print(k)}
This may be working. This executes in less than 20 seconds:
> newVar <-array(,dim=c(243,246,761))
> str(newVar); Sys.time()
logi [1:243, 1:246, 1:761] NA NA NA NA NA NA ...
[1] "2009-02-27 22:35:48 EST"
> newVar[,,] <- apply(myVar, 1:2, function(x) rollmean(x, 8));
Sys.time()
[1] "2009-02-27 22:35:57 EST"
> str(newVar)
num [1:243, 1:246, 1:761] 209224 269002 328780 388558 448336 ...
>
--
David Winsemius
Heritage Laboratories
More information about the R-help
mailing list