[R] Averaging 'blocks' of data

[jim holtman]

Here is a way to do it by reading in 60 lines at a time and computing the means:

# create some test data
n <- 360
x <- matrix(runif(360*16800), nrow=16800)
cat(x, file="/tempxx.txt")


# now process the data 60 lines at a time, averaging each 60x60 block
result <- matrix(0, nrow=6, ncol=280)
nextLine <- 1  # next output in the result
# create a list of indices to use to partition the input matrix
colIndex <- split(seq(16800), (seq(16800) - 1) %/% 60)
input <- file("/tempxx.txt", "r")
while (TRUE){
    # use 'scan' to read in 60 lines at a time
    block <- scan(input, what=0, n=60*16800)
    if (length(block) != 60 * 16800) break  # exit if done
    # convert to a matrix
    block <- matrix(block, nrow=60, byrow=TRUE)
    # compute the mean and store it
    result[nextLine,] <- sapply(colIndex, function(.blk){
        mean(block[, .blk])
    })
    nextLine <- nextLine + 1
}


On Sun, Sep 7, 2008 at 4:46 PM, Steve Murray <smurray444 at hotmail.com> wrote:
>
> Gabor - thanks for your suggestion... I had checked the previous post, but I found (as a new user of R) this approach to be too complicated and I had problems gaining the correct output values. If there is a simpler way of doing this, then please feel free to let me know.
>
> Dylan - thanks, your approach is a good start. In answer to your questions, my data are 43200 columns and 16800 rows as a data frame - I will probably have to read the dataset in segments though, as it won't fit into the memory!  I've been able to follow your example - how would I be able to apply this technique for finding the average of each 60 x 60 block?
>
> Any other suggestions are of course welcome!
>
> Many thanks again,
>
> Steve
>
> _________________________________________________________________
> Discover Bird's Eye View now with Multimap from Live Search
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>



-- 
Jim Holtman
Cincinnati, OH
+1 513 646 9390

What is the problem that you are trying to solve?