[R] Incrementally building histograms

Andre Nathan andre at digirati.com.br
Thu Nov 6 02:01:57 CET 2008


Hello

I need to build a histogram from data (numbers in the [0,1] interval)
stored in a number of different files. The total amount of data is very
large, so I can't load everything to memory and then simply call hist().
Since what I actually need are the histogram counts, I'm currently doing
it like this:

breaks <- seq(0, 1, by = 0.01)
files <- list.files(pattern = "some pattern")
counts <- 0
for (file in files) {
  data <- scan(file, quiet = T)
  h <- hist(data, plot = F, breaks = breaks)
  counts <- counts + h$counts
}
# and then work with `counts' here

Is there a more efficient and/or idiomatic way to do this?

Thanks,
Andre



More information about the R-help mailing list