Hello,

I am trying to do some fairly straightforward data summarization, i.e., the
kind you would do with a pivot table in excel or by using SQL queires. I
have a moderately sized data set of ~70,000 records and I am trying to
compute some group averages and sum values within groups. the code example
below shows how I am trying to go about doing this

pti <-rnorm(70000,10)
fid <- rnorm(70000,100)
finc <- rnorm(70000,1000)


### compute the sums of pti within fid groups
sum_pinc <-aggregate(cbind(fid,pti),list(fid),FUN=sum)

#### compute mean finc within fid groups
tot_finc <- aggregate(cbind(fid,finc),list(fid),FUN=mean)

when I try to do it this way I get an error message telling me that enough
memory cannot be allocated ( I am using R 2.7.1 on Windows XP with 2 GB of
Memory). I figure that there must be a more efficent way to go about doing
this. Please suggest.

I would typically do this kind of task in a database and use SQL to push the
data around. I know RODBC allows you to write SQL to query external DBs. Is
there any mechanisim that allows you to write SQL queies against datasets
internal to R e.g. in the case above


I could do something like

set <- cbind(fid,pti,finc)

select fid, sum(pti)
from set
group by fid

that would be handy!

Thanks,

Spencer

	[[alternative HTML version deleted]]

