Folks,

I'm checking the structure of a dataframe for duplicate parameters at a 
site station (i.e depth should be measured once, not twice), using 
aggregate to count each parameter within a site station.  The fake data 
below has only 26000 rows, and takes roughly 14 seconds.  My real data has 
750000 rows and I had to stop execution after about an hour.  The by() 
function is faster, but I do not understand how to accurately associate 
those results with my test data.

How can I get this to work faster? I can't shake the feeling that it's 
something simple.  Thanks for any pointers.

df <- data.frame(expand.grid('parameter'=LETTERS
                            ,'station'=letters[1:10]
                            ,'site'=1:100
                            )
                )
df$parameter = as.character(df$parameter)
df$station = as.character(df$station)

df1 <- rbind(df, df[runif(nrow(df))>0.99,])  # add some duplicate 
parameters

tt <- df1
system.time(tt <- aggregate(I(df1$parameter)
                           ,list('site'=df1$site 
                                ,'station'=df1$station
                                ,'parameter'=df1$parameter
                                )
                           ,function(x) { length(na.omit(x)) }
                           )
           )
system.time(tt2 <- by(I(df1$parameter)
                     ,list('site'=df1$site 
                          ,'station'=df1$station
                          ,'parameter'=df1$parameter
                          )
                     ,function(x) { length(na.omit(x)) }
                     ,simplify=TRUE
                     )
           )

cur

-- 
Curt Seeliger, Data Ranger
Raytheon Information Services - Contractor to ORD
seeliger.curt@epa.gov
541/754-4638

	[[alternative HTML version deleted]]


