[Rd] as.Date.character speed improvement suggestion

[McGehee, Robert]

R-Devel,
I store and retrieve a large amount of financial data (millions of rows) in a PostgreSQL database keyed by date (and represented in R by class Date). Unfortunately, I frequently find that a great deal of processing time is spent converting dates from character representations to Date class representations in R, presumably because strptime is not fast for large vectors (>10,000 elements). I'd like to suggest a patch that speeds up the date conversion considerably for most every large date vectors (up to 400x in some real life cases).

I suspect most everyone with large vectors of class Date will find that most of their values are duplicated (repeatedly). (There are, after all, only 36,524 days in a century.) Given this, as.Date.character can be sped up substantially for large vectors by only calling strptime on unique dates and then filling in the calculated values for the entire vector. Since the time savings can be several minutes in real-life cases, I think this enhancement should certainly be considered. Also, in a worst case scenario of a long vector with only one duplicated value, the suggested change does not slow down the calculation.

Here's a proof of concept:
as.Date.character2 <- function(x, ...) {
    if (anyDuplicated(x)) {
        ux <- unique(x)
        idx <- match(x, ux)
        y <- as.Date.character(ux, ...)
        return(y[idx])
    }
    as.Date.character(x, ...)
}

## Example1: Construct a 1-million length character vector of 1000 unique dates
## By considering only unique values, speed is >250x faster

> dtch <- format(sample(Sys.Date()-1:1000, 1e6, replace=TRUE))
> system.time(dt1 <- as.Date.character(dtch))
   user  system elapsed 
 12.630  23.628  36.262
> system.time(dt2 <- as.Date.character2(dtch))
   user  system elapsed 
  0.117   0.019   0.136 
> identical(dt1, dt2)
[1] TRUE


## Example2: In a "worst case" scenario of a 1,000,002 length character of 1,000,001 unique dates
## the new function is not any slower (within error).
> dtch <- format(c(Sys.Date(), Sys.Date()+-5e5:5e5))
> system.time(dt1 <- as.Date.character(dtch))
   user  system elapsed 
 20.264  25.584  45.855
> system.time(dt2 <- as.Date.character2(dtch))
   user  system elapsed 
 20.525  24.809  45.335 
> identical(dt1, dt2)
[1] TRUE

Alternatively, this logic should be built in to strptime itself.

Robert