[R] bug in as.POSIXct ?

Fri Dec 5 19:00:46 CET 2003

I also have a data acquisition system with a sampling interval of one minute.
Since I'm working on a unix-based platform I don't know if my method 
of loading this kind of data
into R will work for you, but here it is.

This method *requires* that the incoming data ignore any changes to 
and from daylight savings time.

My time-date information is contained in a character vector named 
'dstr', formatted
     yyyy-mm-dd hh:mm:ss
and always with an 8 hour offset from UTC.

     x <- Sys.getenv("TZ")
     Sys.putenv(TZ="GMT")
     tm <- as.POSIXct(dstr)+28800
     Sys.putenv(TZ=x)

Note that 28800 is the number of seconds in 8 hours.

The result, tm, is a POSIXct object that represents local time, 
respecting daylight savings time.
When printed or used for plot axis labels, it is formatted for 
standard time or daylight savings time, as appropriate, and time 
interval calculations are correct.

Using your example (and assuming the times are both 8 hours behind UTC)

ds1 <- '2003-10-26 0:59:59'
ds2 <- '2003-10-26 1:00:00'

x <- Sys.getenv("TZ")
Sys.putenv(TZ="GMT")
dt1 <- as.POSIXct(ds1)+28800
dt2 <- as.POSIXct(ds2)+28800
Sys.putenv(TZ=x)

>  dt2-dt1
Time difference of 1 secs
>  dt1
[1] "2003-10-26 01:59:59 PDT"
>  dt2
[1] "2003-10-26 01:00:00 PST"

You may or may not want your times displayed in standard and daylight 
savings time as appropriate for the current local time. I did.

I suppose that one way to view this situation is that, as others 
pointed out, "2003-10-26 1:00:00" is a time that exists in both 
standard and daylight savings time (in the U.S., at least; I wouldn't 
assume that the transition takes place at the same time elsewhere). 
Since it exists in both it is best to explicitly tell it which one to 
use. If you don't tell it, a choice will be made for you, but no 
matter which choice is made, there is no guarantee that it will be 
the same as what the data acquisition system is doing. The above code 
is a way of telling R which one to use. In my limited experience, 
data acquisition systems (data loggers) don't deal with the 
standard/daylight savings time issue at all.

Note also that if you can arrange it so that your data acquisition 
system writes its date-times in ISO standard format, your R code will 
be simpler and easier.

At the time when I started working on this project, R was the only 
software available to me that correctly handled the transitions to 
and from standard time and daylight savings time, and this is the 
primary reason I started using R. The authors of the POSIX time 
classes deserve a great deal of credit for their contributions in 
this area--and based on the article in the June 2001 issue of R News 
that (I believe) introduced the classes, they are Brian Ripley and 
Kurt Hornik. That article also clearly describes the issues that 
motivated the design of the POSIX time classes.

-Don

p.s.
Here's what it looks like assuming a constant 7 hours behind UTC, 
i.e., always in pacific daylight savings time:

x <- Sys.getenv("TZ")
Sys.putenv(TZ="GMT")
dt1 <- as.POSIXct(ds1)+25200
dt2 <- as.POSIXct(ds2)+25200
Sys.putenv(TZ=x)

>  print(dt2-dt1)
Time difference of 1 secs
>  print(dt1)
[1] "2003-10-26 00:59:59 PDT"
>  print(dt2)
[1] "2003-10-26 01:00:00 PDT"

At 1:42 PM -0600 12/4/03, apjaworski at mmm.com wrote:
>Thanks very much for your response and the earlier one by Professor Ripley.
>
>The general problem is indeed a tricky one.
>
>My particular problem was much simpler.  I had a bunch of data from a data
>acquisition system with sampling interval of 1 minute.  The system used a
>simple compression scheme, where a data point was reported only when the
>change in response was sufficiently large.  For example, a fragment like
>this
>
>       Oct. 26 0:01:00       y1
>       Oct. 26 0:05:00       y2
>
>means that the values for 0:02, 0:03 0:04 where essentially y1.
>
>I needed to "decompress" the data set, i.e., fill in the gaps, so I was
>checking for differences of 1 minute and that is when I discovered the
>"error".
>
>I am not sure what the difference between Oct. 26 0:59:00 and Oct. 26 1:00
>should really be, but in this particular application it had to be 1 minute.
>Otherwise I generated 60 spurious gaps between these two times.
>
>Andy
>
>
>__________________________________
>Andy Jaworski
>518-1-01
>Process Laboratory
>3M Corporate Research Laboratory
>-----
>E-mail: apjaworski at mmm.com
>Tel:  (651) 733-6092
>Fax:  (651) 736-3122
>
>
>|---------+------------------------------->
>|         |           Jason Turner        |
>|         |           <jasont at indigoindust|
>|         |           rial.co.nz>         |
>|         |                               |
>|         |           12/05/2003 02:14    |
>|         |                               |
>|---------+------------------------------->
> 
>>-----------------------------------------------------------------------------------------------------------------------------|
>   | 
>|
>   |      To:       apjaworski at mmm.com 
>|
>   |      cc:       r-help at stat.math.ethz.ch 
>|
>   |      Subject:  Re: [R] bug in as.POSIXct ? 
>|
> 
>>-----------------------------------------------------------------------------------------------------------------------------|
>
>
>
>
>apjaworski at mmm.com wrote:
>
>>  I think that there is a bug in the as.POSIXct function on Windows.
>>
>>   Here is what I get on Win2000, Pentium III machine in R 1.8.1.
>>
>>
>>>dd1 <- ISOdatetime(2003, 10, 26, 0, 59, 59)
>>>dd2 <- ISOdatetime(2003, 10, 26, 1, 0, 0)
>>>dd2 - dd1
>>
>>  Time difference of 1.000278 hours
>>
>>  Now, the 26th of October was the day that change to the standard time
>>  occurred, so I suspect that this has something to do with that.  In fact
>>
>>
>>>dd1
>>
>>  [1] "2003-10-26 00:59:59 Central Daylight Time"
>>
>>>dd2
>>
>>  [1] "2003-10-26 01:00:00 Central Standard Time"
>>
>>  so it looks like the switch from CDT to CST happens at 1:00 (instead of
>>  2:00 ?).
>>
>
>Or, it did happen at 2:00 CDT, when the time fell back one hour to 1:00
>CST.  1:00 am occured twice on that day, once as CDT and once as CST.  R
>picked the last one.  A bit pathological at first glance, but
>date-handling often is.
>
>As for the dd2 - dd1 value, the "correct" value depends which 1:00 am
>was chosen.  On Windows, this should be 1 hour, 1 second, no?  I'm
>thinking 1:00 am CST == 2:00 am CDT, so in CDT entirely, your expression
>is basicly 02:00:00 CDT - 00:59:59 CDT.
>
>This makes me suspect that Linux picked the former 1:00 am, from your
>report.  Since R gets its date intricacies from the OS, there really
>isn't much that can be done about this, until someone builds a full
>POSIX time implementation that takes all the world's locales and time
>zones into account, and welds it into R.  Volunteers?
>
>It's things like this that make me convert everything to UCT (GMT, or
>Zulu, if you prefer).  Not R's fault; stupid calendar tricks are to
>blame here.
>
>Cheers
>
>Jason
>--
>Indigo Industrial Controls Ltd.
>http://www.indigoindustrial.co.nz
>64-21-343-545
>jasont at indigoindustrial.co.nz
>
>______________________________________________
>R-help at stat.math.ethz.ch mailing list
>https://www.stat.math.ethz.ch/mailman/listinfo/r-help

-- 
--------------------------------------
Don MacQueen
Environmental Protection Department
Lawrence Livermore National Laboratory
Livermore, CA, USA