[R-sig-Geo] write/readOGR issue with date / time strings
Corrie Curtice
corrie.curtice at duke.edu
Tue Nov 15 18:49:18 CET 2011
Hi Roger,
Thanks for your patience. This one is confusing to me as I've done exactly
(well, I think exactly) the same steps in the past and it has worked fine.
The only difference is (a) I'm using Rstudio which made me (b) update my
rgdal library version. I've always converted date/time fields using
as.character or format(). Answers to your questions embedded below...
On 11/12/11 3:17 PM, "Roger Bivand" <Roger.Bivand at nhh.no> wrote:
>On Fri, 11 Nov 2011, Corrie Curtice wrote:
>
>>
>>> sessionInfo()
>> R version 2.13.1 (2011-07-08)
>> Platform: x86_64-apple-darwin9.8.0/x86_64 (64-bit)
>>
>> locale:
>> [1] C/en_US.UTF-8/C/C/C/C
>>
>> attached base packages:
>> [1] stats graphics grDevices utils datasets methods base
>>
>> other attached packages:
>> [1] rgdal_0.7-1 sp_0.9-84
>
>OK, you have the most recent release.
>
>>
>> loaded via a namespace (and not attached):
>> [1] grid_2.13.1 lattice_0.19-30 tools_2.13.1
>>
>> I installed rgdal and PROJ4 as follows:
>>
>> setRepositories(ind=1:2)
>> install.packages("rgdal")
>> install.packages("PROJ.4")
>>
>
>Quoting the loading messages would help, to confirm the version of the
>underlying GDAL binary. The PROJ.4 package is not required or used by
>rgdal. My rgdal version on Fedora reports:
>
>> library(rgdal)
>Loading required package: sp
>Geospatial Data Abstraction Library extensions to R successfully loaded
>Loaded GDAL runtime: GDAL 1.8.1, released 2011/06/04
>Path to GDAL shared files: /usr/local/share/gdal
>Loaded PROJ.4 runtime: Rel. 4.7.1, 23 September 2009, [PJ_VERSION: 470]
>Path to PROJ.4 shared files: (autodetected)
>
Here's the output from loading rgdal:
> library(rgdal)
Geospatial Data Abstraction Library extensions to R successfully loaded
Loaded GDAL runtime: GDAL 1.8.0, released 2011/01/12
Path to GDAL shared files:
/Users/corriecurtice/Library/R/2.13/library/rgdal/gdal
Loaded PROJ.4 runtime: Rel. 4.7.1, 23 September 2009, [PJ_VERSION: 470]
Path to PROJ.4 shared files:
/Users/corriecurtice/Library/R/2.13/library/rgdal/proj
>
So there's a slightly newer GDAL release (1.8.1). I'm in the process of
trying to get that installed but not having a lot of luck. I downloaded
and installed gdal v1.8.1-1 leopard (and dependencies). I assume I need to
re-compile the rgdal package so it sees this new version but I get an
error when trying to do that:
corrie-curtices-macbook-pro:Downloads corriecurtice$ !R
R CMD INSTALL
--configure-args='--with-gdal-config=/Library/Frameworks/GDAL.framework/Pro
grams/gdal-config
--with-proj-include=/Library/Frameworks/PROJ.framework/Headers
--with-proj-lib=/Library/Frameworks/PROJ.framework/unix/lib'
rgdal_0.7-4.tar.gz
* installing to library Œ/Users/corriecurtice/Library/R/2.13/library¹
* installing *source* package Œrgdal¹ ...
gdal-config: /Library/Frameworks/GDAL.framework/Programs/gdal-config
checking for gcc... gcc-4.2 -arch i386 -std=gnu99
checking whether the C compiler works... no
configure: error: in
`/private/var/folders/jD/jDD8BZCHHLC-VOQmq7vTGk+++TI/-Tmp-/RtmpTlKhSZ/R.INS
TALL253fc47f/rgdal':
configure: error: C compiler cannot create executables
See `config.log' for more details
ERROR: configuration failed for package Œrgdal¹
* removing Œ/Users/corriecurtice/Library/R/2.13/library/rgdal¹
* restoring previous Œ/Users/corriecurtice/Library/R/2.13/library/rgdal¹
I can't find the config.log file, the tmp directory referenced is empty
and it's not in my cwd.
>>
>>
>
>I do not see the problem in the R-forge development version, and cannot
>reproduce the problem. This now works in (unreleased) 0.7-4:
>
>library(rgdal)
>dsn <- system.file("vectors", package = "rgdal")[1]
> kiritimati_primary_roads <- readOGR(dsn=dsn,
>layer="kiritimati_primary_roads")
>summary(kiritimati_primary_roads)
>kiritimati_primary_roads$dt <- as.POSIXct("2011-11-11 11:11:11")+(60*0:34)
>summary(kiritimati_primary_roads)
>td <- tempdir()
>writeOGR(kiritimati_primary_roads, td, "kiri1", driver="ESRI Shapefile")
>kiri1 <- readOGR(td, "kiri1")
>summary(kiri1)
>kiri1$dt <- as.POSIXct(as.character(kiri1$dt))
>summary(kiri1)
>
>This failed in 0.7-1 because the POSIXct column is not recognized as also
>POSIXt; this is corrected in 0.7-4. However, this was not the original
>problem. Had you reported the output of:
>
>sapply(spdfUTM at data, class)
>sapply(spdfUTM at data, storage.mode)
>
>we would have known how krillGMTtime was stored - it does not seem to
>have
>been detected as a column requiring conversion to character string, or as
>a column that should trigger an error. In general, as Alex wrote, all
>date
>time columns should be converted to character and converted back on
>reading back from the shapefile. OGR drivers treat field types
>differently, so writeOGR has to take a conservative view of possibilities.
Here's the output from the sapply:
> sapply(spdfUTM at data, class)
krillGMTtime Lat Lon
"factor" "factor" "factor"
> sapply(spdfUTM at data, storage.mode)
krillGMTtime Lat Lon
"integer" "integer" "integer"
To answer Don's question, yes I have bad LAT/LON rows when I first read
them in but they are all 999. I discard them before making my spatial
points data frame, which does in fact use those two columns for the
coordinates. I have another similar dataset where the date/time field
storage.mode is reported as "character" and not "integer", and has the
same issue.
Thanks for the help.
Corrie
>Hope this clarifies,
>
>Roger
>
>> Thanks,
>>
>> Corrie
>>
>>
>>
>> On 11/11/11 5:31 PM, "Roger Bivand" <Roger.Bivand at nhh.no> wrote:
>>
>>> On Fri, 11 Nov 2011, Corrie Curtice wrote:
>>>
>>>> Hello,
>>>>
>>>> Apologies if this has already been addressed somewhere, I did a brief
>>>> search
>>>> of archives but didn't find quite this issue.
>>>
>>> Which versions of what are you using? You should provide the output of
>>> sessionInfo(), the messages given when rgdal loads, and details of how
>>> you
>>> installed rgdal. In addition, a use case would be helpful, but only
>>>when
>>> we know what version you are using. In current rgdal, data frame
>>>columns
>>> of class POSIXt are converted to character strings before writing with
>>> writeOGR().
>>>
>>> Roger
>>>
>>>>
>>>> I'm writing out an ESRI shapefile. My spdf has a date/time field.
>>>> Looks
>>>> like this:
>>>>
>>>>> head(spdfUTM at data)
>>>> krillGMTtime Lat Lon
>>>> 1 2010-05-12 12:34:21 -64.67655969 -62.15040195
>>>> 21 2010-05-12 12:35:12 -64.6771229 -62.1519511
>>>> 38 2010-05-12 12:36:02 -64.67775863 -62.15340614
>>>> 57 2010-05-12 12:36:53 -64.67838269 -62.15494829
>>>> 78 2010-05-12 12:37:43 -64.67901497 -62.15647203
>>>> 100 2010-05-12 12:38:37 -64.67973667 -62.15773224
>>>>
>>>> All three fields are reported to be "factors" by str. writeOGR
>>>>appears
>>>> happy:
>>>>
>>>>> writeOGR(spdfUTM,dd,layer="krillPoints-UTM",driver="ESRI
>>>> Shapefile",verbose=TRUE,overwrite=TRUE)
>>>> $object_type
>>>> [1] "SpatialPointsDataFrame"
>>>> $output_dsn
>>>> [1] "/users/corriecurtice/documents/Data_2010/Shapefiles/"
>>>> $output_layer
>>>> [1] "krillPoints-UTM"
>>>> $output_diver
>>>> [1] "ESRI Shapefile"
>>>> $output_n
>>>> [1] 11179
>>>> $output_nfields
>>>> [1] 3
>>>> $output_fields
>>>> [1] "krillGMTtime" "Lat" "Lon"
>>>> $output_fclasses
>>>> [1] 4 4 4
>>>> $dataset_options
>>>> NULL
>>>> $layer_options
>>>> NULL
>>>> Warning message:
>>>> In writeOGR(spdfUTM, dd, layer = "krillPoints-UTM", driver = "ESRI
>>>> Shapefile", :
>>>> existing layer removed
>>>>
>>>> When I read it right back in again, the date/time field is NAs. This
>>>>is
>>>> also true if I load the shapefile into ArcMap.
>>>>
>>>>> foo <- readOGR(dd,layer="krillPoints-UTM",verbose=TRUE)
>>>> OGR data source with driver: ESRI Shapefile
>>>> Source: "/users/corriecurtice/documents/Data_2010/Shapefiles/", layer:
>>>> "krillPoints-UTM"
>>>> with 11179 features and 3 fields
>>>> Feature type: wkbPoint with 2 dimensions
>>>>
>>>>> head(foo at data)
>>>> krillGMTti Lat Lon
>>>> 1 <NA> -64.67655969 -62.15040195
>>>> 2 <NA> -64.6771229 -62.1519511
>>>> 3 <NA> -64.67775863 -62.15340614
>>>> 4 <NA> -64.67838269 -62.15494829
>>>> 5 <NA> -64.67901497 -62.15647203
>>>> 6 <NA> -64.67973667 -62.15773224
>>>>
>>>> Thoughts? Am I missing something obvious?
>>>>
>>>> I can break it up into separate date and time fields, but since the
>>>> field is
>>>> simply a factor anyways I'm not sure how that would help.
>>>>
>>>> Cheers,
>>>>
>>>> Corrie
>>>>
>>>> ---
>>>> Corrie Curtice
>>>> Research Analyst
>>>> Marine Geospatial Ecology Lab
>>>> Nicholas School of the Environment, Duke University
>>>> http://mgel.env.duke.edu
>>>> em: corrie.curtice at duke.edu
>>>> ph: 252-504-7538
>>>>
>>>>
>>>>
>>>>
>>>> [[alternative HTML version deleted]]
>>>>
>>>> _______________________________________________
>>>> R-sig-Geo mailing list
>>>> R-sig-Geo at r-project.org
>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>>>
>>>
>>> --
>>> Roger Bivand
>>> Department of Economics, NHH Norwegian School of Economics,
>>> Helleveien 30, N-5045 Bergen, Norway.
>>> voice: +47 55 95 93 55; fax +47 55 95 95 43
>>> e-mail: Roger.Bivand at nhh.no
>>>
>>
>> _______________________________________________
>> R-sig-Geo mailing list
>> R-sig-Geo at r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>
>
>--
>Roger Bivand
>Department of Economics, NHH Norwegian School of Economics,
>Helleveien 30, N-5045 Bergen, Norway.
>voice: +47 55 95 93 55; fax +47 55 95 95 43
>e-mail: Roger.Bivand at nhh.no
More information about the R-sig-Geo
mailing list