[R-sig-Geo] writeOGR issue in dbf

Thu Mar 18 20:40:57 CET 2010

On Thu, 18 Mar 2010, Robert J. Hijmans wrote:

> I believe that this is part of the DBF specification (there is also a
> limit on the number of characters in a variable name. 12?). This was
> largely ignored by ArcView (and the versions of excel that write dbf
> files), but apparently not by ArcGIS (apart from the missing warning
> message, that is probably a good thing). Perhaps a future version of
> readOGR could check for and fix variable names that are not DBF
> compliant when writing shp files?

Since OGR uses drivers, incorporating lots of driver-specific fixes is not 
a practical solution, as many others accept field names that DBF chokes 
on. If I get a well-tested patch in R for writeOGR() with no side-effects, 
I might consider it. There is C code in foreign/src/Rdbfwrite.c, but I 
think that it will permit a leading underscore - it crops to 11 characters 
plus a closing \0. The only driver-specific check in writeOGR() now is to 
to warn when KML files not in geographical coordinates are written.

Roger

> Robert
>
> On Thu, Mar 18, 2010 at 9:40 AM, Wayne Richter
> <wxrichte at gw.dec.state.ny.us> wrote:
>> This should indeed be the problem. From the Help: "Field names must start with a letter. Field names must contain only letters, numbers, and underscores." Unfortunately, ESRI software can behave in a cryptic, inconsistent or unhelpful way when it encounters a field name that it doesn't like. A hidden trailing space can really mess you up.
>>
>>
>>
>> Wayne Richter
>> New York State Department of Environmental Conservation
>> 625 Broadway
>> Albany, New York 12233-4756
>> 518-402-8974
>>
>>>>>
>> From:   "Robert J. Hijmans" <r.hijmans at gmail.com>
>> To:     caspar hallmann <caspar.hallmann at gmail.com>
>> CC:     r-sig-geo <r-sig-geo at stat.math.ethz.ch>
>> Date:   3/18/2010 12:30 PM
>> Subject:        Re: [R-sig-Geo] writeOGR issue in dbf
>>
>> I think the problem is an illegal variable name  "dist.m"
>>
>> This works for me:
>>
>> data(meuse)
>> coordinates(meuse)=~x+y
>> names(meuse)[12] = 'distm'
>> writeOGR(meuse,".","test",driver="ESRI Shapefile")
>>
>> Robert
>>
>> On Thu, Mar 18, 2010 at 9:26 AM, Roger Bivand <Roger.Bivand at nhh.no> wrote:
>>> On Thu, 18 Mar 2010, caspar hallmann wrote:
>>>
>>>> Dear list,
>>>>
>>>> I am using writeOGR to create an ESRI shapefile from a
>>>> SpatialPointsDataFrame. When opening the shapefile in ArcMap and ask
>>>> for the attributes, I get an error "A column was specified that does
>>>> not exist".
>>>
>>> I only have other tools using OGR (GRASS, QGIS) and Excel 2003 SP3 on
>>> Windows XP. Your example cannot be reproduced using those. I suspect from
>>> trying to interpret the entrails of ESRI "help", that an integer column is
>>> required reproducing the FID values already encoded in the *.shp. Try:
>>>
>>> meuse$FIDs <- 1:nrow(meuse)
>>>
>>> and write out again. This doesn't match your simple copying to get it to
>>> read, though. If others could comment who have access to Arc (which
>>> version), that would be useful. Please include a copy of the messages
>>> displayed when rgdal is loaded, so that we have control of the versions.
>>>
>>> Does changing locale make a difference?
>>>
>>> Roger
>>>
>>>> Similarly, when opening the dbf of the shapefile in Paradox I get
>>>> "Unable to open table. Corrupt table/index header".
>>>> This does not only happen to my own Dataset but also the meuse data.
>>>>
>>>> data(meuse)
>>>> coordinates(meuse)=~x+y
>>>> class(meuse)
>>>> writeOGR(meuse,".","test",driver="ESRI Shapefile")
>>>> # then try ArcMap or Paradox fails as mentioned above.
>>>> #Strangely, when i do
>>>> write.dbf(read.dbf("test.dbf"),file="test2")
>>>> #then renaming test2 into test, my attribute tables open up fine, and
>>>> Paradox stops complaining.
>>>>
>>>> any clues?
>>>>
>>>> Caspar.
>>>>
>>>>> sessionInfo()
>>>>
>>>> R version 2.10.1 (2009-12-14)
>>>> i386-pc-mingw32
>>>>
>>>> locale:
>>>> [1] LC_COLLATE=Dutch_Netherlands.1252  LC_CTYPE=Dutch_Netherlands.1252
>>>>  LC_MONETARY=Dutch_Netherlands.1252 LC_NUMERIC=C
>>>> [5] LC_TIME=Dutch_Netherlands.1252
>>>>
>>>> attached base packages:
>>>> [1] stats     graphics  grDevices utils     datasets  methods   base
>>>>
>>>> other attached packages:
>>>> [1] pscl_1.03.3        coda_0.13-4        mvtnorm_0.9-9
>>>> automap_1.0-5      MASS_7.3-5         lme4_0.999375-32
>>>> Matrix_0.999375-33 maptools_0.7-29
>>>> [9] lattice_0.18-3     foreign_0.8-39     raster_0.9.9-24
>>>> rgdal_0.6-24       gstat_0.9-66       sp_0.9-57
>>>>
>>>> loaded via a namespace (and not attached):
>>>> [1] grid_2.10.1  tools_2.10.1
>>>>
>>>> _______________________________________________
>>>> R-sig-Geo mailing list
>>>> R-sig-Geo at stat.math.ethz.ch
>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>>>
>>>
>>> --
>>> Roger Bivand
>>> Economic Geography Section, Department of Economics, Norwegian School of
>>> Economics and Business Administration, Helleveien 30, N-5045 Bergen,
>>> Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43
>>> e-mail: Roger.Bivand at nhh.no
>>>
>>> _______________________________________________
>>> R-sig-Geo mailing list
>>> R-sig-Geo at stat.math.ethz.ch
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>>
>>
>> _______________________________________________
>> R-sig-Geo mailing list
>> R-sig-Geo at stat.math.ethz.ch
>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>
>> _______________________________________________
>> R-sig-Geo mailing list
>> R-sig-Geo at stat.math.ethz.ch
>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>
>
> _______________________________________________
> R-sig-Geo mailing list
> R-sig-Geo at stat.math.ethz.ch
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>

-- 
Roger Bivand
Economic Geography Section, Department of Economics, Norwegian School of
Economics and Business Administration, Helleveien 30, N-5045 Bergen,
Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43
e-mail: Roger.Bivand at nhh.no