[R-sig-Geo] Long time to load shapefiles

[Roger Bivand]

On Thu, 17 Oct 2013, philippe massicotte wrote:

> Thank you everyone for your answers. I think I'll just load it and save 
> the workspace for further uses. I still wonder why this is taking such 
> long time to load whereas in QGIS or ArcGIS it is a matter of seconds.

Why do you wonder? You could check instead, taking subsets of the fields, 
and subsets of the features in your object. You could also use alternative 
formats, but I don't think that the choice of driver is important here. 
Note that few statisticians would consider global analysis of data sets 
this large sensible unless very specific conditions were met; the 
additional variability after the first several thousand observations 
probably doesn't tell you much. Once the data are converted to sp classes, 
things go rather faster, as has been pointed out.

Roger

> Have a great day,Phil
>
>> Date: Thu, 17 Oct 2013 17:19:39 +1100
>> Subject: Re: [R-sig-Geo] Long time to load shapefiles
>> From: mdsumner at gmail.com
>> To: pmassicotte at hotmail.com
>> CC: r-sig-geo at r-project.org
>>
>> I have a similar spec to yours, with i7 instead.
>>
>> I have successfully read it, but first time it crashed (I think
>> because memory was exhausted).
>>
>> It took about 20min.
>>
>> I would work on subsetting the part you want with an ogr2ogr
>> incantation if you want to stay away from GIS:
>> http://www.gdal.org/ogr2ogr.html
>>
>> QGIS or similar would be able to read / crop this no-sweat, but it
>> depends on exactly how you want to subset it, and that may be a topic
>> for another forum
>>
>> Cheers, Mike
>>
>> On Thu, Oct 17, 2013 at 2:18 AM, philippe massicotte
>> <pmassicotte at hotmail.com> wrote:
>>> Hi everyone.
>>> I'm trying to open some shapefiles, but the process is still not complete after 25 minutes.  Looking at the shapefiles in ArcGIS, I find that these files have roughly 500K entries. I was wondering if this processing time to open the shapefiles is normal. Here is my configuration:
>>> Intel i5-2500 @ 3.3 GHz16 GB of RAMWindows 7 (64 bits)
>>> For information, you can download the shapefiles here:
>>> http://www.usna.edu/Users/oceano/pguth/srtm/rivers.zip
>>> Here is my code:
>>> library(rgdal)shape=readOGR("af_riv_15s.shp", layer="af_riv_15s")
>>>
>>> Thank you for your help,Phil
>>>
>>>
>>>
>>>
>>>         [[alternative HTML version deleted]]
>>>
>>> _______________________________________________
>>> R-sig-Geo mailing list
>>> R-sig-Geo at r-project.org
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>
>>
>>
>> --
>> Michael Sumner
>> Hobart, Australia
>> e-mail: mdsumner at gmail.com
>
> 	[[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-Geo mailing list
> R-sig-Geo at r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>

-- 
Roger Bivand
Department of Economics, NHH Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; fax +47 55 95 95 43
e-mail: Roger.Bivand at nhh.no