[R-sig-Geo] read shapefile into R larger than 2GB

[Roger Bivand]

On Wed, 4 Apr 2012, Rainer M Krug wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 03/04/12 20:44, Carlos Valenzuela wrote:
>> Hello all, I was hoping someone may be able to help me with this 
>> problem. I am trying to read a shapefile into R that is larger than 
>> 2GB. I?ve tried
>
> In my opinion, a 2GB shape file is insane....

Yes, it doesn't seem well-considered. Is this LiDAR data? Are you 
intending to do statistics on the complete data set? What would you infer 
from the results?

You have not included the output of sessionInfo() - I suspect that you are 
using a 32-bit system, which would fail in any case.

Roger

>
> Is the attribute table (.dbf) file as big or is it the shp? If it is the 
> .dbf, you have to look if you need all attributes. If it is the .shp, 
> you could possibly split the shaope file in more then one actual layer? 
> Also, import into a SpatiaLite or even PostGIS database might help you - 
> then you can easier import a subset of features.
>
> Cheers,
>
> Rainer
>
>
>> using readShapePoly() in spdep as well as the readOGR() in rgdal with no luck.,
>>
>> Using the readShapePoly(), I get:, ?failed on DBF filefseek? on  a series of lines (over
>> 284,000),
>>
>> When using rgdal, I get this:
>>
>> Warning message:, In readOGR(".", "nameoffiles") :, Deleted feature IDs: 284284, 284285,
>> 284286, 284287, 284288, 284289, 284290, 284291, 284292, 284293, 284294, 284295, 284296, 284297,
>> 284298, 284299, 284300, 284301, 284302, 284303, 284304, 284305, 284306, 284307, 284308, 284309,
>> 284310, 284311, 284312, 284313, 284314, 284315, 284316, 284317, 284318, 284319, 284320, 284321,
>> 284322, 284323, 284324, 284325, 284326, 284327, 284328, 284329, 284330, 284331, 284332, 284333,
>> 284334, 284335, 284336, 284337, 284338, 284339, 284340, 284341, 284342, 284343, 284344, 284345,
>> 284346, 284347, 284348, 284349, 284350, 284351, 284352, 284353, 284354, 284355, 284356, 284357,
>> 284358, 284359, 284360, 284361, 284362, 284363, 284364, 284365, 284366, 284367, 284368, 284369,
>> 284370, 284371, 284372, 284373, 284374, 284375, 284376, 284377, 284378, 284379, 284380, 284381,
>> 284382, 284383, 284384, 284385, 284386, 284387, 284388, 284389, 284390, 284391, 284392, 284393,
>> 284394, 284395, 284396, 284397, 284398, 284399, 284400, 284401, 284402, 284403, 284404, 284405,
>> 284 [... truncated],
>>
>> In other words, I only get half of the data imported into R and need to get all of it in.
>>
>> Thank you,
>>
>> Carlos
>>
>> [[alternative HTML version deleted]]
>>
>>
>>
>>
>> _______________________________________________ R-sig-Geo mailing list R-sig-Geo at r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>
>
> - --
> Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation Biology, UCT), Dipl. Phys.
> (Germany)
>
> Centre of Excellence for Invasion Biology
> Stellenbosch University
> South Africa
>
> Tel :       +33 - (0)9 53 10 27 44
> Cell:       +33 - (0)6 85 62 59 98
> Fax :       +33 - (0)9 58 10 27 44
>
> Fax (D):    +49 - (0)3 21 21 25 22 44
>
> email:      Rainer at krugs.de
>
> Skype:      RMkrug
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.11 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAk98CbsACgkQoYgNqgF2egq1jQCggSM/x65ppcpy8oT3FMptgCpP
> dgkAn3Ls1yTRnyk5zxouGA216vU4iKKX
> =eY6g
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> R-sig-Geo mailing list
> R-sig-Geo at r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>

-- 
Roger Bivand
Department of Economics, NHH Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; fax +47 55 95 95 43
e-mail: Roger.Bivand at nhh.no