[R-sig-Geo] Issue with rgrass7 and readRAST

Sun Jul 12 13:27:31 CEST 2020

Hi Roger,

I believe I may have been correct about the memory issue... it's possible,
in GRASS, that I did not restrict my REGION sufficiently, using g.region.
Additionally, I applied a MASK for my country of interest, Mozambique,
before starting R, which yielded:

srtm <- readRAST("mozambique_srtm_patch",cat=FALSE)
Creating BIL support files...
Exporting raster as integer values (bytes=4)
 100%
Warning messages:
1: In showSRID(uprojargs, format = "PROJ", multiline = "NO") :
  Discarded ellps unknown in CRS definition: +proj=merc +lat_ts=0 +lon_0=0
+x_0=0 +y_0=0 +R=6378137 +units=m +no_defs
2: In showSRID(uprojargs, format = "PROJ", multiline = "NO") :
  Discarded datum unknown in CRS definition
> image(srtm)
Error in rasterImage(m, bb[1, 1], bb[2, 1], bb[1, 2], bb[2, 2], interpolate
= interpolate,  :
  cannot allocate memory block of size 16777216 Tb

Still, a huge amount of memory needed (is 16777216 Tb correct??)

My thought was to resample the SRTM data in GRASS to 1km resolution, then
start R, and proceed, which quite honestly, for my purposes, really is
sufficient. By doing this, everything worked perfectly! I was able to read
the srtm data quickly and generate an image. I am puzzled by the "cannot
allocate memory block of size 16777216 Tb" message with my GRASS Region
settings in place.

Thank you for all your help!

Best,
Tom

On Sat, Jul 11, 2020 at 11:02 PM Thomas Adams <tea3rd using gmail.com> wrote:

> Hi Roger,
>
> r.out.bin from GRASS did succeed... r.out.bin
> input=mozambique_srtm_patch using teaiii output=mozambique_srtm_patch null=-999
> Exporting raster as integer values (bytes=2)
>
> A very large file!!
> -rw-r--r--   1 teaiii teaiii 1045368225220 Jul 11 13:26
> mozambique_srtm_patch
>
> Tom
>
>
> On Sat, Jul 11, 2020 at 12:12 PM Roger Bivand <Roger.Bivand using nhh.no> wrote:
>
>> Briefly, please try to run r.out.bin manually, to the same output or
>> another. This is what is failing on theGRASS side.
>>
>> Roger
>>
>> -------
>> Roger Bivand
>> Norwegian School of Economics, Bergen, Norway
>> roger.bivand using nhh.no
>>
>> ------------------------------
>> *Fra:* Thomas Adams <tea3rd using gmail.com>
>> *Sendt:* lørdag 11. juli 2020, 17:59
>> *Til:* Roger Bivand
>> *Kopi:* r-sig-Geo using r-project.org
>> *Emne:* Re: [R-sig-Geo] Issue with rgrass7 and readRAST
>>
>> Hi Roger!
>>
>> Here is what was written in
>> /media/teaiii/development/grass/greensboro_mercator/teaiii/.tmp/teaiii-Leopard-WS:
>>
>> drwxr-xr-x 3 teaiii teaiii   4096 Sep 23  2019 ..
>> -rw-r--r-- 1 teaiii teaiii 131072 Jul  9 10:02 801.0.gpkg
>> -rw-r--r-- 1 teaiii teaiii 131072 Jul  9 10:04 466.0.gpkg
>> -rw-r--r-- 1 teaiii teaiii      0 Jul  9 20:32 801.0
>> -rw-r--r-- 1 teaiii teaiii      0 Jul  9 22:40 188.0
>> -rw-r--r-- 1 teaiii teaiii      0 Jul  9 22:51 222.0
>>
>> And there is plenty of space on the disk drive...
>>
>> Tom
>>
>> On Sat, Jul 11, 2020 at 9:26 AM Roger Bivand <Roger.Bivand using nhh.no> wrote:
>>
>>> Tom
>>>
>>> On Fri, 10 Jul 2020, Thomas Adams wrote:
>>>
>>> > Hi,
>>> >
>>> > I'm using R in combination with GRASS GIS 7.8.3, attempting to read a
>>> GRASS
>>> > raster file into R with the rgrass7 interface. I have done this kind of
>>> > thing countless times previously without problems. It's quite possible
>>> I am
>>> > running into memory issues, but my Ubuntu 18.04 Linux computer has 32
>>> GB of
>>> > RAM.
>>> >
>>>
>>> I have not succeeded in reproducing this, I'm afraid. I created a
>>> location
>>> with cell counts to match, and ran r.out.bin first to check whether
>>> there
>>> were any issues writing the data to a temporary file. Then I ran the
>>> equivalent of your command (readRAST()), again without problems. I'm on
>>> a
>>> 16 GB PC, so the problem is puzzling. I'm curious about the free space
>>> on
>>> your
>>>
>>> /media/teaiii/development/grass/greensboro_mercator/teaiii/.tmp/
>>> teaiii-Leopard-WS
>>>
>>> directory. Did you check whether any output arrived there, since the
>>> error
>>> message is coming from r.out.bin?
>>>
>>> The error is on line 53 in raster/r.out.bin/main.c, when the system
>>> function fwrite() returns an error.
>>>
>>> Thanks for your patience.
>>>
>>> Roger
>>>
>>>
>>> > My GRASS raster file:
>>> >
>>> > Type of Map:  raster               Number of Categories: 0
>>> > Data Type:    CELL
>>> > Rows:         22663
>>> > Columns:      28794
>>> > Total Cells:  652558422
>>> > ~90m resolution
>>> >
>>> > Using...
>>> >
>>> > srtm <- readRAST("mozambique_srtm_patch",cat=FALSE)
>>> >
>>> > After quite some time, I get...
>>> >
>>> > Error in execGRASS("r.out.bin", flags = rOutBinFlags, input =
>>> vname[i],  :
>>> >  The command:
>>> > r.out.bin -b -i input=mozambique_srtm_patch
>>> >
>>> output=/media/teaiii/development/grass/greensboro_mercator/teaiii/.tmp/teaiii-Leopard-WS/mozambique_srtm_patch
>>> > bytes=4 null=-81
>>> > produced an error (1) during execution:
>>> > Creating BIL support files...
>>> > Exporting raster as integer values (bytes=4)
>>> > ERROR: Error writing data
>>> > In addition: Warning messages:
>>> > 1: In showSRID(uprojargs, format = "PROJ", multiline = "NO") :
>>> >  Discarded ellps unknown in CRS definition: +proj=merc +lat_ts=0
>>> +lon_0=0
>>> > +x_0=0 +y_0=0 +R=6378137 +units=m +no_defs
>>> > 2: In showSRID(uprojargs, format = "PROJ", multiline = "NO") :
>>> >  Discarded datum unknown in CRS definition
>>> >
>>> > Libraries loaded include: sp, rgrass7, rgdal, raster...
>>> >
>>> > Any thoughts?
>>> >
>>> > Best,
>>> > Tom
>>> >
>>> >
>>>
>>> --
>>> Roger Bivand
>>> Department of Economics, Norwegian School of Economics,
>>> Helleveien 30, N-5045 Bergen, Norway.
>>> voice: +47 55 95 93 55; e-mail: Roger.Bivand using nhh.no
>>> https://orcid.org/0000-0003-2392-6140
>>> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
>>>
>>
>>
>> --
>> Thomas E Adams, III
>> 1724 Sage Lane
>> Blacksburg, VA 24060
>> tea3rd using gmail.com (personal)
>> tea using terrapredictions.org (work)
>>
>> 1 (513) 739-9512 (cell)
>>
>>
>>
>
> --
> Thomas E Adams, III
> 1724 Sage Lane
> Blacksburg, VA 24060
> tea3rd using gmail.com (personal)
> tea using terrapredictions.org (work)
>
> 1 (513) 739-9512 (cell)
>
>

-- 
Thomas E Adams, III
1724 Sage Lane
Blacksburg, VA 24060
tea3rd using gmail.com (personal)
tea using terrapredictions.org (work)

1 (513) 739-9512 (cell)

	[[alternative HTML version deleted]]