[R-sig-Geo] readRAST6() in {spgrass6}

[Roger Bivand]

On Mon, 16 May 2011, Dylan Beaudette wrote:

> It is hard to know what the upper-limit is on Windows-- as the memory
> is usually fragmented by the operating system.

Correct, the upper limit on other OS is higher in practice.

>
> Since you are using GRASS, I would first try a coarsened version of
> your terrain-shape rasters via the built-in re-sampling functionality.
> While this is not ideal, as it is NN-based, it will give a quick way
> to try out methods.
>
> After you have initialized your GRASS connection in R, execute something like:
>
> system('g.region res=10 -ap')
>
> This will align the region along a 10x10 map unit grid. All further
> interaction with raster data will be automatically re-sampled to this
> grid system.

Right, for some res=, you will get a systematic sample that will let you 
decimate your data. Randomly shifting the window while resampling will 
help you see whether your classification model boundaries change.

As both Dylan and I have said, there is no good reason for calibrating the 
classification model with more data than necessary, even if it were 
possible. The only reason to use very much data would be if discrimination 
between very rare classes is your target, but even then you could stratify 
your sample to get better representation of critical areas.

You seem to be looking for classification signatures, from which you are 
going to predict. Sampling will give you distributions of these 
signatures, which could be used for similation by tile. If you are very 
concerned about the statistical quality of your classification signatures, 
you could go fuzzy, but your main goal is to get to the distributions of 
these signatures, IMO. Once you have them, you can predict. You do not 
need to have GBs of dat in memory to do this adequately.

So one could agree that software (OS, R, whatever) is your limitation, but 
it is only a limitation if you are not able to consider more statistical 
approaches to your apparent problem, which is finding out how to classify 
your input, and then to predict to output.

Roger

>
> Dylan
>
> On Mon, May 16, 2011 at 2:25 PM, Sam Prentice <sep at umail.ucsb.edu> wrote:
>> Thanks you for your suggestions. This error is not affected by choice of
>> mapset, and is not limited by my hardware.
>>
>> Further info: I'm running 64-bit windows, 16MB memory, w/multiple cores.
>> GRASS 6.4.1, R 2.12.2. I'm running standalone R using initGRASS() to set
>> GRASS env parameters. The GRASS files I'm trying to read-in are DEM
>> derivatives, each ~170MB. The intermediate .tmp file in the error message is
>> 300MB.
>>
>> It's been suggested that I am "likely trying to import too much data into
>> R". Assuming non-limiting computer power, what is the upper bound on amount
>> of data that can be read into R from GRASS? If not a fixed amount, what are
>> the main conditions that cause it to vary? I did not see this addressed in
>> the {spgrass6} documentation, so I'm assuming it's a base R limitation, but
>> maybe not? Quantifying this limitation will help determined how to move
>> forward (e.g, resampling at smaller scale versus subsetting my DEM
>> derivatives).
>>
>> Thanks,
>> Sam
>>
>> |-----Original Message-----
>> |From: Pierre Roudier [mailto:pierre.roudier at gmail.com]
>> |Sent: Tuesday, May 10, 2011 3:40 PM
>> |To: Sam Prentice
>> |Cc: r-sig-geo at r-project.org
>> |Subject: Re: [R-sig-Geo] readRAST6() in {spgrass6}
>> |
>> |Could that message have anything to do with the fact that this is a layer
>> from
>> |the PERMANENT mapset?
>> |
>> |My two cents,
>> |
>> |Pierre
>> |
>> |2011/5/11 Sam Prentice <sep at umail.ucsb.edu>:
>> |> Hi,
>> |>
>> |>
>> |>
>> |> I'm running R 2.12.2 via Tinn-R on Windows Server 2008. I'm using R
>> |> for cluster analysis for terrain classification and I'm getting the
>> |> following error when parsing GRASS data into R:
>> |>
>> |>
>> |>
>> |>> x =
>> |>
>> |readRAST6(c("param_elev3","param_crosC_1m","param_longC_1m","param
>> |_slope5","
>> |> param_profC","param_miniC_1m","param_maxiC_1m"))
>> |>
>> |> D:/GRASSdata/Sedgwick2/PERMANENT/.tmp/param_elev3 has GDAL driver
>> |> GTiff
>> |>
>> |> and has 6224 rows and 6242 columns
>> |>
>> |>
>> |>
>> |> Error in deleteDataset(DS) :
>> |>
>> |>                GDAL Error 1: Deleting
>> |> D:/GRASSdata/Sedgwick2/PERMANENT/.tmp/param_elev3 failed:
>> |>
>> |> Permission denied
>> |>
>> |>
>> |>
>> |> On the surface this looked like an issue with user privileges, since I
>> |> do not have admin-level user privileges on this machine. However, this
>> |> has been corrected - I now have permissions on the .tmp directory
>> |> listed in the error, and I can create, append, and delete any file in
>> |> that location, but the error is still occurring.
>> |>
>> |>
>> |>
>> |> Thoughts?
>> |>
>> |>
>> |>
>> |> Thanks,
>> |>
>> |> Sam
>> |>
>> |>
>> |>        [[alternative HTML version deleted]]
>> |>
>> |> _______________________________________________
>> |> R-sig-Geo mailing list
>> |> R-sig-Geo at r-project.org
>> |> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>> |>
>> |
>> |
>> |
>> |--
>> |Scientist
>> |Landcare Research, New Zealand
>>
>> _______________________________________________
>> R-sig-Geo mailing list
>> R-sig-Geo at r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>
>
> _______________________________________________
> R-sig-Geo mailing list
> R-sig-Geo at r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>

-- 
Roger Bivand
Economic Geography Section, Department of Economics, Norwegian School of
Economics and Business Administration, Helleveien 30, N-5045 Bergen,
Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43
e-mail: Roger.Bivand at nhh.no