[R-sig-Geo] Replacing values in a RasterBrick

[BEUTEL Terry S]

Thanks Michael.
I tried pulling it all into memory and running on a heftier computer and it now works. I also changed the datatype to INT2U which sped things up further without losing significant information in my data.

Consequently the ff option now probably isn’t necessary for me, but if its simple to do as you say, and speeds up processing and/or pushes up the size of raster/bricks/stacks that users can process it would always help, and I’d be happy to trial it on data here if that would help.

Cheers
Terry


From: Michael Sumner [mailto:mdsumner at gmail.com]
Sent: Thursday, 7 September 2017 6:38 PM
To: BEUTEL Terry S; r-sig-geo at r-project.org
Subject: Re: [R-sig-Geo] Replacing values in a RasterBrick


On Thu, 7 Sep 2017 at 08:22 BEUTEL Terry S <Terry.Beutel at daf.qld.gov.au<mailto:Terry.Beutel at daf.qld.gov.au>> wrote:
Hi.

I'm working with the Raster package. I have a RasterBrick of 374 monthly rainfall rasters called Rain2.


> Rain2
class       : RasterBrick
dimensions  : 374, 315, 117810, 374  (nrow, ncol, ncell, nlayers)
resolution  : 0.05, 0.05  (x, y)
extent      : 137.925, 153.675, -29.275, -10.575  (xmin, xmax, ymin, ymax)
coord. ref. : +proj=longlat +ellps=GRS80 +no_defs
data source : K:\RainStuff\RainRaster.grd
names       :    X198601,    X198602,    X198603,    X198604 ...
min values  :       0.00,       0.00,       0.00,       0.00 ...
max values  :  962.01929,  534.42188,  929.24780,  790.83087 ...



I want to replace the 374 monthly rainfall pixel values at a particular location (defined by the intersection of a single row and a single column) with a different series of 374 values. But when I try the reassignment, I run into memory issues....

As far as I understand it there is no actual partial-write for the this format, it's actually a complicated topic given the range of ways raster can store data either on file or in memory.

A Raster in memory is a long-form matrix with the matrix flattened out (left right top to bottom, different to R's matrix), and the columns are "layers". On disk, the rasterfile (.grd) is a flat binary with an orientation ordering (either band sequential, band interleaved by line or band interleaved by pixel) and so that can't be easily overridden without knowing the details.

Given the size of the file I would first try pulling it all into memory and rewriting in whole:

Rain2 <- readAll(Rain2)

## then run your updates
## ...
##  now write back to file

writeRaster(Rain2, "RainRaster_test.grd")

and check.

If that fails, because your system can't perform this (wasteful, brute force) read and write then you could use ff - it's possible to transparently map an ff array to an existing RasterBrick (in rasterfile.grd format) and update it using ff's efficient memory mapping.

Let  me know if you that sounds promising and you want to explore more, it's a simple thing I should finish up and put on CRAN.

Cheers, Mike.



> Rain2[100,100]<-c(1:374)

Error: cannot allocate vector of size 336.2 Mb



I've also tried using setValues to reassign the values, as per the "Writing functions with the 'raster' package" vignette, but without success e.g. ...

> f2<-

+ function(x,row,col,a){

+ x[row,col]<-setValues(x[row,col],a)

+ return(x)}

>

> a<-c(1:374)

> f2(Rain2,100,101,a)



Error in (function (classes, fdef, mtable)  :

  unable to find an inherited method for function 'setValues' for signature '"matrix"'





I don't work much with the Raster package and I'm sure I'm making an elementary error here, but I'd appreciate any advice on the best way replace these values efficiently.

Many thanks

Terry

------------------------------
The information in this email together with any attachments is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. There is no waiver of any confidentiality/privilege by your inadvertent receipt of this material.
Any form of review, disclosure, modification, distribution and/or publication of this email message is prohibited, unless as a necessary part of Departmental business.
If you have received this message in error, you are asked to inform the sender as quickly as possible and delete this message and any copies of this message from your computer and/or your computer system network.
------------------------------

        [[alternative HTML version deleted]]

_______________________________________________
R-sig-Geo mailing list
R-sig-Geo at r-project.org<mailto:R-sig-Geo at r-project.org>
https://stat.ethz.ch/mailman/listinfo/r-sig-geo<https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_r-2Dsig-2Dgeo&d=DwMFaQ&c=tpTxelpKGw9ZbZ5Dlo0lybSxHDHIiYjksG4icXfalgk&r=WSiYvK4_AlHC5jVoClfL7UaU2gIlHzsDOL_wKj5virk&m=ytXH4ODNC7WcBTPkAGOxRKX_UKtPAi_340UheMnZFnM&s=bC3e32XH9U0dtqHsyPoiSfF3PsFtd3JgsCByvId2nOI&e=>
--
Dr. Michael Sumner
Software and Database Engineer
Australian Antarctic Division
203 Channel Highway
Kingston Tasmania 7050 Australia

	[[alternative HTML version deleted]]