[R-sig-Geo] Replacing values in a RasterBrick

[Michael Sumner]

On Sun, 10 Sep 2017 at 10:45 BEUTEL Terry S <Terry.Beutel at daf.qld.gov.au>
wrote:

> Thanks Michael.
>
> I tried pulling it all into memory and running on a heftier computer and
> it now works. I also changed the datatype to INT2U which sped things up
> further without losing significant information in my data.
>
>
>
> Consequently the ff option now probably isn’t necessary for me, but if its
> simple to do as you say, and speeds up processing and/or pushes up the size
> of raster/bricks/stacks that users can process it would always help, and
> I’d be happy to trial it on data here if that would help.
>
>
>

Thanks!  I'll take you up on that when I look at it again.

Cheers, Mike.



> Cheers
>
> Terry
>
>
>
>
>
> *From:* Michael Sumner [mailto:mdsumner at gmail.com]
> *Sent:* Thursday, 7 September 2017 6:38 PM
> *To:* BEUTEL Terry S; r-sig-geo at r-project.org
> *Subject:* Re: [R-sig-Geo] Replacing values in a RasterBrick
>
>
>
>
>
> On Thu, 7 Sep 2017 at 08:22 BEUTEL Terry S <Terry.Beutel at daf.qld.gov.au>
> wrote:
>
> Hi.
>
> I'm working with the Raster package. I have a RasterBrick of 374 monthly
> rainfall rasters called Rain2.
>
>
> > Rain2
> class       : RasterBrick
> dimensions  : 374, 315, 117810, 374  (nrow, ncol, ncell, nlayers)
> resolution  : 0.05, 0.05  (x, y)
> extent      : 137.925, 153.675, -29.275, -10.575  (xmin, xmax, ymin, ymax)
> coord. ref. : +proj=longlat +ellps=GRS80 +no_defs
> data source : K:\RainStuff\RainRaster.grd
> names       :    X198601,    X198602,    X198603,    X198604 ...
> min values  :       0.00,       0.00,       0.00,       0.00 ...
> max values  :  962.01929,  534.42188,  929.24780,  790.83087 ...
>
>
>
> I want to replace the 374 monthly rainfall pixel values at a particular
> location (defined by the intersection of a single row and a single column)
> with a different series of 374 values. But when I try the reassignment, I
> run into memory issues....
>
>
>
> As far as I understand it there is no actual partial-write for the this
> format, it's actually a complicated topic given the range of ways raster
> can store data either on file or in memory.
>
>
>
> A Raster in memory is a long-form matrix with the matrix flattened out
> (left right top to bottom, different to R's matrix), and the columns are
> "layers". On disk, the rasterfile (.grd) is a flat binary with an
> orientation ordering (either band sequential, band interleaved by line or
> band interleaved by pixel) and so that can't be easily overridden without
> knowing the details.
>
>
>
> Given the size of the file I would first try pulling it all into memory
> and rewriting in whole:
>
>
>
> Rain2 <- readAll(Rain2)
>
>
>
> ## then run your updates
>
> ## ...
>
> ##  now write back to file
>
>
>
> writeRaster(Rain2, "RainRaster_test.grd")
>
>
>
> and check.
>
>
>
> If that fails, because your system can't perform this (wasteful, brute
> force) read and write then you could use ff - it's possible to
> transparently map an ff array to an existing RasterBrick (in rasterfile.grd
> format) and update it using ff's efficient memory mapping.
>
>
>
> Let  me know if you that sounds promising and you want to explore more,
> it's a simple thing I should finish up and put on CRAN.
>
>
>
> Cheers, Mike.
>
>
>
>
>
> > Rain2[100,100]<-c(1:374)
>
> Error: cannot allocate vector of size 336.2 Mb
>
>
>
> I've also tried using setValues to reassign the values, as per the
> "Writing functions with the 'raster' package" vignette, but without success
> e.g. ...
>
> > f2<-
>
> + function(x,row,col,a){
>
> + x[row,col]<-setValues(x[row,col],a)
>
> + return(x)}
>
> >
>
> > a<-c(1:374)
>
> > f2(Rain2,100,101,a)
>
>
>
> Error in (function (classes, fdef, mtable)  :
>
>   unable to find an inherited method for function 'setValues' for
> signature '"matrix"'
>
>
>
>
>
> I don't work much with the Raster package and I'm sure I'm making an
> elementary error here, but I'd appreciate any advice on the best way
> replace these values efficiently.
>
> Many thanks
>
> Terry
>
> ------------------------------
> The information in this email together with any attachments is intended
> only for the person or entity to which it is addressed and may contain
> confidential and/or privileged material. There is no waiver of any
> confidentiality/privilege by your inadvertent receipt of this material.
> Any form of review, disclosure, modification, distribution and/or
> publication of this email message is prohibited, unless as a necessary part
> of Departmental business.
> If you have received this message in error, you are asked to inform the
> sender as quickly as possible and delete this message and any copies of
> this message from your computer and/or your computer system network.
> ------------------------------
>
>         [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-Geo mailing list
> R-sig-Geo at r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_r-2Dsig-2Dgeo&d=DwMFaQ&c=tpTxelpKGw9ZbZ5Dlo0lybSxHDHIiYjksG4icXfalgk&r=WSiYvK4_AlHC5jVoClfL7UaU2gIlHzsDOL_wKj5virk&m=ytXH4ODNC7WcBTPkAGOxRKX_UKtPAi_340UheMnZFnM&s=bC3e32XH9U0dtqHsyPoiSfF3PsFtd3JgsCByvId2nOI&e=>
>
> --
>
> Dr. Michael Sumner
> Software and Database Engineer
> Australian Antarctic Division
> 203 Channel Highway
> Kingston Tasmania 7050 Australia
>
-- 
Dr. Michael Sumner
Software and Database Engineer
Australian Antarctic Division
203 Channel Highway
Kingston Tasmania 7050 Australia

	[[alternative HTML version deleted]]