[R-sig-Geo] RasterLayerSparse?

[Jonathan Greenberg]

Hi Nevil (cc'ing r-sig-geo):

How are they sparse, exactly?  Is this something where you could be
using a vector-like format instead (x,y,category time 1, category time
2), etc?  It might be looking towards the spacetime data models for
some inspiration, since they have a lot of different sparse and
semi-sparse formats.

I'm not quite following why a compressed format wouldn't work in terms
of storage space -- raster, and the functions like getValueBlock (for
instance) don't care what raster format the files are in -- in your
example, those 160 layers could be MUCH smaller (on disk) if they are
truly sparse (lots of 0s or similar values), and still provide the
read/write capabilities just like an uncompressed TIFF would.

--j

On Fri, Jul 11, 2014 at 3:37 AM, nevil amos <nevil.amos at gmail.com> wrote:
> Thanks for the suggestion.
>
> I am compiling a raster stack with ~160 layers of 8000x5000 categorical
> rasters I then need to do an apply() to get unique value combinations for
> each corresponding cell in the rasters, so reading from compressed format (
> as I might do if I were plotting them woes not work.
>
>
> On Fri, Jul 11, 2014 at 7:35 AM, Jonathan Greenberg <jgrn at illinois.edu>
> wrote:
>>
>> Nevil:
>>
>> Have you considered using an alternative backing format that has
>> compression, like an LZW Tiff?  Raster can wrap these with no problem
>> but you benefit from a significantly smaller footprint.
>>
>> --j
>>
>> On Fri, Jul 4, 2014 at 11:00 AM, nevil amos <nevil.amos at gmail.com> wrote:
>> > I note mention of RasterLayerSparse in the raster package documentation
>> > e.g. under setValues()
>> >
>> > Is it possible to setValues from a sparse matrix? can the values be
>> > stored
>> > in the raster object in sparse form to save memory?
>> > Trying to create "sparse raster"  the sparse matrix is not accepted:
>> > library(raster)
>> > library(Matrix)
>> > m<-matrix(c(0,0,1),3,3)
>> > M<-Matrix(m)
>> >> M
>> > 3 x 3 sparse Matrix of class "dtCMatrix"
>> >
>> > [1,] . . .
>> > [2,] . . .
>> > [3,] 1 1 1
>> > ext<-extent(c(0,3,0,3))
>> > R<-raster(ext,3,3)
>> > setValues(R,M)
>> > Error in setValues(R, M) : values must be a vector
>> >
>> >         [[alternative HTML version deleted]]
>> >
>> > _______________________________________________
>> > R-sig-Geo mailing list
>> > R-sig-Geo at r-project.org
>> > https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>
>>
>>
>> --
>> Jonathan A. Greenberg, PhD
>> Assistant Professor
>> Global Environmental Analysis and Remote Sensing (GEARS) Laboratory
>> Department of Geography and Geographic Information Science
>> University of Illinois at Urbana-Champaign
>> 259 Computing Applications Building, MC-150
>> 605 East Springfield Avenue
>> Champaign, IL  61820-6371
>> Phone: 217-300-1924
>> http://www.geog.illinois.edu/~jgrn/
>> AIM: jgrn307, MSN: jgrn307 at hotmail.com, Gchat: jgrn307, Skype: jgrn3007
>
>



-- 
Jonathan A. Greenberg, PhD
Assistant Professor
Global Environmental Analysis and Remote Sensing (GEARS) Laboratory
Department of Geography and Geographic Information Science
University of Illinois at Urbana-Champaign
259 Computing Applications Building, MC-150
605 East Springfield Avenue
Champaign, IL  61820-6371
Phone: 217-300-1924
http://www.geog.illinois.edu/~jgrn/
AIM: jgrn307, MSN: jgrn307 at hotmail.com, Gchat: jgrn307, Skype: jgrn3007