[R-sig-Geo] Error in basename(x) : path too long

[Ben Tupper]

Hi John,

Using a raster::stack() should the perfect solution for you; I work with very large raster stacks all of the time.  From the introductory vignette https://cran.r-project.org/web/packages/raster/vignettes/Raster.pdf

"A notable feature of the raster package is that it can work with raster
datasets that are stored on disk and are too large to be loaded into memory
(RAM). The package can work with large files because the objects it creates
from these files only contain information about the structure of the data, such
as the number of rows and columns, the spatial extent, and the filename, but it
does not attempt to read all the cell values in memory."

Have you tried something like...

library(raster)
S <- stack(vector_of_filenames)

Ben




> On Jul 31, 2017, at 5:19 AM, John Wasige <johnwasige at gmail.com> wrote:
> 
> Thank you Ben,
> I would like to generate annual sums & means for each year from a list rasters (not stack or brick because of memory issues with R). For 15 years I have 345 individual rasters and frequency of 23 per year. 
> 
> I will be very glad for an idea on how to do that.
> 
> Thanks
> 
> John
> 
> 
> 
> 
> 
> On Sat, Jul 29, 2017 at 5:37 PM, Ben Tupper <btupper at bigelow.org> wrote:
> Hi again,
> 
> A late thought - I'm still on the first cups of coffee.
> 
> It looks to me like you are iterating over a stack to select certain layers to sum.  You could achieve the same outcome with possibly much less work.  The following example will create a sum of 24-layer blocks along a stack of rasters.
> 
> # from https://gist.github.com/btupper/20879e0b46e5ed63d402d7cff424dbb7
> #' Split a vector into groups of MAX (or possibly fewer)
> #'
> #' @param v vector or list to split
> #' @param MAX numeric the maximum size per group
> #' @return a list of the vector split into groups
> split_vector <- function(v, MAX = 200){
>     nv <- length(v)
>     if (nv <= MAX) return(list('1' = v))
>     split(v, findInterval(1:nv, seq(from = 1, to = nv, by = MAX)))
> }
> 
> 
> library(raster)
> 
> N <- 345
> n <- 24
> nc <- 4
> nr <- 3
> 
> R <- raster(matrix(runif(nc*nr), ncol = nc, nrow = nr))
> 
> RR <- stack(lapply(seq_len(N), function(i) R))
> 
> ix <- split_vector(seq_len(N), MAX = n)
> 
> SS <- lapply(ix, function(index) sum(RR[[index]]))
> 
> So, S[[1]], which looks like this...
> 
> $`1`
> class       : RasterLayer
> dimensions  : 3, 4, 12  (nrow, ncol, ncell)
> resolution  : 0.25, 0.3333333  (x, y)
> extent      : 0, 1, 0, 1  (xmin, xmax, ymin, ymax)
> coord. ref. : NA
> data source : in memory
> names       : layer
> values      : 0.9451534, 20.0503  (min, max)
> 
> ... is the sum of the first 24 layers of RR. SS[[2]] will be the sum of the next 24, and so on.
> 
> Is that what you are trying to do?
> 
> Cheers,
> Ben
> 
> 
> 
> 
> > On Jul 29, 2017, at 7:56 AM, John Wasige <johnwasige at gmail.com> wrote:
> >
> > ​Dear all,
> >
> > I am running the script below & I get the following error:
> > Error in basename(x) : path too long
> > ​
> > What could be the problem?
> > Thanks for your help
> > John
> >
> > ​### Script​
> >
> > setwd("I:/Mauritius2001_2015")  # directory of data
> > newlist= read.csv('I:/Mauritius2001_2015/Mauritius.csv',header=F)
> >
> > refr <- raster(paste("I:/Mauritius2001_2015/",newlist[i,1],sep = ""))
> > refr[!is.na(refr)] <- 0
> > for(i in seq(1,345,by=23)){
> >  rsum <- refr
> >  for(p in 0:22){
> >    r <- raster(paste("I:/Mauritius2001_2015/",newlist[i+p],sep = ""))
> >    rsum <- rsum + r
> >  }
> >  #   rsum <- rsum
> >  writeRaster(rsum,
> > filename=paste("D:/Mauritius2001_2015/Annual/",substr(newlist[i],1,6),".tif",sep=''),
> > format="GTiff", overwrite=TRUE)
> > }
> >
> >       [[alternative HTML version deleted]]
> >
> > _______________________________________________
> > R-sig-Geo mailing list
> > R-sig-Geo at r-project.org
> > https://stat.ethz.ch/mailman/listinfo/r-sig-geo
> 
> Ben Tupper
> Bigelow Laboratory for Ocean Sciences
> 60 Bigelow Drive, P.O. Box 380
> East Boothbay, Maine 04544
> http://www.bigelow.org
> 
> Ecocast Reports: http://seascapemodeling.org/ecocast.html
> Tick Reports: https://report.bigelow.org/tick/
> Jellyfish Reports: https://jellyfish.bigelow.org/jellyfish/
> 
> 
> 
> 
> 
> 
> -- 
> John Wasige
> "There are no REGRATES in LIFE, just lessons (Jennifer Aniston)”

Ben Tupper
Bigelow Laboratory for Ocean Sciences
60 Bigelow Drive, P.O. Box 380
East Boothbay, Maine 04544
http://www.bigelow.org

Ecocast Reports: http://seascapemodeling.org/ecocast.html
Tick Reports: https://report.bigelow.org/tick/
Jellyfish Reports: https://jellyfish.bigelow.org/jellyfish/