[R-sig-Geo] Problem with operations on large files ("cannotallocate vector of size xx")

[Kamil Konowalik]

Hi Ákos,
no - it's not that. I tried it already - my maximum RAM is 8 Gb but something is taken by Windows and other things. So even though I see that I should be able to finish operation (after rm() and gc() I have around 7Gb free) I always get "Error: cannot allocate vector of size 6.6 Gb".
But thanks for the suggestion!
Best regards,
Kamil


Dnia 1 lutego 2017 13:52 Bede-Fazekas Ákos <bfalevlist at gmail.com> napisał(a):

> Dear Kamil,
> 
> Have you tried this?:
> memory.limit(10000) # increase the memory limit of R from the default 
> 4006 MB to 10000MB
> See details:
> https://stat.ethz.ch/R-manual/R-devel/library/base/html/Memory-limits.html
> 
> I have 64 bit Windows with 64 bit R, but without memory.limit() I get 
> similar error when working with large data.
> 
> HTH,
> Ákos Bede-Fazekas
> Hungarian Academy of Sciences
> 
> 2017.02.01. 13:19 keltezéssel, Kamil Konowalik írta:
> > Hi All,
> > I have a problem with operations on large datasets. I have a several raster files (43) and I would like to make a PCA on them.
> > I'm using those simple commands:
> >
> > files <- list.files("G:/GIS/raster_TIFF",pattern='tif',full.names=TRUE)
> > s <- raster::stack(files)
> > pca<-princomp(na.omit(values(s)), cor=TRUE)
> >
> > But after the last one I'm getting an error: "cannot allocate vector of size 6.6 Gb".
> > I searched the internet and there are some advices but I couldn't found anything helpful. I'm using Windows 7 64 bit with 8 GB of RAM so it is not an issue with 32 bit. Also I would prefer to find a solution which can use my current resources rather than buying additional RAM. I thought that maybe "bigmemory" package will be a solution but I couldn't make it work. I tried also different file types (ASCII, RASTER, GTiff) but that doesn't help neither.
> > It should be a simple and a common problem but I can't figure answer on my own so if you have any working solution I would appreciate it.
> > Thanks in advance!
> > Best regards,
> > Kamil
> >
> > _______________________________________________
> > R-sig-Geo mailing list
> > R-sig-Geo at r-project.org
> > https://stat.ethz.ch/mailman/listinfo/r-sig-geo
> >
> 
> _______________________________________________
> R-sig-Geo mailing list
> R-sig-Geo at r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>