Mark,
Thanks for the additional info. In the current version (2.0-31) or 'raster'
I have have introduced a bug in the (updated and now faster) aggregate
function when using the option "expand=FALSE". This has been fixed in
version 2.0-35 which will be available from R-Forge in a couple of hours
and will go to CRAN soon.
Robert

On Thu, Nov 29, 2012 at 7:28 AM, Mark Payne <markpayneatwork@gmail.com>wrote:

> Hi Robert,
>
> Here we are. First the description of the source raster (bathr), then
> the aggregate command, then the min.bath results.
>
> > bathr
> class       : RasterLayer
> dimensions  : 10801, 21601, 233312401  (nrow, ncol, ncell)
> resolution  : 0.01666667, 0.01666667  (x, y)
> extent      : -180.0083, 180.0083, -90.00833, 90.00833  (xmin, xmax, ymin,
> ymax)
> coord. ref. : NA
> data source : in memory
> names       : layer
> values      : -10577, 8430  (min, max)
>
> > min.bath <- aggregate(bathr,fact=60,fun=min,expand=FALSE)
> Warning message:
> In .rasterFromRasterFile(grdfile, band = band, objecttype) :
>   size of values file does not match the number of cells (given the data
> type)
> > file.info(filename(min.bath))$size
> [1] 458
> > min.bath
> class       : RasterLayer
> dimensions  : 180, 360, 64800  (nrow, ncol, ncell)
> resolution  : 1, 1  (x, y)
> extent      : -180.0083, 179.9917, -89.99167, 90.00833  (xmin, xmax, ymin,
> ymax)
> coord. ref. : NA
> data source :
> /tmp/R_raster_tmp/mpayne/raster_tmp_2012-11-29_160211_05063.grd
> names       : layer
> values      : -10577, 4922  (min, max)
>
> Note that the aggregation factor does not fit evenly into the bathr -
> there is one cell in each direction extra. However, the problem does
> not seem to pop up if expand=TRUE
> > min.bath <- aggregate(bathr,fact=60,fun=min,expand=TRUE)
> > min.bath
> class       : RasterLayer
> dimensions  : 181, 361, 65341  (nrow, ncol, ncell)
> resolution  : 1, 1  (x, y)
> extent      : -180.0083, 180.9917, -90.99167, 90.00833  (xmin, xmax, ymin,
> ymax)
> coord. ref. : NA
> data source :
> /tmp/R_raster_tmp/mpayne/raster_tmp_2012-11-29_161114_96636.grd
> names       : layer
> values      : -10577, 4910  (min, max)
>
> >
>
> That seems to be a pretty strong hint in the direction of what
> aggregate is doing. I've also checked the disk space and there is
> plenty available.
>
> Another fun fact - there is a huge difference in execution time with
> and without the expand=TRUE argument..
>
> > system.time(min.bath <- aggregate(bathr,fact=60,fun=min,expand=FALSE))
>    user  system elapsed
>  14.144 133.857 159.024
> Warning message:
> In .rasterFromRasterFile(grdfile, band = band, objecttype) :
>   size of values file does not match the number of cells (given the data
> type)
> > system.time(min.bath <- aggregate(bathr,fact=60,fun=min,expand=TRUE))
>    user  system elapsed
>   6.089  29.250  37.178
> >
>
> Interesting.....
>
> Mark
>
> On 28 November 2012 20:22, Robert J. Hijmans <r.hijmans@gmail.com> wrote:
> > Mark,
> >
> > It means that aggregate create a temporary file (because it is a large
> > dataset) that is not correct. Given then number of cells and the
> datatype,
> > the file should have a certain size, which apparently it has not. A
> common
> > cause of this is a full disk (temp folder) which you can perhaps fix with
> > removeTmpFiles.
> > It could be because of a bug in aggregate. It could be a false alarm.
>  Can
> > yo send me the print of  "min.bath" and of
> > "file.info(filename(min.bath))$size"
> >
> > Robert
> >
> >
> > On Tue, Nov 27, 2012 at 3:37 AM, Mark Payne <markpayneatwork@gmail.com>
> > wrote:
> >>
> >> Hi,
> >>
> >> I have a large raster file (netcdf) that I am trying to process ie
> >>
> >> > bathr
> >> class       : RasterLayer
> >> dimensions  : 10800, 21600, 233280000  (nrow, ncol, ncell)
> >> resolution  : 0.01666667, 0.01666667  (x, y)
> >> extent      : -180, 180, -90, 90  (xmin, xmax, ymin, ymax)
> >> coord. ref. : NA
> >> data source : /work/up/common/bathymetry/ETOPO1/ETOPO1_Ice_c_gmt4.nc
> >> names       : z
> >> values      : -10803, 8333  (min, max)
> >> zvar        : z
> >>
> >> Processing this with aggregate() gives the following warning:
> >>
> >> > min.bath <- aggregate(bathr,fact=out.agg,fun=min,expand=FALSE)
> >> Warning message:
> >> In .rasterFromRasterFile(grdfile, band = band, objecttype) :
> >>   size of values file does not match the number of cells (given the data
> >> type)
> >>
> >> What does this warning mean? I've seen previously on the list that
> >> there was a bug associated with this warning - the message has only
> >> appeared for me since upgrading from about raster 2.0-12 (can't
> >> remember the exact version, unfortunately) to 2.0-31.
> >>
> >> Best wishes,
> >>
> >> Mark
> >>
> >> _______________________________________________
> >> R-sig-Geo mailing list
> >> R-sig-Geo@r-project.org
> >> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
> >
> >
>

	[[alternative HTML version deleted]]

