[R-sig-Geo] decimals in NA value using raster::writeRaster with .ascii format
Zack Holden
zaholdenfs at gmail.com
Mon Apr 14 18:54:34 CEST 2014
Hi Edzer,
Thank you for your helpful comments. I am generating ascii files as
inputs to PCraster. I realize now that when producing grids for other
software, I need to be more careful about assigning data type and NA
values to my outputs when I write them. When my work is all done in R,
this hasn't ever been a problem. Using:
writeRaster(file="myraster.asc", datatype="FLT4S", NAval=-Inf )
PCraster reads them correctly.
Thanks all for the helpful suggestions.
Best,
Zack
On Mon, Apr 14, 2014 at 6:58 AM, Edzer Pebesma
<edzer.pebesma at uni-muenster.de> wrote:
>
>
> On 04/11/2014 05:46 PM, Zack Holden wrote:
>> Robert,
>> Thank you for your help. I failed to notice the "datatype vs. dataType
>> difference. Calling 'datatype' does indeed solve my integer test case.
>> However, some of the raster datasets that I am processing are actually
>> floating point. When I call writeRaster(datatype = 'FLT4S'), I still
>> see the long string of decimals in the first value in my ascii grid.
>>
>> Is there any way to handle this in R? From your response, it sounds
>> like this might be an artifact of GDAL, and I may need to manually (or
>> with text manipulations) edit out the decimal strings.
>
> Look for
>
> https://r-forge.r-project.org/scm/viewvc.php/pkg/raster/R/writeAllAscii.R?view=markup&root=raster
>
> and lower the 15, or else change the formatC call, then rebuild raster.
>
> As described earlier, raster does this on purpose: the gdal asciigrid
> reader makes the wrong assumption about the data type when a grid starts
> with too many missing values that look like an integer (like -9999 but
> unlike -9999.0). This has been reported over a decade ago, and may be
> seen as an indication of gdal's liveliness. Not only OpenSSL suffers
> from too few eyes to make bugs shallow.
>
> What software are you using that can't digest the 15-digit float?
>
>>
>> Thanks for any additional suggestions.
>>
>> Javier, thanks for pointing out the available GDAL drivers in
>> PCraster, I'm just getting acquainted with that software program.
>>
>> Cheers,
>> Zack
>>
>>
>> On Fri, Apr 11, 2014 at 8:48 AM, Jonathan Greenberg <jgrn at illinois.edu> wrote:
>>> Robert:
>>>
>>> What is the practical difference between those two? -- does dataType
>>> get passed off to GDAL, while datatype is interpreted by your R code?
>>>
>>> --j
>>>
>>> On Thu, Apr 10, 2014 at 7:06 PM, Robert J. Hijmans <r.hijmans at gmail.com> wrote:
>>>> Zack,
>>>>
>>>> You are using: dataType="INT2U"
>>>>
>>>> but it should be: datatype="INT2U" # (or any other INT data type)
>>>>
>>>> That fixes it. If the datatype is FLT* (the default), the first value
>>>> gets decimals to assure that GDAL and others treat the values as
>>>> floats, not integers.
>>>>
>>>> Robert
>>>>
>>>> On Thu, Apr 10, 2014 at 3:36 PM, Jonathan Greenberg <jgrn at illinois.edu> wrote:
>>>>> Zack:
>>>>>
>>>>> Hopefully Robert will get back to you (I confirmed it -- it looks like
>>>>> a bug to me) -- but in the meantime, you might want to try my
>>>>> gdalUtils package, which uses gdal_translate to convert files. You
>>>>> can use one of the many ASCII formats -- you'll want to use parameters
>>>>> like:
>>>>> gdal_translate(...,a_nodata=-9999,ot="UInt16",format="...")
>>>>>
>>>>> You can see the available formats at: http://www.gdal.org/formats_list.html
>>>>>
>>>>> --j
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> On Thu, Apr 10, 2014 at 5:29 PM, Zack Holden <zaholdenfs at gmail.com> wrote:
>>>>>> Thanks for getting back to me Jonathan.
>>>>>>
>>>>>> In reality, I am creating the files from an existing DEM. However, I
>>>>>> get the same behavior regardless of whether I generate the file in R
>>>>>> (as with my example code) or with actual data.
>>>>>> I have since tested on 4 machines. One windows machine with an older
>>>>>> version of R successfully generates the desired ascii file without the
>>>>>> decimals. The other 3 computers (Linux and windows machines) add the
>>>>>> decimals. As a workaround, I've tried using -Inf as my NAflag. This
>>>>>> eliminates the decimals, but results in a large string of white space
>>>>>> in front of the first row of data. These still cause problems for
>>>>>> PCraster (the next step in the processing chain).
>>>>>>
>>>>>> I'm still stumped.
>>>>>>
>>>>>> Thanks again,
>>>>>> Zack
>>>>>>
>>>>>>
>>>>>> On Thu, Apr 10, 2014 at 4:22 PM, Jonathan Greenberg <jgrn at illinois.edu> wrote:
>>>>>>> Zack:
>>>>>>>
>>>>>>> Are you creating the files from within R, or are you just trying to
>>>>>>> convert an existing file? If the latter, can you make one of those
>>>>>>> files available (e.g. via google drive or dropbox or something like
>>>>>>> that)?
>>>>>>>
>>>>>>> --j
>>>>>>>
>>>>>>> On Thu, Apr 10, 2014 at 3:06 PM, Zack Holden <zaholdenfs at gmail.com> wrote:
>>>>>>>> Dear list,
>>>>>>>>
>>>>>>>> I am creating ascii files as inputs to the PCraster software (no other
>>>>>>>> gdal format is allowed). When I write my ascii file using
>>>>>>>> raster::writeRaster, the first cell of every raster that I write has a
>>>>>>>> large number of decimal values appended to it. This causes errors
>>>>>>>> later in the model workflow.
>>>>>>>>
>>>>>>>> This also happens to the first NA value in the file, so that all NA
>>>>>>>> values in the file are e.g. -9999 except the first cell. which appears
>>>>>>>> as: -9999.00000000000
>>>>>>>>
>>>>>>>> Can anyone suggest a method for ensuring that all of my NA values are
>>>>>>>> identical and with no trailing decimals?
>>>>>>>>
>>>>>>>> A small reproducible example follows. Any help or suggestions is appreciated.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Zack
>>>>>>>>
>>>>>>>> ##################
>>>>>>>> # start code
>>>>>>>> require(raster)
>>>>>>>> r <- raster(nrows=2, ncols=2, xmn=0, xmx=2, ymn=0, ymx=2)
>>>>>>>> r <- setValues(r, seq(1:ncell(r)))
>>>>>>>> rexpand <- extend(r, c(2,2))
>>>>>>>> NAvalue(rexpand) <- -9999
>>>>>>>>
>>>>>>>> temp.dir <- "/home/zholden/"
>>>>>>>> writeRaster(rexpand, file=paste(temp.dir, "testraster.asc",sep=""),
>>>>>>>> NAflag=-9999, dataType="INT2U", overwrite=T)
>>>>>>>> # end code
>>>>>>>> ##################
>>>>>>>>
>>>>>>>> # opening in a text editor reveals trailing decimals in the first NA value.
>>>>>>>> system(paste("gedit ", temp.dir, "testraster.asc", sep=""))
>>>>>>>>
>>>>>>>> ################################
>>>>>>>> sessionInfo()
>>>>>>>> R version 3.0.2 (2013-09-25)
>>>>>>>> Platform: x86_64-redhat-linux-gnu (64-bit)
>>>>>>>>
>>>>>>>> locale:
>>>>>>>> [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
>>>>>>>> LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
>>>>>>>> LC_MONETARY=en_US.UTF-8
>>>>>>>> [6] LC_MESSAGES=en_US.UTF-8 LC_PAPER=en_US.UTF-8 LC_NAME=C
>>>>>>>> LC_ADDRESS=C LC_TELEPHONE=C
>>>>>>>> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
>>>>>>>>
>>>>>>>> attached base packages:
>>>>>>>> [1] stats graphics grDevices utils datasets methods base
>>>>>>>>
>>>>>>>> other attached packages:
>>>>>>>> [1] rgdal_0.8-16 raster_2.2-12 sp_1.0-14
>>>>>>>>
>>>>>>>> loaded via a namespace (and not attached):
>>>>>>>> [1] grid_3.0.2 lattice_0.20-23 tools_3.0.2
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> R-sig-Geo mailing list
>>>>>>>> R-sig-Geo at r-project.org
>>>>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Jonathan A. Greenberg, PhD
>>>>>>> Assistant Professor
>>>>>>> Global Environmental Analysis and Remote Sensing (GEARS) Laboratory
>>>>>>> Department of Geography and Geographic Information Science
>>>>>>> University of Illinois at Urbana-Champaign
>>>>>>> 259 Computing Applications Building, MC-150
>>>>>>> 605 East Springfield Avenue
>>>>>>> Champaign, IL 61820-6371
>>>>>>> Phone: 217-300-1924
>>>>>>> http://www.geog.illinois.edu/~jgrn/
>>>>>>> AIM: jgrn307, MSN: jgrn307 at hotmail.com, Gchat: jgrn307, Skype: jgrn3007
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Jonathan A. Greenberg, PhD
>>>>> Assistant Professor
>>>>> Global Environmental Analysis and Remote Sensing (GEARS) Laboratory
>>>>> Department of Geography and Geographic Information Science
>>>>> University of Illinois at Urbana-Champaign
>>>>> 259 Computing Applications Building, MC-150
>>>>> 605 East Springfield Avenue
>>>>> Champaign, IL 61820-6371
>>>>> Phone: 217-300-1924
>>>>> http://www.geog.illinois.edu/~jgrn/
>>>>> AIM: jgrn307, MSN: jgrn307 at hotmail.com, Gchat: jgrn307, Skype: jgrn3007
>>>>>
>>>>> _______________________________________________
>>>>> R-sig-Geo mailing list
>>>>> R-sig-Geo at r-project.org
>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>>>
>>>> _______________________________________________
>>>> R-sig-Geo mailing list
>>>> R-sig-Geo at r-project.org
>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>>
>>>
>>>
>>> --
>>> Jonathan A. Greenberg, PhD
>>> Assistant Professor
>>> Global Environmental Analysis and Remote Sensing (GEARS) Laboratory
>>> Department of Geography and Geographic Information Science
>>> University of Illinois at Urbana-Champaign
>>> 259 Computing Applications Building, MC-150
>>> 605 East Springfield Avenue
>>> Champaign, IL 61820-6371
>>> Phone: 217-300-1924
>>> http://www.geog.illinois.edu/~jgrn/
>>> AIM: jgrn307, MSN: jgrn307 at hotmail.com, Gchat: jgrn307, Skype: jgrn3007
>>
>> _______________________________________________
>> R-sig-Geo mailing list
>> R-sig-Geo at r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>
>
> --
> Edzer Pebesma
> Institute for Geoinformatics (ifgi), University of Münster
> Heisenbergstraße 2, 48149 Münster, Germany. Phone: +49 251
> 83 33081 http://ifgi.uni-muenster.de GPG key ID 0xAC227795
>
>
> _______________________________________________
> R-sig-Geo mailing list
> R-sig-Geo at r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>
More information about the R-sig-Geo
mailing list