[R-sig-Geo] decimals in NA value using raster::writeRaster with .ascii format

Edzer Pebesma edzer.pebesma at uni-muenster.de
Mon Apr 14 15:58:17 CEST 2014 


On 04/11/2014 05:46 PM, Zack Holden wrote:
> Robert,
> Thank you for your help. I failed to notice the "datatype vs. dataType
> difference. Calling 'datatype' does indeed solve my integer test case.
> However, some of the raster datasets that I am processing are actually
> floating point. When I call writeRaster(datatype = 'FLT4S'), I still
> see the long string of decimals in the first value in my ascii grid.
> 
> Is there any way to handle this in R? From your response, it sounds
> like this might be an artifact of GDAL, and I may need to manually (or
> with text manipulations) edit out the decimal strings.

Look for

https://r-forge.r-project.org/scm/viewvc.php/pkg/raster/R/writeAllAscii.R?view=markup&root=raster

and lower the 15, or else change the formatC call, then rebuild raster.

As described earlier, raster does this on purpose: the gdal asciigrid
reader makes the wrong assumption about the data type when a grid starts
with too many missing values that look like an integer (like -9999 but
unlike -9999.0). This has been reported over a decade ago, and may be
seen as an indication of gdal's liveliness. Not only OpenSSL suffers
from too few eyes to make bugs shallow.

What software are you using that can't digest the 15-digit float?

> 
> Thanks for any additional suggestions.
> 
> Javier, thanks for pointing out the available GDAL drivers in
> PCraster, I'm just getting acquainted with that software program.
> 
> Cheers,
> Zack
> 
> 
> On Fri, Apr 11, 2014 at 8:48 AM, Jonathan Greenberg <jgrn at illinois.edu> wrote:
>> Robert:
>>
>> What is the practical difference between those two? -- does dataType
>> get passed off to GDAL, while datatype is interpreted by your R code?
>>
>> --j
>>
>> On Thu, Apr 10, 2014 at 7:06 PM, Robert J. Hijmans <r.hijmans at gmail.com> wrote:
>>> Zack,
>>>
>>> You are using:     dataType="INT2U"
>>>
>>> but it should be:     datatype="INT2U"  # (or any other INT data type)
>>>
>>> That fixes it. If the datatype is FLT* (the default), the first value
>>> gets decimals to assure that GDAL and others treat the values as
>>> floats, not integers.
>>>
>>> Robert
>>>
>>> On Thu, Apr 10, 2014 at 3:36 PM, Jonathan Greenberg <jgrn at illinois.edu> wrote:
>>>> Zack:
>>>>
>>>> Hopefully Robert will get back to you (I confirmed it -- it looks like
>>>> a bug to me) -- but in the meantime, you might want to try my
>>>> gdalUtils package, which uses gdal_translate to convert files.  You
>>>> can use one of the many ASCII formats -- you'll want to use parameters
>>>> like:
>>>> gdal_translate(...,a_nodata=-9999,ot="UInt16",format="...")
>>>>
>>>> You can see the available formats at: http://www.gdal.org/formats_list.html
>>>>
>>>> --j
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> On Thu, Apr 10, 2014 at 5:29 PM, Zack Holden <zaholdenfs at gmail.com> wrote:
>>>>> Thanks for getting back to me Jonathan.
>>>>>
>>>>> In reality, I am creating the files from an existing DEM. However, I
>>>>> get the same behavior regardless of whether I generate the file in R
>>>>> (as with my example code) or with actual data.
>>>>> I have since tested on 4 machines. One windows machine with an older
>>>>> version of R successfully generates the desired ascii file without the
>>>>> decimals. The other 3 computers (Linux and windows machines) add the
>>>>> decimals. As a workaround, I've tried using -Inf as my NAflag. This
>>>>> eliminates the decimals, but results in a large string of white space
>>>>> in front of the first row of data. These still cause problems for
>>>>> PCraster (the next step in the processing chain).
>>>>>
>>>>> I'm still stumped.
>>>>>
>>>>> Thanks again,
>>>>> Zack
>>>>>
>>>>>
>>>>> On Thu, Apr 10, 2014 at 4:22 PM, Jonathan Greenberg <jgrn at illinois.edu> wrote:
>>>>>> Zack:
>>>>>>
>>>>>> Are you creating the files from within R, or are you just trying to
>>>>>> convert an existing file?  If the latter, can you make one of those
>>>>>> files available (e.g. via google drive or dropbox or something like
>>>>>> that)?
>>>>>>
>>>>>> --j
>>>>>>
>>>>>> On Thu, Apr 10, 2014 at 3:06 PM, Zack Holden <zaholdenfs at gmail.com> wrote:
>>>>>>> Dear list,
>>>>>>>
>>>>>>> I am creating ascii files as inputs to the PCraster software (no other
>>>>>>> gdal format is allowed). When I write my ascii file using
>>>>>>> raster::writeRaster, the first cell of every raster that I write has a
>>>>>>> large number of decimal values appended to it. This causes errors
>>>>>>> later in the model workflow.
>>>>>>>
>>>>>>> This also happens to the first NA value in the file, so that all NA
>>>>>>> values in the file are e.g. -9999 except the first cell. which appears
>>>>>>> as: -9999.00000000000
>>>>>>>
>>>>>>> Can anyone suggest a method for ensuring that all of my NA values are
>>>>>>> identical and with no trailing decimals?
>>>>>>>
>>>>>>> A small reproducible example follows. Any help or suggestions is appreciated.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Zack
>>>>>>>
>>>>>>> ##################
>>>>>>> # start code
>>>>>>> require(raster)
>>>>>>> r <- raster(nrows=2, ncols=2, xmn=0, xmx=2, ymn=0, ymx=2)
>>>>>>> r <- setValues(r, seq(1:ncell(r)))
>>>>>>> rexpand <- extend(r, c(2,2))
>>>>>>> NAvalue(rexpand) <- -9999
>>>>>>>
>>>>>>> temp.dir <- "/home/zholden/"
>>>>>>> writeRaster(rexpand, file=paste(temp.dir, "testraster.asc",sep=""),
>>>>>>> NAflag=-9999, dataType="INT2U", overwrite=T)
>>>>>>> # end code
>>>>>>> ##################
>>>>>>>
>>>>>>> # opening in a text editor reveals trailing decimals in the first NA value.
>>>>>>> system(paste("gedit ", temp.dir, "testraster.asc", sep=""))
>>>>>>>
>>>>>>> ################################
>>>>>>>  sessionInfo()
>>>>>>> R version 3.0.2 (2013-09-25)
>>>>>>> Platform: x86_64-redhat-linux-gnu (64-bit)
>>>>>>>
>>>>>>> locale:
>>>>>>>  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
>>>>>>> LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8
>>>>>>> LC_MONETARY=en_US.UTF-8
>>>>>>>  [6] LC_MESSAGES=en_US.UTF-8    LC_PAPER=en_US.UTF-8       LC_NAME=C
>>>>>>>                LC_ADDRESS=C               LC_TELEPHONE=C
>>>>>>> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
>>>>>>>
>>>>>>> attached base packages:
>>>>>>> [1] stats     graphics  grDevices utils     datasets  methods   base
>>>>>>>
>>>>>>> other attached packages:
>>>>>>> [1] rgdal_0.8-16  raster_2.2-12 sp_1.0-14
>>>>>>>
>>>>>>> loaded via a namespace (and not attached):
>>>>>>> [1] grid_3.0.2      lattice_0.20-23 tools_3.0.2
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> R-sig-Geo mailing list
>>>>>>> R-sig-Geo at r-project.org
>>>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Jonathan A. Greenberg, PhD
>>>>>> Assistant Professor
>>>>>> Global Environmental Analysis and Remote Sensing (GEARS) Laboratory
>>>>>> Department of Geography and Geographic Information Science
>>>>>> University of Illinois at Urbana-Champaign
>>>>>> 259 Computing Applications Building, MC-150
>>>>>> 605 East Springfield Avenue
>>>>>> Champaign, IL  61820-6371
>>>>>> Phone: 217-300-1924
>>>>>> http://www.geog.illinois.edu/~jgrn/
>>>>>> AIM: jgrn307, MSN: jgrn307 at hotmail.com, Gchat: jgrn307, Skype: jgrn3007
>>>>
>>>>
>>>>
>>>> --
>>>> Jonathan A. Greenberg, PhD
>>>> Assistant Professor
>>>> Global Environmental Analysis and Remote Sensing (GEARS) Laboratory
>>>> Department of Geography and Geographic Information Science
>>>> University of Illinois at Urbana-Champaign
>>>> 259 Computing Applications Building, MC-150
>>>> 605 East Springfield Avenue
>>>> Champaign, IL  61820-6371
>>>> Phone: 217-300-1924
>>>> http://www.geog.illinois.edu/~jgrn/
>>>> AIM: jgrn307, MSN: jgrn307 at hotmail.com, Gchat: jgrn307, Skype: jgrn3007
>>>>
>>>> _______________________________________________
>>>> R-sig-Geo mailing list
>>>> R-sig-Geo at r-project.org
>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>>
>>> _______________________________________________
>>> R-sig-Geo mailing list
>>> R-sig-Geo at r-project.org
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>
>>
>>
>> --
>> Jonathan A. Greenberg, PhD
>> Assistant Professor
>> Global Environmental Analysis and Remote Sensing (GEARS) Laboratory
>> Department of Geography and Geographic Information Science
>> University of Illinois at Urbana-Champaign
>> 259 Computing Applications Building, MC-150
>> 605 East Springfield Avenue
>> Champaign, IL  61820-6371
>> Phone: 217-300-1924
>> http://www.geog.illinois.edu/~jgrn/
>> AIM: jgrn307, MSN: jgrn307 at hotmail.com, Gchat: jgrn307, Skype: jgrn3007
> 
> _______________________________________________
> R-sig-Geo mailing list
> R-sig-Geo at r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
> 

-- 
Edzer Pebesma
Institute for Geoinformatics (ifgi), University of Münster
Heisenbergstraße 2, 48149 Münster, Germany. Phone: +49 251
83 33081 http://ifgi.uni-muenster.de GPG key ID 0xAC227795

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 490 bytes
Desc: OpenPGP digital signature
URL: <https://stat.ethz.ch/pipermail/r-sig-geo/attachments/20140414/603c28f5/attachment.bin>

More information about the R-sig-Geo mailing list