So you think this is not an R problem then?
The executable failing the assertion is rsession.exe


2014-06-11 18:14 GMT-03:00 Roger Bivand <Roger.Bivand@nhh.no>:

> I suggest that you put together a complete reproducible example with
> access to the file on a server, and send this to the maintainers of ncdf
> and RNetCDF. This looks either like an issue in compiler flags, or an issue
> in the code of the external library used.
>
> Roger
>
>
> On Wed, 11 Jun 2014, Pablo Alfaro wrote:
>
>  Hmm, just tried using RNetCDF (v 1.6.2-2) and it has the same problem.
>>
>> When using
>>
>> require('RNetCDF')
>> nc <- open.nc('C:/testsMCH/test/wrfout_d02_2014-06-09_00%3A00%3A00')
>> var.get.nc(nc, "T2", start=c(1, 1, 1), count=c(NA, NA, NA))
>>
>> I get the same problem on 64bit R, no problem on 32bits.
>>
>> Seems like a problem with the version of posixio.c used in 64 bit R. Just
>> found a version of posixio.c on the web (
>> http://www.nco.ncep.noaa.gov/pmb/codes/nwprod/util/sorc/
>> ncdump.fd/libsrc/posixio.c)
>> which has the assertion on line 417.
>>
>> assert(offset >= 0);
>>
>> offset is a variable of type off_t which is defined in <sys/types.h> and
>> apparently requires that you enable Large File Support on the compiler
>> command line to be a 64 bit integer.
>> Can it be that the 32 bit version of R is using Large File Support but the
>> 64 bit version is not?
>>
>> Best
>> Pablo
>>
>>
>>
>>
>> 2014-06-11 15:07 GMT-03:00 Pablo Alfaro <palfaro@motionsoft.com.uy>:
>>
>>  Hi Michael, Pascal, thank you both for your replies.
>>>
>>> Yeah, Sorry, I'm using R 3.0.3 64 (or 32 for the working version) bit, on
>>> Windows 7 x64 and the NetCDF library is ncdf (not ncdf4) version 1.6.6,
>>> the
>>> last available version.
>>>
>>> I need it to run on Windows since the servers I run the code on are
>>> Windows.
>>>
>>> I tried downloading and installing the package you mention Michael but
>>> when loading it, it says the x64 architecture is not supported (the
>>> author
>>> mentions it in the site as well). The thing is that ncdf works on x86 so
>>> it
>>> leaves me with the same options.
>>>
>>> RNetCDF seems to use NetCDF v3, but maybe it doesn't have this problem. I
>>> wil try that and see if it helps.
>>>
>>> The assertion on the error complains an offset which isn't >= 0. If I
>>> were
>>> to take a guess, that offset is a 32 bit signed integer and that's why it
>>> breaks only with large files (bigger than 2^31 - 1 bytes). But I can't
>>> tell
>>> for sure. Also it's weird that it works on 32 bits.
>>>
>>> Best
>>> Pablo
>>>
>>>
>>> 2014-06-10 23:13 GMT-03:00 Michael Sumner <mdsumner@gmail.com>:
>>>
>>>
>>>
>>>>
>>>> On Wed, Jun 11, 2014 at 11:49 AM, Pascal Oettli <kridox@ymail.com>
>>>> wrote:
>>>>
>>>>  Hello,
>>>>>
>>>>> Do you get the same error if you run your code out of Rstudio?
>>>>>
>>>>> However, another approach would be to open your NetCDF file with
>>>>> 'brick' function from the raster package.
>>>>>
>>>>>
>>>>
>>>> I don't think either of these will help, raster simply runs ncdf (or
>>>> preferably ncdf4) to do the file read. You might find it easier to move
>>>> to
>>>> ncdf4 if you can, but potentially the stuff I write below is just an
>>>> unwelcome aside. I have seen a similar error in a file I've tried to
>>>> use,
>>>> so I'll have another look at that and see if I can isolate where the
>>>> problem lies. At minimum we really need to know the version of the
>>>> NetCDF
>>>> library in use and perhaps details of how it was compiled so at minimum
>>>> you
>>>> should provide the ncdf package version as well as R's.
>>>>
>>>> It's possible to install ncdf4 on Windows, but it's not supported (no
>>>> binary .zip on CRAN) but you can do it yourself (again not supported,
>>>> and
>>>> the archive here is not updated):
>>>> http://cirrus.ucsd.edu/~pierce/ncdf/
>>>>
>>>> It's much more generally available to do this on Linux, though there's
>>>> always bumps in the road.
>>>>
>>>> RNetCDF is actively updated on CRAN and is another option to see if the
>>>> read is supported (I don't know how to find out what version of the
>>>> library
>>>> was used for the CRAN Windows binary, but presambly it is < 4.0).
>>>>
>>>> Cheers, Mike.
>>>>
>>>>
>>>>
>>>>  Regards,
>>>>> Pascal
>>>>>
>>>>> On Wed, Jun 11, 2014 at 9:10 AM, Pablo Alfaro <
>>>>> palfaro@motionsoft.com.uy>
>>>>> wrote:
>>>>>
>>>>>> Hi, I'm having trouble opening a large netcdf file (2.45 GB) with the
>>>>>> following code on R 3.0.3 64 bit
>>>>>>
>>>>>> require('ncdf')
>>>>>> nc <- open.ncdf('C:/testsMCH/test/wrfout_d02_2014-06-09_00%3A00%
>>>>>> 3A00')
>>>>>> get.var.ncdf(nc, 'T2', start=c(1, 1, 1), count=c(-1, -1, -1))
>>>>>>
>>>>>> I get this error:
>>>>>> Assertion failed!
>>>>>>
>>>>>> Program: C:\Program Files\RStudio\bin\x64\rsession.exe
>>>>>> File: posixio.c, Line 417
>>>>>>
>>>>>> Expression: offset >= 0
>>>>>>
>>>>>> This application has requested the Runtime to terminate it in an
>>>>>>
>>>>> unusual
>>>>>
>>>>>> way.
>>>>>> Please contact the application's support team for more information.
>>>>>>
>>>>>>
>>>>>> In R 32 bit there are no problems.
>>>>>>
>>>>>> It is the same problem as reported here:
>>>>>>
>>>>>>  http://r.789695.n4.nabble.com/Problems-with-Raster-and-clim-
>>>>> pact-packages-with-large-netcdf-files-2-7G-in-x64-bit-
>>>>> R-td4119638.html#a4691956
>>>>>
>>>>>>
>>>>>> Does anyone have any clue as to why this may be?
>>>>>> I can upload the file on an FTP server but it'll take me a while.
>>>>>>
>>>>>> Thanks in advance!
>>>>>> Pablo
>>>>>>
>>>>>> --
>>>>>> Ing. Pablo Alfaro Pi??eyro
>>>>>>
>>>>>> MotionSoft Consulting S.R.L.
>>>>>> Tel: 095 938 487
>>>>>>
>>>>>>         [[alternative HTML version deleted]]
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> R-sig-Geo mailing list
>>>>>> R-sig-Geo@r-project.org
>>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Pascal Oettli
>>>>> Project Scientist
>>>>> JAMSTEC
>>>>> Yokohama, Japan
>>>>>
>>>>> _______________________________________________
>>>>> R-sig-Geo mailing list
>>>>> R-sig-Geo@r-project.org
>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Michael Sumner
>>>> Software and Database Engineer
>>>> Australian Antarctic Division
>>>> Hobart, Australia
>>>> e-mail: mdsumner@gmail.com
>>>>
>>>>
>>>
>>>
>>> --
>>> Ing. Pablo Alfaro Pi??eyro
>>>
>>> MotionSoft Consulting S.R.L.
>>> Tel: 095 938 487
>>>
>>>
>>
>>
>>
>>
> --
> Roger Bivand
> Department of Economics, Norwegian School of Economics,
> Helleveien 30, N-5045 Bergen, Norway.
> voice: +47 55 95 93 55; fax +47 55 95 91 00
> e-mail: Roger.Bivand@nhh.no
>
>


-- 
Ing. Pablo Alfaro Piñeyro
MotionSoft Consulting S.R.L.
Tel: 095 938 487


2014-06-11 18:14 GMT-03:00 Roger Bivand <Roger.Bivand@nhh.no>:

> I suggest that you put together a complete reproducible example with
> access to the file on a server, and send this to the maintainers of ncdf
> and RNetCDF. This looks either like an issue in compiler flags, or an issue
> in the code of the external library used.
>
> Roger
>
>
> On Wed, 11 Jun 2014, Pablo Alfaro wrote:
>
>  Hmm, just tried using RNetCDF (v 1.6.2-2) and it has the same problem.
>>
>> When using
>>
>> require('RNetCDF')
>> nc <- open.nc('C:/testsMCH/test/wrfout_d02_2014-06-09_00%3A00%3A00')
>> var.get.nc(nc, "T2", start=c(1, 1, 1), count=c(NA, NA, NA))
>>
>> I get the same problem on 64bit R, no problem on 32bits.
>>
>> Seems like a problem with the version of posixio.c used in 64 bit R. Just
>> found a version of posixio.c on the web (
>> http://www.nco.ncep.noaa.gov/pmb/codes/nwprod/util/sorc/
>> ncdump.fd/libsrc/posixio.c)
>> which has the assertion on line 417.
>>
>> assert(offset >= 0);
>>
>> offset is a variable of type off_t which is defined in <sys/types.h> and
>> apparently requires that you enable Large File Support on the compiler
>> command line to be a 64 bit integer.
>> Can it be that the 32 bit version of R is using Large File Support but the
>> 64 bit version is not?
>>
>> Best
>> Pablo
>>
>>
>>
>>
>> 2014-06-11 15:07 GMT-03:00 Pablo Alfaro <palfaro@motionsoft.com.uy>:
>>
>>  Hi Michael, Pascal, thank you both for your replies.
>>>
>>> Yeah, Sorry, I'm using R 3.0.3 64 (or 32 for the working version) bit, on
>>> Windows 7 x64 and the NetCDF library is ncdf (not ncdf4) version 1.6.6,
>>> the
>>> last available version.
>>>
>>> I need it to run on Windows since the servers I run the code on are
>>> Windows.
>>>
>>> I tried downloading and installing the package you mention Michael but
>>> when loading it, it says the x64 architecture is not supported (the
>>> author
>>> mentions it in the site as well). The thing is that ncdf works on x86 so
>>> it
>>> leaves me with the same options.
>>>
>>> RNetCDF seems to use NetCDF v3, but maybe it doesn't have this problem. I
>>> wil try that and see if it helps.
>>>
>>> The assertion on the error complains an offset which isn't >= 0. If I
>>> were
>>> to take a guess, that offset is a 32 bit signed integer and that's why it
>>> breaks only with large files (bigger than 2^31 - 1 bytes). But I can't
>>> tell
>>> for sure. Also it's weird that it works on 32 bits.
>>>
>>> Best
>>> Pablo
>>>
>>>
>>> 2014-06-10 23:13 GMT-03:00 Michael Sumner <mdsumner@gmail.com>:
>>>
>>>
>>>
>>>>
>>>> On Wed, Jun 11, 2014 at 11:49 AM, Pascal Oettli <kridox@ymail.com>
>>>> wrote:
>>>>
>>>>  Hello,
>>>>>
>>>>> Do you get the same error if you run your code out of Rstudio?
>>>>>
>>>>> However, another approach would be to open your NetCDF file with
>>>>> 'brick' function from the raster package.
>>>>>
>>>>>
>>>>
>>>> I don't think either of these will help, raster simply runs ncdf (or
>>>> preferably ncdf4) to do the file read. You might find it easier to move
>>>> to
>>>> ncdf4 if you can, but potentially the stuff I write below is just an
>>>> unwelcome aside. I have seen a similar error in a file I've tried to
>>>> use,
>>>> so I'll have another look at that and see if I can isolate where the
>>>> problem lies. At minimum we really need to know the version of the
>>>> NetCDF
>>>> library in use and perhaps details of how it was compiled so at minimum
>>>> you
>>>> should provide the ncdf package version as well as R's.
>>>>
>>>> It's possible to install ncdf4 on Windows, but it's not supported (no
>>>> binary .zip on CRAN) but you can do it yourself (again not supported,
>>>> and
>>>> the archive here is not updated):
>>>> http://cirrus.ucsd.edu/~pierce/ncdf/
>>>>
>>>> It's much more generally available to do this on Linux, though there's
>>>> always bumps in the road.
>>>>
>>>> RNetCDF is actively updated on CRAN and is another option to see if the
>>>> read is supported (I don't know how to find out what version of the
>>>> library
>>>> was used for the CRAN Windows binary, but presambly it is < 4.0).
>>>>
>>>> Cheers, Mike.
>>>>
>>>>
>>>>
>>>>  Regards,
>>>>> Pascal
>>>>>
>>>>> On Wed, Jun 11, 2014 at 9:10 AM, Pablo Alfaro <
>>>>> palfaro@motionsoft.com.uy>
>>>>> wrote:
>>>>>
>>>>>> Hi, I'm having trouble opening a large netcdf file (2.45 GB) with the
>>>>>> following code on R 3.0.3 64 bit
>>>>>>
>>>>>> require('ncdf')
>>>>>> nc <- open.ncdf('C:/testsMCH/test/wrfout_d02_2014-06-09_00%3A00%
>>>>>> 3A00')
>>>>>> get.var.ncdf(nc, 'T2', start=c(1, 1, 1), count=c(-1, -1, -1))
>>>>>>
>>>>>> I get this error:
>>>>>> Assertion failed!
>>>>>>
>>>>>> Program: C:\Program Files\RStudio\bin\x64\rsession.exe
>>>>>> File: posixio.c, Line 417
>>>>>>
>>>>>> Expression: offset >= 0
>>>>>>
>>>>>> This application has requested the Runtime to terminate it in an
>>>>>>
>>>>> unusual
>>>>>
>>>>>> way.
>>>>>> Please contact the application's support team for more information.
>>>>>>
>>>>>>
>>>>>> In R 32 bit there are no problems.
>>>>>>
>>>>>> It is the same problem as reported here:
>>>>>>
>>>>>>  http://r.789695.n4.nabble.com/Problems-with-Raster-and-clim-
>>>>> pact-packages-with-large-netcdf-files-2-7G-in-x64-bit-
>>>>> R-td4119638.html#a4691956
>>>>>
>>>>>>
>>>>>> Does anyone have any clue as to why this may be?
>>>>>> I can upload the file on an FTP server but it'll take me a while.
>>>>>>
>>>>>> Thanks in advance!
>>>>>> Pablo
>>>>>>
>>>>>> --
>>>>>> Ing. Pablo Alfaro Pi??eyro
>>>>>>
>>>>>> MotionSoft Consulting S.R.L.
>>>>>> Tel: 095 938 487
>>>>>>
>>>>>>         [[alternative HTML version deleted]]
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> R-sig-Geo mailing list
>>>>>> R-sig-Geo@r-project.org
>>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Pascal Oettli
>>>>> Project Scientist
>>>>> JAMSTEC
>>>>> Yokohama, Japan
>>>>>
>>>>> _______________________________________________
>>>>> R-sig-Geo mailing list
>>>>> R-sig-Geo@r-project.org
>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Michael Sumner
>>>> Software and Database Engineer
>>>> Australian Antarctic Division
>>>> Hobart, Australia
>>>> e-mail: mdsumner@gmail.com
>>>>
>>>>
>>>
>>>
>>> --
>>> Ing. Pablo Alfaro Pi??eyro
>>>
>>> MotionSoft Consulting S.R.L.
>>> Tel: 095 938 487
>>>
>>>
>>
>>
>>
>>
> --
> Roger Bivand
> Department of Economics, Norwegian School of Economics,
> Helleveien 30, N-5045 Bergen, Norway.
> voice: +47 55 95 93 55; fax +47 55 95 91 00
> e-mail: Roger.Bivand@nhh.no
>
>


-- 
Ing. Pablo Alfaro Piñeyro
MotionSoft Consulting S.R.L.
Tel: 095 938 487

	[[alternative HTML version deleted]]

