[R-sig-Geo] rPython installation problem

Thu Feb 27 21:42:03 CET 2014

Javier:

I guess my first suggestion is to try out my rasterEngine function in
the spatial.tools package -- it is designed to perform parallel
processing of raster images entirely within R.  I have a tutorial
here:

http://www.geog.illinois.edu/~jgrn/software-and-datasets/rasterengine-tutorial/

You might want to look at the more complex randomForest example.  I
have a bunch of updates to rasterEngine that haven't made it into the
tutorial yet, that might also be useful, including having > 1 output
file.  The help for rasterEngine has some of these updates.

Let me know how it goes!

--j

On Thu, Feb 27, 2014 at 11:55 AM, Javier Martínez López
<javier.martinez at um.es> wrote:
> Dear Jonathan,
>
> That is a nice question! I need to repetitively apply the Mahalanobis
> distance to a stack of eleven raster maps comprising very large areas in
> order to compare the similarity of different areas (see the eHabitat R
> library [1] which is used for the DOPA explorer [2]). I am just trying to
> speed up the processing by hosting the maps in GRASS GIS, using the Pygrass
> module for importing them as numpy arrays, using a parallel processing
> script for calculating the mahalanobis distance which also benefits from MKL
> enhancements, and eventually continue some processing in R, if I find a fast
> way, in combination with the existent eHabitat library, or directly
> producing the resulting tiff and csv files... Does it make sense to you at
> all? I plan to upload a first working version of the code to github [3] this
> or next week, as soon as I solve some errors.
>
> Thank you for your interest!
>
> Cheers,
>
> Javier
>
>
> [1](https://github.com/javimarlop/eHabitat)
> [2](http://ehabitat-wps.jrc.ec.europa.eu/dopa_explorer/)
> [3](https://github.com/javimarlop/eHab_grasspy)
>
>
> Quoting Jonathan Greenberg <jgrn at illinois.edu>:
>
>> Javier:
>>
>> If you don't mind me asking -- when you said "I need to combine R and
>> Python to process large raster files" is there a particular function
>> in python you need to apply to the rasters, or are you specifically
>> referring to some benefit in python of working with arbitrarily large
>> raster files?
>>
>> --j
>>
>> On Thu, Feb 27, 2014 at 4:24 AM, Javier Martínez López
>> <javier.martinez at um.es> wrote:
>>>
>>> Dear Roger,
>>>
>>> your help is very much appreciated! When I try to locate libpython2.7 is
>>> does not come up, but the file is present in my $HOME/anaconda/lib.
>>> Systemwide only python 2.6 is present due to a conflict with 2.7 in
>>> CentOS.
>>>
>>> I tried something like:
>>>
>>> R CMD INSTALL rPython_0.0-4.tar.gz
>>> --configure-args='LD_LIBRARY_PATH=/home/majavie/anaconda/lib/'
>>>
>>> and
>>>
>>> R CMD INSTALL rPython_0.0-4.tar.gz
>>>
>>> --configure-args='PYTHONLIBFILE=/home/majavie/anaconda/lib/libpython2.7.so'
>>>
>>> but still getting the same error, and when try:
>>>
>>> export LD_LIBRARY_PATH=/home/majavie/anaconda/lib/:$PATH
>>>
>>> then I cannot even start R!... so, I might try to install a local version
>>> of
>>> R from source, just to try if this one works!
>>>
>>> Thank you again and cheers,
>>>
>>> Javier
>>>
>>>
>>>
>>>
>>>
>>> Quoting Roger Bivand <Roger.Bivand at nhh.no>:
>>>
>>>> On Wed, 26 Feb 2014, Javier Martínez López wrote:
>>>>
>>>>> Hello all,
>>>>>
>>>>> I need to combine R and Python to process large raster files and I
>>>>> would
>>>>> like to use the "rPython" library for direct transformation of the data
>>>>> objects created using those environments. However, when I try to
>>>>> install it,
>>>>> it produces the following error related to the python libraries (see
>>>>> below).
>>>>> I just installed the Anaconda python distribution locally (no admin
>>>>> rights)
>>>>> in my home folder (Linux CentOS), which might be causing the problem,
>>>>> but I
>>>>> do not know how to deal with this issue, so any help is welcome!
>>>>
>>>>
>>>>
>>>> This is not a specific question for this list, but there may be interest
>>>> in this fresh and as yet unproven package. What does:
>>>>
>>>> locate libpython2.7
>>>>
>>>> say on your system? On current Fedora, I see:
>>>>
>>>> /usr/lib64/libpython2.7.so
>>>>
>>>> with:
>>>>
>>>> $ python --version
>>>> Python 2.7.5
>>>>
>>>> as per the advice in:
>>>>
>>>> http://cran.r-project.org/web/packages/rPython/INSTALL
>>>>
>>>> Your system should provide python, but may have only the python, not
>>>> both
>>>> that and the python-devel RPMs installed. If you have installed
>>>> libpython2.7.so locally, you may need to adjust your LD_LIBRARY_PATH so
>>>> that
>>>> the library is visible. If you can get your sysadmin to add the
>>>> python-devel
>>>> RPM, life may be easier. Also, write to the package maintainer
>>>> suggesting
>>>> that libpython be checked in configure.
>>>>
>>>> Hope this helps,
>>>>
>>>> Roger
>>>>
>>>>>
>>>>> Thank you very much and cheers,
>>>>>
>>>>> Javier
>>>>>
>>>>>> install.packages('rPython')
>>>>>
>>>>>
>>>>> Installing package into
>>>>> /home/majavie/R/x86_64-redhat-linux-gnu-library/3.0
>>>>> (as lib is unspecified)
>>>>> trying URL
>>>>>
>>>>> 'http://cran.mirror.garr.it/mirrors/CRAN/src/contrib/rPython_0.0-4.tar.gz'
>>>>> Content type 'text/plain' length 37034 bytes (36 Kb)
>>>>> opened URL
>>>>> ==================================================
>>>>> downloaded 36 Kb
>>>>>
>>>>> * installing *source* package rPython ...
>>>>> ** package rPython successfully unpacked and MD5 sums checked
>>>>> a specific python version to use was not provided
>>>>> defaulting to the standard python in the system
>>>>> checking for gcc... gcc -m64 -std=gnu99
>>>>> checking whether the C compiler works... yes
>>>>> checking for C compiler default output file name... a.out
>>>>> checking for suffix of executables...
>>>>> checking whether we are cross compiling... no
>>>>> checking for suffix of object files... o
>>>>> checking whether we are using the GNU C compiler... yes
>>>>> checking whether gcc -m64 -std=gnu99 accepts -g... yes
>>>>> checking for gcc -m64 -std=gnu99 option to accept ISO C89... none
>>>>> needed
>>>>> configure: creating ./config.status
>>>>> config.status: creating src/Makevars
>>>>> ** libs
>>>>> gcc -m64 -std=gnu99 -I/usr/include/R -DNDEBUG  -I/usr/local/include
>>>>> -I/home/majavie/anaconda/include/python2.7
>>>>> -I/home/majavie/anaconda/include/python2.7 -D
>>>>> PYTHONLIBFILE=libpython2.7.so -fpic  -O2 -g -pipe -Wall
>>>>> -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector
>>>>> --param=ssp-buffer-size=4 -m64 -mtune=generic  -c pycall.c -o pycall.o
>>>>> gcc -m64 -std=gnu99 -shared -L/usr/local/lib64 -o rPython.so pycall.o
>>>>> -lpthread -ldl -lutil -lm -lpython2.7 -Xlinker -export-dynamic
>>>>> -L/usr/lib64/R/lib -lR
>>>>> /usr/bin/ld: cannot find -lpython2.7
>>>>> collect2: ld returned 1 exit status
>>>>> make: *** [rPython.so] Error 1
>>>>> ERROR: compilation failed for package rPython
>>>>> * removing /home/majavie/R/x86_64-redhat-linux-gnu-library/3.0/rPython
>>>>> * restoring previous
>>>>> /home/majavie/R/x86_64-redhat-linux-gnu-library/3.0/rPython
>>>>>
>>>>> The downloaded source packages are in
>>>>>  /tmp/RtmpfPBtPF/downloaded_packages
>>>>> Warning message:
>>>>> In install.packages("rPython") :
>>>>> installation of package rPython had non-zero exit status
>>>>>
>>>>> _______________________________________________
>>>>> R-sig-Geo mailing list
>>>>> R-sig-Geo at r-project.org
>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>>>
>>>>
>>>>
>>>> --
>>>> Roger Bivand
>>>> Department of Economics, Norwegian School of Economics,
>>>> Helleveien 30, N-5045 Bergen, Norway.
>>>> voice: +47 55 95 93 55; fax +47 55 95 95 43
>>>> e-mail: Roger.Bivand at nhh.no
>>>>
>>>
>>>
>>>
>>> --
>>> Javier Martínez-López
>>> Ecology and Hydrology Department
>>> Biology Faculty
>>> University of Murcia
>>> Espinardo E-30100
>>> Murcia - Spain
>>>
>>> E-mail: javier.martinez at um.es
>>> web: http://webs.um.es/javier.martinez
>>>
>>>
>>> _______________________________________________
>>> R-sig-Geo mailing list
>>> R-sig-Geo at r-project.org
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>
>>
>>
>>
>> --
>> Jonathan A. Greenberg, PhD
>> Assistant Professor
>> Global Environmental Analysis and Remote Sensing (GEARS) Laboratory
>> Department of Geography and Geographic Information Science
>> University of Illinois at Urbana-Champaign
>> 259 Computing Applications Building, MC-150
>> 605 East Springfield Avenue
>> Champaign, IL  61820-6371
>> Phone: 217-300-1924
>> http://www.geog.illinois.edu/~jgrn/
>> AIM: jgrn307, MSN: jgrn307 at hotmail.com, Gchat: jgrn307, Skype: jgrn3007
>>
>
>
>
> --
> Javier Martínez-López
> Ecology and Hydrology Department
> Biology Faculty
> University of Murcia
> Espinardo E-30100
> Murcia - Spain
>
> E-mail: javier.martinez at um.es
> web: http://webs.um.es/javier.martinez
>

-- 
Jonathan A. Greenberg, PhD
Assistant Professor
Global Environmental Analysis and Remote Sensing (GEARS) Laboratory
Department of Geography and Geographic Information Science
University of Illinois at Urbana-Champaign
259 Computing Applications Building, MC-150
605 East Springfield Avenue
Champaign, IL  61820-6371
Phone: 217-300-1924
http://www.geog.illinois.edu/~jgrn/
AIM: jgrn307, MSN: jgrn307 at hotmail.com, Gchat: jgrn307, Skype: jgrn3007