[Bioc-devel] xps build problem on veracruz2
Hervé Pagès
hpages at fredhutch.org
Wed Apr 19 22:51:53 CEST 2017
Hi Christian,
So I installed ROOT 5 from source on veracruz2. It's in
/usr/local/root.
However, Apple's SIP (System Integrity Protection, new and
enabled by default on El Capitan) is getting in the way when
trying to install xps. That's because xps configure and build
process relies on DYLD_LIBRARY_PATH. Problem is that this
environment variable (and any other variables that control
dynamic loading) is not inherited by child processes when SIP
is on:
veracruz2:~ biocbuild$ if test "${DYLD_LIBRARY_PATH}"; then echo 'yep!';
else echo 'nope!'; fi
yep!
veracruz2:~ biocbuild$ sh
sh-3.2$ if test "${DYLD_LIBRARY_PATH}"; then echo 'yep!'; else echo
'nope!'; fi
nope!
That breaks xps configure script:
veracruz2:~ biocbuild$ export LD_LIBRARY_PATH=$DYLD_LIBRARY_PATH
veracruz2:~ biocbuild$ echo $LD_LIBRARY_PATH
/usr/local/mysql/lib:/usr/local/root/lib/root:/ImageMagick-7.0.5/lib:/usr/local/ensembl-vep/htslib
veracruz2:~ biocbuild$ R CMD INSTALL xps
* installing to library
‘/Library/Frameworks/R.framework/Versions/3.4/Resources/library’
* installing *source* package ‘xps’ ...
checking for gcc... clang
checking for C compiler default output file name... a.out
checking whether the C compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables...
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether clang accepts -g... yes
checking for clang option to accept ANSI C... none needed
checking how to run the C preprocessor... clang -E
checking for gcc... (cached) clang
checking whether we are using the GNU C compiler... (cached) yes
checking whether clang accepts -g... (cached) yes
checking for clang option to accept ANSI C... (cached) none needed
found ROOT version 5.34/36 in directory /usr/local/root
xps configuration error:
You must set the shell variable LD_LIBRARY_PATH to the
directory where ROOT resides and re-run R CMD INSTALL
e.g., (using Bourne shell syntax):
export "LD_LIBRARY_PATH=$ROOTSYS/lib:$LD_LIBRARY_PATH"
R CMD INSTALL xps
Please consult the README file for more information
ERROR: configuration failed for package ‘xps’
* removing
‘/Library/Frameworks/R.framework/Versions/3.4/Resources/library/xps’
* restoring previous
‘/Library/Frameworks/R.framework/Versions/3.4/Resources/library/xps’
That also breaks the dynlib mechanism because, after I managed to
produce xps.so, it turns out that this shared object is linked to
the ROOT libraries via the @rpath mechanism:
veracruz2:src biocbuild$ otool -L xps.so
xps.so:
xps.so (compatibility version 0.0.0, current version 0.0.0)
@rpath/libGui.so (compatibility version 0.0.0, current version 0.0.0)
@rpath/libCore.so (compatibility version 0.0.0, current version 0.0.0)
@rpath/libCint.so (compatibility version 0.0.0, current version 0.0.0)
@rpath/libRIO.so (compatibility version 0.0.0, current version 0.0.0)
@rpath/libNet.so (compatibility version 0.0.0, current version 0.0.0)
@rpath/libHist.so (compatibility version 0.0.0, current version 0.0.0)
@rpath/libGraf.so (compatibility version 0.0.0, current version 0.0.0)
@rpath/libGraf3d.so (compatibility version 0.0.0, current version 0.0.0)
@rpath/libGpad.so (compatibility version 0.0.0, current version 0.0.0)
@rpath/libTree.so (compatibility version 0.0.0, current version 0.0.0)
@rpath/libRint.so (compatibility version 0.0.0, current version 0.0.0)
@rpath/libPostscript.so (compatibility version 0.0.0, current version
0.0.0)
@rpath/libMatrix.so (compatibility version 0.0.0, current version 0.0.0)
@rpath/libPhysics.so (compatibility version 0.0.0, current version 0.0.0)
@rpath/libMathCore.so (compatibility version 0.0.0, current version 0.0.0)
@rpath/libThread.so (compatibility version 0.0.0, current version 0.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
version 1226.10.1)
@rpath/libGed.so (compatibility version 0.0.0, current version 0.0.0)
@rpath/libTreePlayer.so (compatibility version 0.0.0, current version
0.0.0)
@rpath/libTreeViewer.so (compatibility version 0.0.0, current version
0.0.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version
120.1.0)
so it can't be loaded in R:
> dyn.load("xps.so")
Error in dyn.load("xps.so") :
unable to load shared object
'/Users/biocbuild/bbs-3.5-bioc/xps/src/xps.so':
dlopen(/Users/biocbuild/bbs-3.5-bioc/xps/src/xps.so, 6): Library not
loaded: @rpath/libGui.so
Referenced from: /Users/biocbuild/bbs-3.5-bioc/xps/src/xps.so
Reason: image not found
because R has no access to DYLD_LIBRARY_PATH:
> Sys.getenv("DYLD_LIBRARY_PATH")
[1] ""
This can be addressed by adding the following flag when linking:
-rpath $(shell $(ROOTCONFIG) --prefix)/lib
Do you think you can revisit xps configure and build process? Make sure
you test it on a machine where SIP is enabled.
Thanks,
H.
On 03/24/2017 12:14 PM, cstrato wrote:
>
>
> On 03/24/17 19:55, Hervé Pagès wrote:
>> On 03/24/2017 11:37 AM, cstrato wrote:
>>>
>>>
>>> On 03/24/17 19:23, Hervé Pagès wrote:
>>>> On 03/24/2017 11:10 AM, cstrato wrote:
>>>>>
>>>>>
>>>>> On 03/24/17 18:02, Hervé Pagès wrote:
>>>>>> On 03/24/2017 06:52 AM, cstrato wrote:
>>>>>>> R/Bioc is still building on Mavericks,
>>>>>>
>>>>>> Not for R devel (3.4). The R folks have switched to El Capitan a few
>>>>>> days ago:
>>>>>>
>>>>>
>>>>> You are right, I did not check R devel.
>>>>>
>>>>>>
>>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__r.research.att.com_&d=DwIDaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=aV7U6Qu8HkkL9dhD7thXz2c2geZd1KmfWnoZkiyu6hs&s=EDYb8eN2bAg_TtTfDURARDLiz4AoKggk2QLfABIdxTA&e=
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> and before was built on Snow
>>>>>>> Leopard (which many people are sill using).
>>>>>>>
>>>>>>> Personally I think that it does not make much difference whether
>>>>>>> Mavericks or El Capitan (or Yosemite) is used to build R/Bioc.
>>>>>>
>>>>>> How much experience you have with setting a Mavericks or El Capitan
>>>>>> build machine to build and distribute thousands of package binaries
>>>>>> for
>>>>>> hundreds ot thousands of users?
>>>>>>
>>>>>
>>>>> You probably misunderstood what I wanted to say.
>>>>>
>>>>> It is clear to me that you are doing a great job distributing
>>>>> thousands
>>>>> of package binaries. No one does know it better than me with the
>>>>> special
>>>>> problems you have to build binaries for xps. I really appreciate that
>>>>> during all these years you and Dan (and others) managed to support xps
>>>>> like all other BioC packages.
>>>>>
>>>>> I meant that from the user standpoint it probably does not matter much
>>>>> which of these three systems are used to build BioC, in contrast to
>>>>> Sierra.
>>>>
>>>> Of course it matters. If you use an older OS than the one we use to
>>>> produce the binaries then some binaries won't work for you. You keep
>>>> missing the whole point.
>>>>
>>>> H.
>>>>
>>>
>>> Sorry, but I do understand this point. Users who are still using e.g.
>>> Snow Leopard (because they think this was the best system) will have
>>> problems. For that reason I thought that maybe it is best to use the
>>> system which is currently used by most users.
>>
>> That would be the thing to do if we didn't have neither forward- nor
>> backward- compatibility. But we *do* have forward-compatibility. So
>> there is no reason to use the system which is currently used by most
>> users. It's enough to make sure that we use a system that is
>> *compatible* with what most users have. And also not too old because
>> it's hard to find powerful hardware that runs old OS X versions and
>> because many software components needed for the builds are not
>> available or not maintained anymore for old OS X versions.
>>
>> Like Dan said, it's a tradeoff.
>>
>> H.
>>
>
> You are right, using a system which is compatible with what most users
> have, is the best choice.
>
> Christian
>
>>
>>>
>>> Christian
>>>
>>>
>>>>>
>>>>> But as you said below backward-compatibility is always lost, so the
>>>>> question which system to use to build R/BioC is always tricky. Maybe,
>>>>> the best (?) decision would be to use the system which most Mac users
>>>>> are currently using, but I don't know.
>>>>>
>>>>> Best regards,
>>>>> Christian
>>>>>
>>>>>>>
>>>>>>> However, Sierra is different, and when the CRAN people are
>>>>>>> experimenting
>>>>>>> with clang 4.0.0 for producing the Mac binaries, as Herve has
>>>>>>> mentioned,
>>>>>>> then backwards-compatibility would probably be lost anyhow.
>>>>>>
>>>>>> I think you misunderstood what Dan said. Backward-compatibility is
>>>>>> always lost i.e. binaries built on a given OS X versions are not
>>>>>> guaranteed to be backward compatible with older OS X versions. That's
>>>>>> why building them on the latest OS X version is a bad idea.
>>>>>>
>>>>>>>
>>>>>>> But I understand that this is a decision the CRAN people have to
>>>>>>> make.
>>>>>>
>>>>>> You're welcome to discuss this choice on the R-SIG-Mac mailing list.
>>>>>>
>>>>>> Cheers,
>>>>>> H.
>>>>>>
>>>>>>>
>>>>>>> Best regards,
>>>>>>> Christian
>>>>>>>
>>>>>>>
>>>>>>> On 03/24/17 01:10, Dan Tenenbaum wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>> ----- Original Message -----
>>>>>>>>> From: "Hervé Pagès" <hpages at fredhutch.org>
>>>>>>>>> To: "cstrato" <cstrato at aon.at>, "bioc-devel"
>>>>>>>>> <bioc-devel at r-project.org>
>>>>>>>>> Sent: Thursday, March 23, 2017 12:14:38 PM
>>>>>>>>> Subject: Re: [Bioc-devel] xps build problem on veracruz2
>>>>>>>>
>>>>>>>>> On 03/23/2017 11:09 AM, cstrato wrote:
>>>>>>>>>> Dear Herve,
>>>>>>>>>>
>>>>>>>>>> Thank you for your explanation.
>>>>>>>>>>
>>>>>>>>>> The reason that xps does not work with ROOT 6 is that I have
>>>>>>>>>> tried it
>>>>>>>>>> but there seem to be so many changes, that I did not succeed.
>>>>>>>>>> Since for xps there is no advantage using ROOT 6 vs ROOT 5, and
>>>>>>>>>> ROOT 5
>>>>>>>>>> was still supported, I have decided to stay with ROOT 5.
>>>>>>>>>
>>>>>>>>> OK
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> BTW, I have also one question:
>>>>>>>>>> Why did you decide to set up a new Mac with El Capitan instead of
>>>>>>>>>> using
>>>>>>>>>> the newest OS Sierra? (I have the impression that most Mac users
>>>>>>>>>> are
>>>>>>>>>> either happy to stay with their old OS or they upgrade to the
>>>>>>>>>> newest
>>>>>>>>>> one.)
>>>>>>>>>
>>>>>>>>> Same reason as for the choice of compilers: that's what the R
>>>>>>>>> folks
>>>>>>>>> decided to use for producing the Mac binaries of R and CRAN
>>>>>>>>> packages.
>>>>>>>>> We're just following their lead on that.
>>>>>>>>>
>>>>>>>>
>>>>>>>> Also, it's always good not to require users to upgrade if they
>>>>>>>> don't
>>>>>>>> have to. Building on El Capitan means users will not have to
>>>>>>>> upgrade
>>>>>>>> to macOS Sierra if they don't want to. Building on Sierra would
>>>>>>>> mean R
>>>>>>>> and packages would not be backwards-compatible with El Capitan.
>>>>>>>>
>>>>>>>> But it's a tradeoff that also involves the difficulty of
>>>>>>>> maintaining
>>>>>>>> build machines with old OSes, and wanting to take advantage of
>>>>>>>> newer
>>>>>>>> compiler technology. Otherwise R/Bioc would still be building on
>>>>>>>> Mavericks, or Snow Leopard...
>>>>>>>>
>>>>>>>> Dan
>>>>>>>>
>>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>> H.
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Best regards,
>>>>>>>>>> Christian
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 03/23/17 17:47, Hervé Pagès wrote:
>>>>>>>>>>> Hi Christian,
>>>>>>>>>>>
>>>>>>>>>>> The CRAN folks are currently experimenting with clang 4.0.0 for
>>>>>>>>>>> producing the Mac binaries of R and CRAN packages so we are
>>>>>>>>>>> using
>>>>>>>>>>> the same on veracruz2. This is a version of clang that is
>>>>>>>>>>> ahead of
>>>>>>>>>>> what's in XCode 8.x or XCode 7.x. So I guess that means we'll
>>>>>>>>>>> have
>>>>>>>>>>> to compile ROOT from source on veracruz2.
>>>>>>>>>>>
>>>>>>>>>>> BTW any reason not to make xps work with ROOT 6?
>>>>>>>>>>>
>>>>>>>>>>> Cheers,
>>>>>>>>>>> H.
>>>>>>>>>>>
>>>>>>>>>>> On 03/23/2017 07:28 AM, cstrato wrote:
>>>>>>>>>>>> Dear Valerie,
>>>>>>>>>>>>
>>>>>>>>>>>> I have seen that you have set up a new Mac server, veracruz2,
>>>>>>>>>>>> running El
>>>>>>>>>>>> Capitan.
>>>>>>>>>>>>
>>>>>>>>>>>> Although the development version of xps does even run on Mac OS
>>>>>>>>>>>> Sierra,
>>>>>>>>>>>> one issue still remains the same:
>>>>>>>>>>>>
>>>>>>>>>>>> You need to install the latest ROOT version 5, since xps does
>>>>>>>>>>>> not
>>>>>>>>>>>> run
>>>>>>>>>>>> with ROOT 6!
>>>>>>>>>>>>
>>>>>>>>>>>> So you need to install on veracruz2 the same root version that
>>>>>>>>>>>> you
>>>>>>>>>>>> have
>>>>>>>>>>>> installed on toluca2 running Maverics, i.e.
>>>>>>>>>>>> root_v5.34.36.macosx64-10.11-clang70.dmg
>>>>>>>>>>>>
>>>>>>>>>>>> However, if you have installed on El Capitan XCode 8.x
>>>>>>>>>>>> instead of
>>>>>>>>>>>> XCode
>>>>>>>>>>>> 7.x, then you need to compile ROOT from source, i.e.:
>>>>>>>>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__root.cern.ch_download_root-5Fv5.34.36.source.tar.gz&d=DwICAg&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=q9mk6yIytaNZlSdiLX_dFwchX8Tb7ra6x3WBBNIcs2o&s=Lz7YkqZ3XwjRsYIXVTbSvbDvTM-jTyoWvoVSa1PdBDw&e=
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> The README file of xps does explain how to compile ROOT for
>>>>>>>>>>>> Sierra. This
>>>>>>>>>>>> should also be valid for El Capitan running XCode 8.x.
>>>>>>>>>>>>
>>>>>>>>>>>> Thank you in advance.
>>>>>>>>>>>> Best regards,
>>>>>>>>>>>> Christian
>>>>>>>>>>>> _._._._._._._._._._._._._._._._._._
>>>>>>>>>>>> C.h.r.i.s.t.i.a.n S.t.r.a.t.o.w.a
>>>>>>>>>>>> V.i.e.n.n.a A.u.s.t.r.i.a
>>>>>>>>>>>> e.m.a.i.l: cstrato at aon.at
>>>>>>>>>>>> _._._._._._._._._._._._._._._._._._
>>>>>>>>>>>>
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> Bioc-devel at r-project.org mailing list
>>>>>>>>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_bioc-2Ddevel&d=DwICAg&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=q9mk6yIytaNZlSdiLX_dFwchX8Tb7ra6x3WBBNIcs2o&s=0bNMm-aoHuwWs9yBRjyGHTxT0y3UceNADHgMjtosTWU&e=
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Hervé Pagès
>>>>>>>>>
>>>>>>>>> Program in Computational Biology
>>>>>>>>> Division of Public Health Sciences
>>>>>>>>> Fred Hutchinson Cancer Research Center
>>>>>>>>> 1100 Fairview Ave. N, M1-B514
>>>>>>>>> P.O. Box 19024
>>>>>>>>> Seattle, WA 98109-1024
>>>>>>>>>
>>>>>>>>> E-mail: hpages at fredhutch.org
>>>>>>>>> Phone: (206) 667-5791
>>>>>>>>> Fax: (206) 667-1319
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Bioc-devel at r-project.org mailing list
>>>>>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_bioc-2Ddevel&d=DwIF-g&c=eRAMFD45gAfqt84VtBcfhQ&r=TF6f93hjWmgMzjqP9F3thRifibmFvfjc5Ae-bzNwDGo&m=WB1ofcLb-W4SN6VNAgoSRdgRXQRPaelptAH2g0Ur7q8&s=IDfsJGqV_D7hzqLryd27eoZNIuiAIfSNATUnxMy61oo&e=
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>>
--
Hervé Pagès
Program in Computational Biology
Division of Public Health Sciences
Fred Hutchinson Cancer Research Center
1100 Fairview Ave. N, M1-B514
P.O. Box 19024
Seattle, WA 98109-1024
E-mail: hpages at fredhutch.org
Phone: (206) 667-5791
Fax: (206) 667-1319
More information about the Bioc-devel
mailing list