[R] Missing dependencies in pkg installs
Conklin, Mike (GfK)
Mike.Conklin at gfk.com
Thu Jun 22 23:02:51 CEST 2017
I am using debug on the .install_packages function...stepping through. Once the temporary folder is created and the tar file expanded I run file_test and get a FALSE back indicating that the configure file is not executable.
[1] "/tmp/RtmpMM6iC1/R.INSTALLc5ca415e4310/stringi"
Browse[2]> dir(new)
[1] "DESCRIPTION" "INSTALL" "LICENSE" "MD5"
[5] "NAMESPACE" "NEWS" "R" "cleanup"
[9] "configure" "configure.win" "inst" "man"
[13] "src"
Browse[2]> test_file("-x",paste0(new,"/configure")
+ )
Error in test_file("-x", paste0(new, "/configure")) :
could not find function "test_file"
Browse[2]> file_test("-x",paste0(new,"/configure"))
/tmp/RtmpMM6iC1/R.INSTALLc5ca415e4310/stringi/configure
FALSE
However, at the same time, in a separate session I have the system look at the very same file and clearly the execution bit is set.
[root at dcex1102shinypr ~]# ls -l /tmp/RtmpMM6iC1/R.INSTALLc5ca415e4310/stringi
total 312
-rwxr-xr-x 1 root root 66 Apr 2 2015 cleanup
-rwxr-xr-x 1 root root 173757 Apr 7 11:43 configure
-rw-r--r-- 1 root root 669 Jun 23 2015 configure.win
-rw-r--r-- 1 root root 1451 Apr 7 15:08 DESCRIPTION
drwxr-xr-x 2 root root 35 Jun 22 22:26 inst
-rw-r--r-- 1 root root 6207 Apr 7 11:21 INSTALL
-rw-r--r-- 1 root root 3580 Mar 21 13:29 LICENSE
drwxr-xr-x 2 root root 4096 Jun 22 22:26 man
-rw-r--r-- 1 root root 63692 Apr 7 15:08 MD5
-rw-r--r-- 1 root root 6204 Oct 24 2016 NAMESPACE
-rw-r--r-- 1 root root 22407 Apr 7 11:44 NEWS
drwxr-xr-x 2 root root 4096 Jun 22 22:26 R
drwxr-xr-x 3 root root 8192 Jun 22 22:26 src
[root at dcex1102shinypr ~]#
If I CTRL-Z out of R in the first session I confirm the same result - the system shows the file as executable
--
W. Michael Conklin
EVP Marketing & Data Sciences
GfK
T +1 763 417 4545 | M +1 612 567 8287
-----Original Message-----
From: Duncan Murdoch [mailto:murdoch.duncan at gmail.com]
Sent: Thursday, June 22, 2017 3:06 PM
To: Conklin, Mike (GfK); Martin Maechler; David Winsemius
Cc: r-help at r-project.org
Subject: Re: [R] Missing dependencies in pkg installs
On 22/06/2017 3:42 PM, Conklin, Mike (GfK) wrote:
> Not much progress..... I step through debug and it gets to the do.install() function which immediately errors with the same "configuration not executable" error.
I believe that is a locally defined function, which means you can set a breakpoint within it but only if R is compiled with source info, or you can manually call debug(do_install) after it has been created (typically just before it is called is a safe place to do this), and do the usual process.
It's still very mysterious to me why configure is losing its executable bit. What I would do if I could reproduce this would be to run right up to the file_test("-x","configure") line, then print my working directory, and ctrl-Z out of R to look at the files there. That should confirm that configure is not executable.
Then the hard part is figuring out why it isn't....
Duncan
>
> So, made a tempfunc that was a copy of tools:::.install_packages and
> edited the file_test("-x","configure") line to return a TRUE
>
> now I get a Permission Denied error (even if I run as root)
>
>> tempfunc("/home/meconk/stringi_1.1.5.tar.gz")
> * installing to library '/usr/lib64/R/library'
> * installing *source* package 'stringi' ...
> ** package 'stringi' successfully unpacked and MD5 sums checked
> sh: ./configure: Permission denied
> ERROR: configuration failed for package 'stringi'
> * removing '/usr/lib64/R/library/stringi'
> Error in do_exit(status = status) : .install_packages() exit status 1
> Error in do_exit(status = status) : .install_packages() exit status 1
>
> --
> W. Michael Conklin
> EVP Marketing & Data Sciences
> GfK
> T +1 763 417 4545 | M +1 612 567 8287
>
>
> -----Original Message-----
> From: Duncan Murdoch [mailto:murdoch.duncan at gmail.com]
> Sent: Thursday, June 22, 2017 11:09 AM
> To: Conklin, Mike (GfK); Martin Maechler; David Winsemius
> Cc: r-help at r-project.org
> Subject: Re: [R] Missing dependencies in pkg installs
>
> On 22/06/2017 11:15 AM, Conklin, Mike (GfK) wrote:
>> Following Duncan's instructions I find that the system and R find that configure IS executable but if trying to install via install.packages I get the same error.
>> I also tried using R CMD INSTALL from the terminal and install.packages with a local file pointing to the very same tar.gz file that shows the executable bit set. All result in the same "configure is not executable" result.
>>
>
> That's very mysterious. This is hard to debug, because R runs a separate process to do the installs. If you want to track this down you can debug the code as follows.
>
> 1. Manually download the tarball, using any method (perhaps download.packages("stringi", destdir = ".", type = "source").
>
> 2. In R, run
>
> debug(tools:::.install_packages)
> tools:::.install_packages("stringi_1.1.5.tar.gz")
>
> You can single step until you see the message, and try to diagnose why it is happening.
>
> Watch out, this function will exit R at the end.
>
> Duncan Murdoch
>
>> [meconk at dcex1102shinypr ~]$ ls -l stringi/configure -rwxr-xr-x 1
>> meconk meconk 173757 Apr 7 11:43 stringi/configure
>> [meconk at dcex1102shinypr ~]$ sudo R [sudo] password for meconk:
>>
>> R version 3.4.0 (2017-04-21) -- "You Stupid Darkness"
>> Copyright (C) 2017 The R Foundation for Statistical Computing
>> Platform: x86_64-redhat-linux-gnu (64-bit)
>>
>> R is free software and comes with ABSOLUTELY NO WARRANTY.
>> You are welcome to redistribute it under certain conditions.
>> Type 'license()' or 'licence()' for distribution details.
>>
>> Natural language support but running in an English locale
>>
>> R is a collaborative project with many contributors.
>> Type 'contributors()' for more information and 'citation()' on how to
>> cite R or R packages in publications.
>>
>> Type 'demo()' for some demos, 'help()' for on-line help, or
>> 'help.start()' for an HTML browser interface to help.
>> Type 'q()' to quit R.
>>
>> [Previously saved workspace restored]
>>
>>> getwd()
>> [1] "/home/meconk"
>>> file_test("-x","stringi/configure")
>> stringi/configure
>> TRUE
>>> install.packages("stringi") Installing package into '/usr/lib64/R/library'
>> (as 'lib' is unspecified)
>> trying URL 'https://cloud.r-project.org/src/contrib/stringi_1.1.5.tar.gz'
>> Content type 'application/x-gzip' length 3645872 bytes (3.5 MB)
>> ==================================================
>> downloaded 3.5 MB
>>
>> * installing *source* package 'stringi' ...
>> ** package 'stringi' successfully unpacked and MD5 sums checked
>> ERROR: 'configure' exists but is not executable -- see the 'R Installation and Administration Manual'
>> * removing '/usr/lib64/R/library/stringi'
>>
>> The downloaded source packages are in
>> '/tmp/Rtmpxw9twb/downloaded_packages'
>> Updating HTML index of packages in '.Library'
>> Making 'packages.html' ... done
>> Warning message:
>> In install.packages("stringi") :
>> installation of package 'stringi' had non-zero exit status
>>
>> --
>> W. Michael Conklin
>> EVP Marketing & Data Sciences
>> GfK
>> T +1 763 417 4545 | M +1 612 567 8287
>>
>> -----Original Message-----
>> From: R-help [mailto:r-help-bounces at r-project.org] On Behalf Of
>> Duncan Murdoch
>> Sent: Thursday, June 22, 2017 8:23 AM
>> To: Martin Maechler; David Winsemius
>> Cc: r-help at r-project.org
>> Subject: Re: [R] Missing dependencies in pkg installs
>>
>> The "configure exists but is not executable" problem is somewhat
>> common on Windows, because there's usually no such thing as an
>> executable bit there. (Cygwin does something to fake one, but
>> Windows generally
>> doesn't.) If you create a tarball there by default you get no executable bits marked in it.
>>
>> For a long time, R CMD build has dealt with this issue by using the internal tar() function. It corrects the executable bit with a warning during the build.
>>
>> So if people are getting that error, something has gone wrong. A few
>> guesses:
>>
>> - They are on Linux but using a mount of a Windows volume that doesn't preserve the bit.
>>
>> - They are using a tarball produced in some strange way on Windows, e.g. by calling tar directly instead of R CMD build.
>>
>> - There's a bug in R in detecting the executable bit.
>>
>> Martin saw configure marked executable in the tarball from CRAN,
>> which means the second is unlikely (unless it was very recently
>> fixed), but the first is still possible. One test is to manually
>> expand the tarball, then use both ls and R's test to see if the
>> executable bit is set and is detected by R. For example, after
>> downloading stringi_1.1.5.tar.gz you expand it using
>>
>> tar zxvf stringi_1.1.5.tar.gz
>>
>> and use
>>
>> ls -l stringi/configure
>>
>> to see if the system thinks it is executable, and in R,
>>
>> file_test("-x", "stringi/configure")
>>
>> to see what R sees.
>>
>> Duncan Murdoch
>>
>> On 22/06/2017 3:38 AM, Martin Maechler wrote:
>>>>>>>> David Winsemius <dwinsemius at comcast.net>
>>>>>>>> on Wed, 21 Jun 2017 18:04:13 -0700 writes:
>>>
>>> >> On Jun 21, 2017, at 1:39 PM, Conklin, Mike (GfK) <Mike.Conklin at gfk.com> wrote:
>>> >>
>>> >> I have a Ubuntu server with an R installation that has 384 packages installed. We are trying to replicate the system on a Red Hat Enterprise server. I downloaded the list of packages from the Ubuntu machine and read it into an R session on the new machine. Then I ran install.packages(listofpackages). Now I have 352 packages on the new machine but several very common packages (like much of the tidyverse, ggplot2, Hmisc) failed to install because of missing dependencies (most of which are the other packages that failed to install).
>>> >>
>>> >> R version 3.4.0 (2017-04-21)
>>> >> Platform: x86_64-redhat-linux-gnu (64-bit)
>>>
>>> > I'd make sure you have reviewed this:
>>>
>>> >
>>> https://cran.r-project.org/doc/manuals/r-patched/R-admin.html#Essent
>>> i
>>> a
>>> l-programs-and-libraries
>>>
>>> yes, but see also below ..
>>>
>>> > Best;
>>> > David.
>>>
>>> >> Running under: Red Hat Enterprise Linux Server 7.2 (Maipo)
>>> >>
>>> >> Matrix products: default
>>> >> BLAS/LAPACK: /usr/lib64/R/lib/libRblas.so
>>> >>
>>> >> locale:
>>> >> [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
>>> >> [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
>>> >> [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
>>> >> [7] LC_PAPER=en_US.UTF-8 LC_NAME=C
>>> >> [9] LC_ADDRESS=C LC_TELEPHONE=C
>>> >> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
>>> >>
>>> >> attached base packages:
>>> >> [1] stats graphics grDevices utils datasets methods base
>>> >>
>>> >> other attached packages:
>>> >> [1] dplyr_0.7.0 shiny_1.0.3
>>> >>
>>> >> loaded via a namespace (and not attached):
>>> >> [1] compiler_3.4.0 magrittr_1.5 assertthat_0.2.0 R6_2.2.2
>>> >> [5] htmltools_0.3.6 tools_3.4.0 glue_1.1.1 tibble_1.3.3
>>> >> [9] Rcpp_0.12.11 digest_0.6.12 xtable_1.8-2 httpuv_1.3.3
>>> >> [13] mime_0.5 rlang_0.1.1
>>> >>>
>>> >>
>>> >>
>>> >> If I try and install Hmisc for example I get the following errors:
>>> >> for the first error ERROR: 'configure' exists but is not executable --
>>> >> see the 'R Installation and dministration Manual' I did
>>> >> not find the R Installation and Administration Manual
>>> >> helpful.
>>>
>>> why? It does assume you spend some time understanding what it is
>>> talking about.
>>> OTOH, it is *THE* official document on the topic, written and
>>> constantly updated by the R core team.
>>>
>>> Anyway: " ERROR: 'configure' exists but is not executable "
>>>
>>> is I think a crucial piece of information .. it's from stringi's
>>> installation failure, and the top level directory of stringi (1.1-5)
>>> in a Unix like (ls -l with user group names removed) looks like:
>>>
>>> -rw-r--r--. 1451 Apr 7 15:08 DESCRIPTION
>>> -rw-r--r--. 6207 Apr 7 11:21 INSTALL
>>> -rw-r--r--. 3580 Mar 21 13:29 LICENSE
>>> -rw-r--r--. 63692 Apr 7 15:08 MD5
>>> -rw-r--r--. 6204 Oct 24 2016 NAMESPACE
>>> -rw-r--r--. 22407 Apr 7 11:44 NEWS
>>> drwxr-xr-x. 8192 Mar 28 10:26 R
>>> -rwxr-xr-x. 66 Apr 2 2015 cleanup
>>> -rw-rw-r--. 32193 Apr 8 05:46 config.log
>>> -rwxrwxr-x. 40648 Apr 8 05:46 config.status
>>> -rwxr-xr-x. 173757 Apr 7 11:43 configure
>>> -rw-r--r--. 669 Jun 23 2015 configure.win
>>> drwxr-xr-x. 512 Apr 7 11:50 inst
>>> drwxr-xr-x. 8192 Mar 28 10:26 man
>>> drwxr-xr-x. 16384 Apr 8 05:47 src
>>>
>>> Note the "x"s in the 'configure' line's " -rwxr-xr-x. ":
>>> This means the file is executable and a reasonable shell can just
>>> "call the file" and it will be executed, but that's the part which
>>> failed for you.
>>>
>>> .. this *is* peculiar as it looks like some of the standard Unix
>>> tools may be misbehaving for you .. I assume it could be some OS
>>> security "feature" playing against you..
>>>
>>> >>
>>> >>
>>> >>> install.packages("Hmisc")
>>> >> Installing package into '/usr/lib64/R/library'
>>> >> (as 'lib' is unspecified)
>>> >> also installing the dependencies 'stringi', 'evaluate', 'reshape2', 'stringr', knitr', 'ggplot2', 'htmlTable', 'viridis'
>>> >>
>>> >> trying URL 'https://cloud.r-project.org/src/contrib/stringi_1.1.5.tar.gz'
>>> >> Content type 'application/x-gzip' length 3645872 bytes (3.5 MB)
>>> >> ==================================================
>>> >> downloaded 3.5 MB
>>> >>
>>> >> trying URL 'https://cloud.r-project.org/src/contrib/evaluate_0.10.tar.gz'
>>> >> Content type 'application/x-gzip' length 21914 bytes (21 KB)
>>> >> ==================================================
>>> >> downloaded 21 KB
>>> >>
>>> >> trying URL 'https://cloud.r-project.org/src/contrib/reshape2_1.4.2.tar.gz'
>>> >> Content type 'application/x-gzip' length 34688 bytes (33 KB)
>>> >> ==================================================
>>> >> downloaded 33 KB
>>> >>
>>> >> trying URL 'https://cloud.r-project.org/src/contrib/stringr_1.2.0.tar.gz'
>>> >> Content type 'application/x-gzip' length 94095 bytes (91 KB)
>>> >> ==================================================
>>> >> downloaded 91 KB
>>> >>
>>> >> trying URL 'https://cloud.r-project.org/src/contrib/knitr_1.16.tar.gz'
>>> >> Content type 'application/x-gzip' length 1031259 bytes (1007 KB)
>>> >> ==================================================
>>> >> downloaded 1007 KB
>>> >>
>>> >> trying URL 'https://cloud.r-project.org/src/contrib/ggplot2_2.2.1.tar.gz'
>>> >> Content type 'application/x-gzip' length 2213308 bytes (2.1 MB)
>>> >> ==================================================
>>> >> downloaded 2.1 MB
>>> >>
>>> >> trying URL 'https://cloud.r-project.org/src/contrib/htmlTable_1.9.tar.gz'
>>> >> Content type 'application/x-gzip' length 152095 bytes (148 KB)
>>> >> ==================================================
>>> >> downloaded 148 KB
>>> >>
>>> >> trying URL 'https://cloud.r-project.org/src/contrib/viridis_0.4.0.tar.gz'
>>> >> Content type 'application/x-gzip' length 1787731 bytes (1.7 MB)
>>> >> ==================================================
>>> >> downloaded 1.7 MB
>>> >>
>>> >> trying URL 'https://cloud.r-project.org/src/contrib/Hmisc_4.0-3.tar.gz'
>>> >> Content type 'application/x-gzip' length 702759 bytes (686 KB)
>>> >> ==================================================
>>> >> downloaded 686 KB
>>> >>
>>> >> * installing *source* package 'stringi' ...
>>> >> ** package 'stringi' successfully unpacked and MD5 sums checked
>>> >> ERROR: 'configure' exists but is not executable -- see the 'R Installation and dministration Manual'
>>> >> * removing '/usr/lib64/R/library/stringi'
>>> >> ERROR: dependency 'stringi' is not available for package 'stringr'
>>> >> * removing '/usr/lib64/R/library/stringr'
>>> >> ERROR: dependency 'stringr' is not available for package 'evaluate'
>>> >> * removing '/usr/lib64/R/library/evaluate'
>>> >> ERROR: dependency 'stringr' is not available for package 'reshape2'
>>> >> * removing '/usr/lib64/R/library/reshape2'
>>> >> ERROR: dependencies 'evaluate', 'stringr' are not available for package 'knitr'
>>> >> * removing '/usr/lib64/R/library/knitr'
>>> >> ERROR: dependency 'reshape2' is not available for package 'ggplot2'
>>> >> * removing '/usr/lib64/R/library/ggplot2'
>>> >> ERROR: dependencies 'stringr', 'knitr' are not available for package 'htmlTable
>>> >> * removing '/usr/lib64/R/library/htmlTable'
>>> >> ERROR: dependency 'ggplot2' is not available for package 'viridis'
>>> >> * removing '/usr/lib64/R/library/viridis'
>>> >> ERROR: dependencies 'ggplot2', 'htmlTable', 'viridis' are not available for pacage 'Hmisc'
>>> >> * removing '/usr/lib64/R/library/Hmisc'
>>> >>
>>> >>
>>> >> Any help is appreciated.
>>> >> --
>>> >> W. Michael Conklin
>>> >> Executive Vice President
>>> >> Marketing & Data Sciences - North America
>>> >> GfK | 8401 Golden Valley Road | Minneapolis | MN | 55427
>>> >> mike.conklin at gfk.com<mailto:mike.conklin at gfk.com>
>>> >> T +1 763 417 4545 | M +1 612 567 8287
>>> >> www.gfk.com<http://www.gfk.com/>
>>> >>
>>> >>
>>> >> [[alternative HTML version deleted]]
>>> >>
>>> >> ______________________________________________
>>> >> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
>>> >> https://stat.ethz.ch/mailman/listinfo/r-help
>>> >> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>>> >> and provide commented, minimal, self-contained, reproducible code.
>>>
>>> > David Winsemius
>>> > Alameda, CA, USA
>>>
>>> > ______________________________________________
>>> > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
>>> > https://stat.ethz.ch/mailman/listinfo/r-help
>>> > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>>> > and provide commented, minimal, self-contained, reproducible code.
>>>
>>> ______________________________________________
>>> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>> PLEASE do read the posting guide
>>> http://www.R-project.org/posting-guide.html
>>> and provide commented, minimal, self-contained, reproducible code.
>>>
>>
>> ______________________________________________
>> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>
More information about the R-help
mailing list