[R-sig-Debian] Problem updating packages in 2.15.1 on Ubuntu 12.04

[Chris Evans]

[Sorry to those who know all of this, just delete, I think it's worth it 
getting in the list archives and hence the googleSphere.]

Paul Johnson sent the following  at 02/08/12 18:39:
> Greetings
>
> On Thu, Aug 2, 2012 at 4:55 AM, Chris Evans <chrishold at psyctc.org> wrote:
>> Paul Johnson sent the following  at 01/08/12 16:01:
>>
>>> On Wed, Aug 1, 2012 at 1:02 AM, Chris Evans <chrishold at psyctc.org>
>>> wrote:
>>>>
>>>> The specific message is: "Warning: package 'spatial' in library
>>>> '/usr/lib/R/library' will not be updated"
>>>>
>>>> and the number of libraries about which that's the complaint is now
>>>> increasingly almost daily so clearly something is wrong.
>>>>
>>> Yes, the something wrong is the system administrator :)
>>
>>
>> Sweet of you to confirm what I myself said ("Clearly I don't understand
>>
>> something and am doing something wrong.  Please can someone enlighten
>> me?")  However, I am only as good as the information I can find and I
>> honestly don't think that's very clear but let's move on from that bit
>> of who's the wrong one.
>>
>
> People often say I'm sweet :)

!!

>>> Please remember that you have a choice of installing R packages from
>>>   the R repositories using install.packages (or update.packages). That
>>>   downloads source code, compiles and installs. Generally, on Debian,
>>> it will install to /usr/local/lib/R/site-library.  But you may also
>>> install them as Debian packages that have been built on somebody
>>> else's system and wrapped up for your usage.  Those generally install
>>> into /usr/lib/R/site-library.
>>
>>
>> Are these mutually exclusive ways of doing things?  If so, I think we
>> should have a much clearer statement to that effect somewhere.  If not,
>> and I had assumed they were not, it would be useful to be told why the
>> two different library locations are used and how R handles things when
>> it may find copies of the same library in different library locations.
>>
>
> This is a decision made by the R packager of your Linux distribution,
> not by the R team. This pattern you are looking at is unique to
> Debian, although I noticed what Dirk was doing on Debian and thought
> it was really good, and I explained it to the RedHat packager, and
> they are doing something similar now.
>
>   It is a convenient policy, to use /usr/lib/R/library for the packages
> distributed with R,
>   /usr/lib/R/site-library for the packages built as Debian packages, and then
> encourage users who install their own to use /usr/loca/lib/R/site-library or
> ~/R.
>
> I don't use Debian packages very often because I, like you, want to
> compile my own.
>
> In order to diagnose your trouble, please lets concentrate on one
> problem at a time.  Do what I do now. Open a terminal. You run
>
> $ sudo R
>
> and then inside R, run this
>
>> install.packages("spatial", dep=T, repos="http://rweb.quant.ku.edu/cran")
>
> I put my repos there because I'm reasonably sure  it does not suck.  I
> can't speak
> to other servers.
>
> I us install.packages because it gives us a clear avenue to track
> what's going on.
>

That worked and got me this:

chris at chris-tosh:~$ sudo R
[sudo] password for chris:

R version 2.15.1 (2012-06-22) -- "Roasted Marshmallows"
Copyright (C) 2012 The R Foundation for Statistical Computing
ISBN 3-900051-07-0
Platform: i686-pc-linux-gnu (32-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

   Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

[Previously saved workspace restored]

 > install.packages("spatial", dep=T, repos="http://rweb.quant.ku.edu/cran")
Installing package(s) into ‘/home/chris/R/i686-pc-linux-gnu-library/2.15’
(as ‘lib’ is unspecified)
trying URL 'http://rweb.quant.ku.edu/cran/src/contrib/spatial_7.3-4.tar.gz'
Content type 'application/x-gzip' length 44360 bytes (43 Kb)
opened URL
==================================================
downloaded 43 Kb

* installing *source* package ‘spatial’ ...
** package ‘spatial’ successfully unpacked and MD5 sums checked
** libs
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG      -fpic  -O3 -pipe  -g 
  -c init.c -o init.o
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG      -fpic  -O3 -pipe  -g 
  -c krc.c -o krc.o
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG      -fpic  -O3 -pipe  -g 
  -c pps.c -o pps.o
gcc -std=gnu99 -shared -o spatial.so init.o krc.o pps.o -L/usr/lib/R/lib -lR
installing to /home/chris/R/i686-pc-linux-gnu-library/2.15/spatial/libs
** R
** inst
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded

* DONE (spatial)

The downloaded source packages are in
	‘/tmp/RtmpPatSiy/downloaded_packages’


So that (to me) is interesting.  No complaint and just installs into my 
own user space library location.

> That gives me 2 copies of spatial, one installed from the Debian
> package r-cran-spatial, and the one I just compiled.
>
> We need to see the entire output you get when you run the same.
>
> Note that it put the package into /usr/local/lib/R/site-library
> because I'm building this by hand.

And for me it's put it in

‘/home/chris/R/i686-pc-linux-gnu-library/2.15’

which didn't exist when I was first running into the warning message.
>
> How did it know to do that?  Check the file:
>
> /etc/R/Renviron
>
> and look for:
>
> R_LIBS_SITE=${R_LIBS_SITE-'/usr/local/lib/R/site-library:/usr/lib/R/site-library:/usr/lib/R/library'}

Isn't that just the non-user bit of .libPaths()?  Isn't .libPaths() the 
concatenation of the user specific location and that?

> When you run the command, please include everything in the output, even
> he parts you think might not be relevant.

Done, see above.

>
> I'm cutting all the rest of your stuff below because we need to get to
> the bottom
> of the one problem first, and the best way to do that is what I say
> above. I can say for sure your compile of RGtk2 fails because you
> don't have the development packages installed. That's what it means
> when it says:
>
> checking for GTK... no
>
> Also, the other errors you see are related to user privileges.  If you
> forget to "sudo" before R, you are running as a non-root user. Look
> what I get when I do that:
>> update.packages()
> --- Please select a CRAN mirror for use in this session ---
> Loading Tcl/Tk interface ... done
> Warning: package 'rgl' in library '/usr/lib/R/site-library' will not be updated
> Warning: package 'MASS' in library '/usr/lib/R/library' will not be updated
> Warning: package 'mgcv' in library '/usr/lib/R/library' will not be updated
>
> Looks just like your errors, doesn't it??

It does but I have absolutely definitely been running R using "sudo R" 
so that's not the explanation and is part of what has been baffling me.

I _think_ I'm getting a bit clearer.  You used install.packages() and 
clearly that doesn't bother itself with what is already installed.

If I then do "update.packages()" in the same sudo R session I get:

 > update.packages(ask=FALSE)
--- Please select a CRAN mirror for use in this session ---
Warning: package 'Amelia' in library '/usr/lib/R/site-library' will not 
be updated
Warning: package 'bayesm' in library '/usr/lib/R/site-library' will not 
be updated

... and lots more like that.

So now I use install.packages() to force a new install of Amelia into 
.libPaths()[3], i.e. into /lib/R/site-library which is my third location 
on the .libPaths() list:

 > .libPaths()
[1] "/home/chris/R/i686-pc-linux-gnu-library/2.15"
[2] "/usr/local/lib/R/site-library"
[3] "/usr/lib/R/site-library"
[4] "/usr/lib/R/library"

 > install.packages("Amelia", lib=.libPaths()[3], dep=TRUE)
trying URL 'http://stat.ethz.ch/CRAN/src/contrib/Amelia_1.6.3.tar.gz'
Content type 'application/x-gzip' length 1060878 bytes (1.0 Mb)
opened URL
==================================================
downloaded 1.0 Mb

* installing *source* package ‘Amelia’ ...
** package ‘Amelia’ successfully unpacked and MD5 sums checked
** R
** data
** inst
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
    ‘amelia.Rnw’
** testing if installed package can be loaded

* DONE (Amelia)

The downloaded source packages are in
	‘/tmp/RtmpqdkM2D/downloaded_packages’




That succeeded so there's no issue of missing support packages or 
permissions nor of a bad repository.  If I rerun update.packages() now 
there is no warning about Amelia so clearly the checking that 
update.packages() does now recognises that the library is up to date. 
However, there is still a warning that bayesm won't be updated in that 
location.

So, I think we agree this isn't about permissions nor about absence of 
other supporter programs or packages (though I agree that's the issue 
for the other two packages, RGtk2 and rggobi).  To me it looks to be 
something about the way that R uses update.packages().

I _MAY_ be that all that is going on is that R is warning me that 
there's a copy of the libraries that it has updated in 
/usr/local/lib/R/site-library, that is left behind in 
/usr/lib/R/site-library and that I can delete that.

NO: that's not the case, bayesm hasn't been updated in either 
/usr/local/lib/R/site-library or the user directory.

OK.  I stick with my view that there is something about the way 
update.packages() is working for me that does not seem very logical (to me).



> I'm not an unkind person, I'm not trying to insult you by ignoring the
> rest of your  message.  Lets fix one thing at a time.

Fine.  You have definitely helped nudged me on in understanding these 
issues.

I will answer the other things separately.

Many thanks,

Chris


-- 
Chris Evans <chris at psyctc.org> Skype: chris-psyctc
Consultant Psychiatrist in Psychotherapy, Notts. PDD network;
Professor, Psychotherapy, Nottingham University
*If I am writing from one of those roles, it will be clear. Otherwise*
*my views are my own and not representative of those institutions    *
If you have difficulty Emailing me on this address or getting a reply,
send again but cc to:       chris dot evans at nottshc dot nhs dot uk
and to:                     c dot evans at nottingham dot ac dot uk