[Rd] dependencies on system packages

Fri Feb 4 20:29:59 CET 2011

On Feb 4, 2011, at 1:24 PM, Dirk Eddelbuettel wrote:

> 
> Simon,
> 
> On 4 February 2011 at 10:01, Simon Urbanek wrote:
> | Claudia,
> | 
> | thanks for you comments .
> | 
> | On Feb 4, 2011, at 3:18 AM, Claudia Beleites wrote:
> | 
> | > Dear all,
> | > 
> | > From the writing extensions manual:
> | > "Other dependencies (external to the R system) should be listed in the ‘SystemRequirements’ field, possibly amplified in a separate README file."
> | > 
> | > I guess one problem is the user may not realize that the -dev version is needed, and just sees libxml2 installed but the R package installation stopping with the respective error.
> | > 
> | 
> | I'd argue that if a user attempts to install a package from sources instead
> | of using the distribution binaries, he should know what he's doing as there
> | is much more involved (proper tools, usually a different library location
> | etc.). And anyone who knows what he's doing also knows that -dev packages
> | are needed (at the latest when the installation fails you remember ;)). If
> | he doesn't then it should give him a clue that he may want to use something
> | else (and especially Linux users should know better ;)). 
> | 
> | Clearly, it doesn't prevent users from doing stupid things and I completely
> | agree with you that the README should have the instructions as far as the
> | developer knows. And as a package developer you'll learn soon enough when
> | people start complaining ;).
> 
> I respectfully disagree.
> 
> Based on a number of years of supporting users on Linux where people install
> frequently from source, I can assure you that a rather large number of people
> fails.  Not everybody is fluent with compilers, knows about libraries and
> their interdependencies, or can even read configure error messages. We all
> see the r-help messages (or the traffic on the SIG lists).
> 
> People want to use the wealth of software that is CRAN, and we should help
> them.  I have also been involved in by now two attempts to overcome this in
> an automated fashion via cran2deb.  We had that working somewhat reliably
> until parts of the infrastructure misteriously self-destructed (a large
> sqlite table) right when I visited Vienna, and are now in a rewrite which may
> be never ending (for lack of resources).  There was a lot of interest for it
> when it worked, and there is ongoing interest right now (as a few guys from
> across Europe just met last weekend to try to use for BioC builds).
> 
> There were also folks from other distros who tried something similar. What
> Jeroen suggested is along those lines with the needed meta-data. Whether we
> make it per-package (as per Jeroen's idea) or 'per-repo-distro-pair' (which
> is what cran2deb does) is a detail.  
> 
> We need to address this: With 2600+ packages and continued growth, manually
> wading through README is not good enough.  We should do better.  Resources
> (time, money, servers, ...) would help.  Maybe one day someone with more time
> can fold this into a proper sub-project of a larger grant application.  It
> would be worth, and I would try to help, time permitting.
> 

Well, as far as I can see you're only agreeing with me :). I said people like you are solving the issues by creating the corresponding distro-specific description and people should be thankful for that. Also I said that the distro-specific way is the only reliable way as package authors cannot know the intricacies of the distros involved. If the distros could share a system that helps to maintain this, it would be perfect - by all mean supported by us if we can, but it's separate from the package authors. And, finally, I also said that it would be nice to have a parseable field that is disto-independent so the distro-maintainer don't need to weed through READMEs to find dependency sources. I'm sorry to hear that cran2deb self-destructed as your Debian handling of packages is immensely helpful, especially for package with intricate dependencies.

On a similar note, I also think that it would be useful to have a common support already at the package level -- very few people know how to write good configure scripts and there are many pitfalls that catch the unwary, so we could have a template for the most common case of requiring a few libraries. This would also allow some auto-maigc handling of the extended requirement, possibly with some site support ... (e.g. you could imagine searching debs for the library files that are required and coming up with a suggestion of packages etc.).

Cheers,
Simon

> | 
> | Thanks,
> | Simon
> | 
> | 
> | 
> | > Giving the package name for specific distributions is of course polite (if the developer knows it). As developer you may also put into the README that the package's mailing list/forum/wiki/... contains information and ask the user to enter the package name on his distro if it is not already there.
> | > 
> | > 
> | > my 2 ct
> | > 
> | > Claudia
> | > 
> | > 
> | > 
> | > Am 04.02.2011 04:48, schrieb Simon Urbanek:
> | >> Jeroen,
> | >> 
> | >> On Feb 3, 2011, at 9:31 PM, Jeroen Ooms wrote:
> | >> 
> | >>> 
> | >>> Many R packages depend on some unix libraries that are not part of most
> | >>> default installations. I often spend a significant amount of time figuring
> | >>> out where to get the appropriate libraries for compiling these packages,
> | >>> after they give some vague error of something missing. I was wondering why
> | >>> there is no formal system of specifying non-R dependencies in the
> | >>> DESCRIPTION file. If this would be the case, then during the installation of
> | >>> an R package, the user could be prompted to install required system packages
> | >>> (if they have appropriate privileges).
> | >>> 
> | >>> So for example:
> | >>> 
> | >>> Package: XML
> | >>> Version: 3.2-0
> | >>> Depends: R (>= 1.2.0), methods, utils
> | >>> Depends-debian: libxml2-dev
> | >>> Depends-ubuntu: libxml2-dev
> | >>> Depends-redhat: libxml2-devel
> | >>> Depends-suse: libxml2-devel
> | >>> etc.
> | >>> 
> | >>> This might make life for many people just a little easier. If they are root
> | >>> and the package is in their system repositories, than it will install
> | >>> automatically. If not, at least they know for which package to look, or
> | >>> request their sys admin to install.
> | >> 
> | >> Well, there is already such system in place and it is the corresponding descriptions in the distributions. Obviously as an author of the package I don't care what any particular Linux distribution uses as a name for the needed dependencies as the corresponding chaos is distribution-specific. The only person who can reasonably determine the dependencies is the maintainer of the distribution and that's what they do and as a user of the above mentioned distributions you should be thankful to them. Fortunately, normal users don't have to worry about it as major distributions already come with a large set of R packages resolving all dependencies. Hence I don't see any reason why this should have anything to do with the DESCRIPTION file. The improvements I could think of would be a parseable entry or a canonical pointer to dependency sources, but that's a whole another story.
> | >> 
> | >> Cheers,
> | >> Simon
> | >> 
> | >> ______________________________________________
> | >> R-devel at r-project.org mailing list
> | >> https://stat.ethz.ch/mailman/listinfo/r-devel
> | > 
> | > ______________________________________________
> | > R-devel at r-project.org mailing list
> | > https://stat.ethz.ch/mailman/listinfo/r-devel
> | > 
> | > 
> | 
> | ______________________________________________
> | R-devel at r-project.org mailing list
> | https://stat.ethz.ch/mailman/listinfo/r-devel
> 
> -- 
> Dirk Eddelbuettel | edd at debian.org | http://dirk.eddelbuettel.com
> 
> 