[Rd] Issues from GNU tar >= 1.24

[Prof Brian Ripley]

[Moved from R-help with a new subject line.]

R CMD build used with GNU tar >= 1.24 (which is only a couple of 
months old and so not in widespread use) can create tarballs 
containing hard links if the package sources contain (symbolic or 
hard) links.  RGtk2_2.20.4.tar.gz was such a tarball.

I have added support for hard links in R-devel, so such tarballs will 
be unpacked with hard links (or if that fails file copies).

Hard links are supported on POSIX file systems and on NTFS on Windows. 
So this should work fine (and tests correctly) where such file systems 
are in use -- the main exception would be FAT file systems (which 
because of their OS-portability are widely used for external disks and 
USB drives), and those FAT systems tested[*] seem also to work.

The claim about other tars seems false: the other two commonly used 
tars (bsdtar from libarchive used in Mac OS 10.6 and FreeBSD, and AT&T 
Unix tar) deference symbolic links to ordinary files.

So the current position is:

If you have symbolic links in your package sources and use GNU tar >= 
1.24 and use R CMD build to make the package tarball:

- R < 2.11.0 relies on an external tar program
- 2.11.0 <= R <= 2.12.1 may not unpack the tarball correctly on a
   Unix-alike.
- Somewhat accidentally R <= 2.12.1 should unpack it correctly on
   Windows, expanding hard links to file copies.
- R-patched will unpack the tarball correctly, expanding hard links to
   file copies.
- R-devel will unpack the tarball correctly on file systems which
   support hard links, and on the FAT file systems tested.

It seems the best workarounds are

- Do not have links in your package sources (and that advice has been
    added to 'Writing R Extensions')
- If you encounter a tarball which contains links (which can be hard
   to detect: 'tar tvf' does not necessarily list them as such), try
   setting environment variable R_INSTALL_TAR to the path to a 'tar'
   program that handles such links.

[*] Linux, Mac OS X and Windows reported failures on file.link() and 
fell back to file copies.  However, Mac OS X (and not the others) 
seems to support symbolic links on FAT file systems.

On Thu, 6 Jan 2011, Michael Lawrence wrote:

> 
> 
> On Thu, Jan 6, 2011 at 8:53 AM, Prof Brian Ripley <ripley at stats.ox.ac.uk>
> wrote:
>       You need RGtk2 2.20.7 which is now on CRAN.  Others have seen
>       this, but it has taken a while to track down the exact cause.
>
>       The diagnosis was that ML used a recent GNU tar which created a
>       tarball with hard links that R's untar was not prepared to deal
>       with. We consider that is a bug in GNU tar, but untar() has been
>       updated in R-patched to cope.
> 
> 
> After a lot of back and forth with the GNU tar guys, it turns out they do
> not consider this to be a bug. I had to refresh my knowledge of hard linking
> to understand. A hard link is from a file name to the actual inode in the
> file system. Typically every file has a single hard link (the name of the
> file). The -h option used to resolve a symbolic link differently, based on
> whether the hard link count of the target was 1 or >=2. This was practically
> useful in my mind, because symlinks to any files without any explicitly
> added hard links would become a regular file in the archive. They have now
> dropped this distinction, calling it an inconsistency (apparently other
> implementations of tar have never made such a distinction). So symlinks now
> become hard links in the archive (as long as the target is in the archive).
> We may need to keep the fix in untar() to handle this. Either way, RGtk2
> 2.20.7 should work now.
> 
> Thanks,
> Michael
>  
>       If you have such a tarball, try setting the environment variable
>       R_INSTALL_TAR to 'tar' (or whatever GNU tar is called on your
>       system) when installing the tarball.
>
>       For those packaging source packages: in the unusual event that
>       your package sources contains symbolic (or even hard) links,
>       don't use GNU tar 1.24 or 1.25.
>
>       On Thu, 6 Jan 2011, Shige Song wrote:
>
>             Look forward to it.
>
>             Thanks.
>
>             Shige
>
>             On Sat, Jan 1, 2011 at 8:45 AM, Michael Lawrence
>             <lawrence.michael at gene.com> wrote:
>                   Please watch for 2.20.5 and let me know
>                   if it helps. Not really sure what is
>                   going on here, but someone else has
>                   reported the same issue.
>
>                   Thanks,
>                   Michael
>
>                   On Wed, Dec 29, 2010 at 6:44 AM, Shige
>                   Song <shigesong at gmail.com> wrote:
>
>                         Dear All,
>
>                         I am trying to
>                         compile&install the package
>                         "RGtk2" on my Ubuntu 10.04
>                         box. I did not have problem
>                         with earlier versions, but
>                         with the new
>                         version, I got the following
>                         error message :
> 
> 
> ...
> 
> --
> Brian D. Ripley,                  ripley at stats.ox.ac.uk
> Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
> University of Oxford,             Tel:  +44 1865 272861 (self)
> 1 South Parks Road,                     +44 1865 272866 (PA)
> Oxford OX1 3TG, UK                Fax:  +44 1865 272595
> 
> 
> 
>

-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595