[Rd] tempdir() may be deleted during long-running R session

[frederik at ofb.net]

Hi Gabriel,

Thanks for asking for a better solution, as far as actually preventing
temporary files from getting deleted in the first place.

I still don't know very much about other peoples' distributions, but
Arch uses Systemd which is the culprit on my system. Systemd's
'tmpfiles.d(5)' man page says we can create configuration files in
locations like

    /usr/lib/tmpfiles.d/*.conf
    /etc/tmpfiles.d/*.conf

which control when temporary files are deleted. There is an 'x'
specifier which accepts glob paths and can protect everything in
/tmp/Rtmp* ...:

    $ mkdir /tmp/Rtmpaoeu
    $ touch -d "12 days ago" /tmp/Rtmpaoeu
    $ sudo systemd-tmpfiles --clean
    $ ls /tmp/Rtmpaoeu
    ls: cannot access '/tmp/Rtmpaoeu': No such file or directory

    $ sudo sh -c "echo 'x /tmp/Rtmp*' > /etc/tmpfiles.d/R.conf"
    $ mkdir /tmp/Rtmpaoeu
    $ touch -d "12 days ago" /tmp/Rtmpaoeu
    $ sudo systemd-tmpfiles --clean
    $ ls /tmp/Rtmpaoeu
    (still there)

I guess installing such a file is something that would be done by the
various distribution-specific R packages. Even though I run R from a
home-directory compiled version, I have my distribution's binary
package installed globally, and so I would get the benefit of this
protection from the distribution package. If this sounds like it makes
sense then I can ask the Arch package maintainer to do it. Of course I
don't need permission but it would be good to hear if I'm missing or
forgetting something.

Based on what other packages are doing the file should probably be
named:

    /usr/lib/tmpfiles.d/R.conf

and contain:

    x /tmp/Rtmp*

(For example on my system I have stuff like this owned by various
packages:

    $ pacman -Qo /usr/lib/tmpfiles.d/*
    /usr/lib/tmpfiles.d/apache.conf is owned by apache 2.4.25-1
    /usr/lib/tmpfiles.d/bind.conf is owned by bind 9.11.0.P3-3
    /usr/lib/tmpfiles.d/colord.conf is owned by colord 1.3.4-1
    /usr/lib/tmpfiles.d/etc.conf is owned by systemd 232-8
    /usr/lib/tmpfiles.d/gvfsd-fuse-tmpfiles.conf is owned by gvfs 1.30.3-1
    ...

)

Thanks!

Frederick

On Tue, Apr 25, 2017 at 09:03:01AM -0700, Gabriel Becker wrote:
> Martin,
> 
> Thanks for your work on this.
> 
> One thing that seems to be missing from the conversation is that recreating
> the temp directory will prevent future failures when R wants to write a
> temp file, but the files will, of course, not be there. Any code written
> assuming the contract is that the temporary directory, and thus temporary
> files, will not be cleaned up before the R process exits (which was my
> naive assumption before this thread, and is the behavior AFAICT on all the
> systems I regularly use) will still break.
> 
> I'm not saying that's necessarily fixable (though the R keeping a permanent
> pointer to a file in the dir suggested by Malcom might? fix it.), but I
> would argue if it IS fixable, a fix that includes that would be preferable.
> 
> Best,
> ~G
> 
> On Tue, Apr 25, 2017 at 8:53 AM, Martin Maechler <maechler at stat.math.ethz.ch
> > wrote:
> 
> > >>>>> Jeroen Ooms <jeroenooms at gmail.com>
> > >>>>>     on Tue, 25 Apr 2017 15:05:51 +0200 writes:
> >
> >     > On Tue, Apr 25, 2017 at 1:00 PM, Martin Maechler
> >     > <maechler at stat.math.ethz.ch> wrote:
> >     >> As I've found it is not at all hard to add an option
> >     >> which checks the existence and if the directory is no
> >     >> longer "valid", tries to recreate it (and if it fails
> >     >> doing that it calls the famous R_Suicide(), as it does
> >     >> when R starts up and tempdir() cannot be initialized
> >     >> correctly).
> >
> >     > Perhaps this can also fix the problem with mcparallel
> >     > deleting the tempdir() when one of its children dies:
> >
> >    >   file.exists(tempdir()) #TRUE
> >    >   parallel::mcparallel(q('no'))
> >    >   file.exists(tempdir()) # FALSE
> >
> > Thank you, Jeroen, for the extra example.
> >
> > I now have comitted the new feature... (completely back
> > compatible: in R's code tempdir() is not yet called with an
> > argument and the default is  check = FALSE ),
> > actually in a "suicide-free" way ...  which needed only slightly
> > more code.
> >
> > In the worst case, one could save the R session by
> >    Sys.setenv(TEMPDIR = "<something writable>")
> > if for instance /tmp/ suddenly became unwritable for the user.
> >
> > What we could consider is making the default of 'check' settable
> > by an option, and experiment with setting the option to TRUE, so
> > all such problems would be auto-solved (says the incurable optimist ...).
> >
> > Martin
> >
> > ______________________________________________
> > R-devel at r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel
> >
> 
> 
> 
> -- 
> Gabriel Becker, PhD
> Associate Scientist (Bioinformatics)
> Genentech Research
> 
> 	[[alternative HTML version deleted]]
> 
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>