[Bioc-devel] Updates to BiocFileCache, AnnotationHub, and ExperimentHub
Vincent Carey
@tvjc @end|ng |rom ch@nn|ng@h@rv@rd@edu
Wed Apr 7 18:03:52 CEST 2021
For convenience here are relevant sections of the new vignette. Give it a
try and let us know. This is "devel".
4Default Caching Location Update
As of BiocFileCache version > 1.15.1, the default caching location has
changed. The default cache is now controlled by the function
tools::R_user_dir instead of rappdirs::user_cache_dir. Users who have
utilized the default BiocFileCache, to continue using the created cache,
must move the cache and its files to the new default location or delete the
old cache and have to redownload any previous files.
4.1Option 1: Moving Files
The following steps can be used to move the files to the new location:
1.
Determine the old location by running the following in R
rappdirs::user_cache_dir(appname="BiocFileCache")
2.
Determine the new location by running the following in R
tools::R_user_dir("BiocFileCache",
which="cache")
3.
Move the files to the new location. You can do this manually or do the
following steps in R. Remember if you have a lot of cached files, this may
take awhile.
olddir <- path.expand(rappdirs::user_cache_dir(appname="BiocFileCache"))
newdir <- tools::R_user_dir("BiocFileCache", which="cache")
dir.create(path=newdir, recursive=TRUE)
files <- list.files(olddir, full.names =TRUE)
moveres <- vapply(files,
FUN=function(fl){
filename = basename(fl)
newname = file.path(newdir, filename)
file.rename(fl, newname)
},
FUN.VALUE = logical(1))
if(all(moveres)) unlink(olddir, recursive=TRUE)
4.2Option 2: Specify a Cache Location Explicitly
Users may always specify a unique caching location by providing the
cache argument
to the BiocFileCache constructor; however users must always specify this
location as it will not be recognized by default in subsequent runs.
Alternatively, the default caching location may also be controlled by a
user-wise or system-wide environment variable. Users may set the
environment variable BFC_CACHE to the old location to continue using as
default location.
On Wed, Apr 7, 2021 at 11:51 AM Kern, Lori <Lori.Shepherd using roswellpark.org>
wrote:
> There is no guarantee we would be under the right user to have permissions
> to move the cache automatically and would not want to leave it in a broken
> state.
>
> We could start a fresh cache in the new location but there would be no way
> to combined an old cache and a new cache and there would be no way to warn
> people before starting the new cache to give them an opportunity to move
> the old cache to the new location.
>
> This should not affect any cache that is explicitly stated with a
> different name in the constructor or using environment variables; only in
> the case of BiocFileCache() . Most package specific caches created their
> own cache in the constructor so it should not cause the ERROR in that case.
>
>
> Lori Shepherd
>
> Bioconductor Core Team
>
> Roswell Park Comprehensive Cancer Center
>
> Department of Biostatistics & Bioinformatics
>
> Elm & Carlton Streets
>
> Buffalo, New York 14263
>
> ________________________________
> From: Aaron Lun <infinite.monkeys.with.keyboards using gmail.com>
> Sent: Wednesday, April 7, 2021 11:41 AM
> To: Kern, Lori <Lori.Shepherd using RoswellPark.org>
> Cc: bioc-devel using r-project.org <bioc-devel using r-project.org>
> Subject: Re: [Bioc-devel] Updates to BiocFileCache, AnnotationHub, and
> ExperimentHub
>
> Woah, I missed the part where you said that there would be an error.
>
> This does not sound good. Users are going to flip out, especially when
> EHub and AHub are not visible dependencies (e.g., scRNAseq, celldex).
> It also sounds completely unnecessary for EHub and AHub given that the
> new cache can just be populated by fresh downloads. Similarly,
> BiocFileCache::bfcrpath should not be affected, and people using that
> shouldn't be getting an error.
>
> Why not just move the old default cache into the new location
> automatically? This seems like the simplest solution given that
> everyone accessing BFC resources should be doing so through the BFC
> API. And most files are not position-dependent, unless people are
> putting shared libraries in there.
>
> But even if you can't, an error is just too much. We use BiocFileCache
> a lot in our company infrastructure and the brown stuff will hit the
> fan if we have to find every old default cache and delete it. The
> package should handle this for us.
>
> -A
>
> On Wed, Apr 7, 2021 at 4:46 AM Kern, Lori <Lori.Shepherd using roswellpark.org>
> wrote:
> >
> > Mostly to lighten the dependency tree using tools that is built in with
> R would remove one additional dependency. Also clarity; the tools
> directory adds an R folder for distinction that they are used with R
> packages which seemed like if a user was ever investigating, they would
> have a better idea where those files came from.
> >
> >
> > Lori Shepherd
> >
> > Bioconductor Core Team
> >
> > Roswell Park Comprehensive Cancer Center
> >
> > Department of Biostatistics & Bioinformatics
> >
> > Elm & Carlton Streets
> >
> > Buffalo, New York 14263
> >
> > ________________________________
> > From: Bioc-devel <bioc-devel-bounces using r-project.org> on behalf of Aaron
> Lun <infinite.monkeys.with.keyboards using gmail.com>
> > Sent: Wednesday, April 7, 2021 4:10 AM
> > To: bioc-devel using r-project.org <bioc-devel using r-project.org>
> > Subject: Re: [Bioc-devel] Updates to BiocFileCache, AnnotationHub, and
> ExperimentHub
> >
> > rebook and basilisk are also currently using rappdirs. I would be
> > interested in the motivation behind the switch for the Hubs and whether
> > that is applicable to those two packages as well.
> >
> > -A
> >
> > On 4/5/21 6:41 AM, Kern, Lori wrote:
> > > We are in process of making some major updates to the caching in
> BiocFileCache, AnnotationHub, and ExperimentHub. Namely, the default
> caching location will change from using rappdirs::user_cache_dir to
> using tools::R_user_dir eventually relieving the dependency on rappdirs.
> To avoid conflicting default caches, if anyone used an old default caching
> directory, there will be an error to decide how to deal with the old
> location before proceeding and documentation in the vignettes for how to
> resolve. Currently I have update BiocFileCache, the changes were just
> pushed to the devel branch and should propagate tonight. I plan on doing
> the same for both AnnotationHub and ExperimentHub within the next few
> days. We appreciate any feedback or questions with regards to these
> updates.
> > >
> > > This is only relevant to using the default cache location, if a user
> manually specified a unique location, used environment variables, or
> created a package specific cache the code/location is not affected. Anyone
> using package specific caching that utilizes rappdirs is encouraged also to
> consider changing package code to use the now available function in tools.
> > >
> > > Cheers,
> > >
> > >
> > > Lori Shepherd
> > >
> > > Bioconductor Core Team
> > >
> > > Roswell Park Comprehensive Cancer Center
> > >
> > > Department of Biostatistics & Bioinformatics
> > >
> > > Elm & Carlton Streets
> > >
> > > Buffalo, New York 14263
> > >
> > >
> > > This email message may contain legally privileged and/or confidential
> information. If you are not the intended recipient(s), or the employee or
> agent responsible for the delivery of this message to the intended
> recipient(s), you are hereby notified that any disclosure, copying,
> distribution, or use of this email message is prohibited. If you have
> received this message in error, please notify the sender immediately by
> e-mail and delete this email message from your computer. Thank you.
> > > [[alternative HTML version deleted]]
> > >
> > > _______________________________________________
> > > Bioc-devel using r-project.org mailing list
> > >
> https://secure-web.cisco.com/1xDKtwmD7KF6XN2PGsO5WMb8MUBuVd1_7pAaiY6x4Jc1PgpDH8GiGacn-IzghnM40n60o567y87UdWrsqKF1LJRwGTwXOP598j5D7E6SPBKpWfVoTsB78DxjOEhJN0-wwvxq-4tw3xW2W3I1DpwxlybPtN8tG2zlJiSHn9OBn0hHbo8lo81oVIOopo4mAa_ui2PcrX6zQPeh322U9G813vyhB3BsSA7YM-V70ahEdb6JUf-XZBqeY_mSavleRlfFk4-ROT8ng5D16P9XlU-qk1i64Gjc_Q0dExIV1WKOh5Sr7c05LH0vQ6eRG-IHZTN7O/https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fbioc-devel
> > >
> >
> > _______________________________________________
> > Bioc-devel using r-project.org mailing list
> >
> https://secure-web.cisco.com/1xDKtwmD7KF6XN2PGsO5WMb8MUBuVd1_7pAaiY6x4Jc1PgpDH8GiGacn-IzghnM40n60o567y87UdWrsqKF1LJRwGTwXOP598j5D7E6SPBKpWfVoTsB78DxjOEhJN0-wwvxq-4tw3xW2W3I1DpwxlybPtN8tG2zlJiSHn9OBn0hHbo8lo81oVIOopo4mAa_ui2PcrX6zQPeh322U9G813vyhB3BsSA7YM-V70ahEdb6JUf-XZBqeY_mSavleRlfFk4-ROT8ng5D16P9XlU-qk1i64Gjc_Q0dExIV1WKOh5Sr7c05LH0vQ6eRG-IHZTN7O/https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fbioc-devel
> >
> >
> > This email message may contain legally privileged and/or confidential
> information. If you are not the intended recipient(s), or the employee or
> agent responsible for the delivery of this message to the intended
> recipient(s), you are hereby notified that any disclosure, copying,
> distribution, or use of this email message is prohibited. If you have
> received this message in error, please notify the sender immediately by
> e-mail and delete this email message from your computer. Thank you.
>
>
> This email message may contain legally privileged and/or confidential
> information. If you are not the intended recipient(s), or the employee or
> agent responsible for the delivery of this message to the intended
> recipient(s), you are hereby notified that any disclosure, copying,
> distribution, or use of this email message is prohibited. If you have
> received this message in error, please notify the sender immediately by
> e-mail and delete this email message from your computer. Thank you.
> [[alternative HTML version deleted]]
>
> _______________________________________________
> Bioc-devel using r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>
--
The information in this e-mail is intended only for the ...{{dropped:18}}
More information about the Bioc-devel
mailing list