[Bioc-devel] Updates to BiocFileCache, AnnotationHub, and ExperimentHub
Vincent Carey
@tvjc @end|ng |rom ch@nn|ng@h@rv@rd@edu
Wed Apr 7 18:23:50 CEST 2021
The experience:
> ah = AnnotationHub()
Error in AnnotationHub() :
As of AnnotationHub (>2.23.2), default caching location has changed.
Problematic cache: /home/stvjc/.cache/AnnotationHub
To continue with default caching location,
See AnnotationHub vignette TroubleshootingTheCache section on 'Default
Caching Location Update'
Enter a frame number, or 0 to exit
1: AnnotationHub()
Selection: 0
> rappdirs::user_cache_dir(appname="BiocFileCache")
[1] "~/.cache/BiocFileCache"
> rappdirs::user_cache_dir(appname="AnnotationHub")
[1] "~/.cache/AnnotationHub" # solution from vignette: 850MB moved very
quickly...
> olddir <- path.expand(rappdirs::user_cache_dir(appname="AnnotationHub"))
> newdir <- tools::R_user_dir("AnnotationHub", which="cache")
> dir.create(path=newdir, recursive=TRUE)
> files <- list.files(olddir, full.names =TRUE)
> moveres <- vapply(files,
+ FUN=function(fl){
+ filename = basename(fl)
+ newname = file.path(newdir, filename)
+ file.rename(fl, newname)
+ },
+ FUN.VALUE = logical(1))
> if(all(moveres)) unlink(olddir, recursive=TRUE)
> ah = AnnotationHub() # try again
snapshotDate(): 2021-03-15
On Wed, Apr 7, 2021 at 12:03 PM Vincent Carey <stvjc using channing.harvard.edu>
wrote:
> For convenience here are relevant sections of the new vignette. Give it
> a try and let us know. This is "devel".
>
> 4Default Caching Location Update
>
> As of BiocFileCache version > 1.15.1, the default caching location has
> changed. The default cache is now controlled by the function
> tools::R_user_dir instead of rappdirs::user_cache_dir. Users who have
> utilized the default BiocFileCache, to continue using the created cache,
> must move the cache and its files to the new default location or delete the
> old cache and have to redownload any previous files.
> 4.1Option 1: Moving Files
>
> The following steps can be used to move the files to the new location:
>
> 1.
>
> Determine the old location by running the following in R
> rappdirs::user_cache_dir(appname="BiocFileCache")
> 2.
>
> Determine the new location by running the following in R tools::R_user_dir("BiocFileCache",
> which="cache")
> 3.
>
> Move the files to the new location. You can do this manually or do the
> following steps in R. Remember if you have a lot of cached files, this may
> take awhile.
>
> olddir <- path.expand(rappdirs::user_cache_dir(appname="BiocFileCache"))
> newdir <- tools::R_user_dir("BiocFileCache", which="cache")
> dir.create(path=newdir, recursive=TRUE)
> files <- list.files(olddir, full.names =TRUE)
> moveres <- vapply(files,
> FUN=function(fl){
> filename = basename(fl)
> newname = file.path(newdir, filename)
> file.rename(fl, newname)
> },
> FUN.VALUE = logical(1))
> if(all(moveres)) unlink(olddir, recursive=TRUE)
>
> 4.2Option 2: Specify a Cache Location Explicitly
>
> Users may always specify a unique caching location by providing the cache argument
> to the BiocFileCache constructor; however users must always specify this
> location as it will not be recognized by default in subsequent runs.
>
> Alternatively, the default caching location may also be controlled by a
> user-wise or system-wide environment variable. Users may set the
> environment variable BFC_CACHE to the old location to continue using as
> default location.
>
> On Wed, Apr 7, 2021 at 11:51 AM Kern, Lori <Lori.Shepherd using roswellpark.org>
> wrote:
>
>> There is no guarantee we would be under the right user to have
>> permissions to move the cache automatically and would not want to leave it
>> in a broken state.
>>
>> We could start a fresh cache in the new location but there would be no
>> way to combined an old cache and a new cache and there would be no way to
>> warn people before starting the new cache to give them an opportunity to
>> move the old cache to the new location.
>>
>> This should not affect any cache that is explicitly stated with a
>> different name in the constructor or using environment variables; only in
>> the case of BiocFileCache() . Most package specific caches created their
>> own cache in the constructor so it should not cause the ERROR in that case.
>>
>>
>> Lori Shepherd
>>
>> Bioconductor Core Team
>>
>> Roswell Park Comprehensive Cancer Center
>>
>> Department of Biostatistics & Bioinformatics
>>
>> Elm & Carlton Streets
>>
>> Buffalo, New York 14263
>>
>> ________________________________
>> From: Aaron Lun <infinite.monkeys.with.keyboards using gmail.com>
>> Sent: Wednesday, April 7, 2021 11:41 AM
>> To: Kern, Lori <Lori.Shepherd using RoswellPark.org>
>> Cc: bioc-devel using r-project.org <bioc-devel using r-project.org>
>> Subject: Re: [Bioc-devel] Updates to BiocFileCache, AnnotationHub, and
>> ExperimentHub
>>
>> Woah, I missed the part where you said that there would be an error.
>>
>> This does not sound good. Users are going to flip out, especially when
>> EHub and AHub are not visible dependencies (e.g., scRNAseq, celldex).
>> It also sounds completely unnecessary for EHub and AHub given that the
>> new cache can just be populated by fresh downloads. Similarly,
>> BiocFileCache::bfcrpath should not be affected, and people using that
>> shouldn't be getting an error.
>>
>> Why not just move the old default cache into the new location
>> automatically? This seems like the simplest solution given that
>> everyone accessing BFC resources should be doing so through the BFC
>> API. And most files are not position-dependent, unless people are
>> putting shared libraries in there.
>>
>> But even if you can't, an error is just too much. We use BiocFileCache
>> a lot in our company infrastructure and the brown stuff will hit the
>> fan if we have to find every old default cache and delete it. The
>> package should handle this for us.
>>
>> -A
>>
>> On Wed, Apr 7, 2021 at 4:46 AM Kern, Lori <Lori.Shepherd using roswellpark.org>
>> wrote:
>> >
>> > Mostly to lighten the dependency tree using tools that is built in with
>> R would remove one additional dependency. Also clarity; the tools
>> directory adds an R folder for distinction that they are used with R
>> packages which seemed like if a user was ever investigating, they would
>> have a better idea where those files came from.
>> >
>> >
>> > Lori Shepherd
>> >
>> > Bioconductor Core Team
>> >
>> > Roswell Park Comprehensive Cancer Center
>> >
>> > Department of Biostatistics & Bioinformatics
>> >
>> > Elm & Carlton Streets
>> >
>> > Buffalo, New York 14263
>> >
>> > ________________________________
>> > From: Bioc-devel <bioc-devel-bounces using r-project.org> on behalf of Aaron
>> Lun <infinite.monkeys.with.keyboards using gmail.com>
>> > Sent: Wednesday, April 7, 2021 4:10 AM
>> > To: bioc-devel using r-project.org <bioc-devel using r-project.org>
>> > Subject: Re: [Bioc-devel] Updates to BiocFileCache, AnnotationHub, and
>> ExperimentHub
>> >
>> > rebook and basilisk are also currently using rappdirs. I would be
>> > interested in the motivation behind the switch for the Hubs and whether
>> > that is applicable to those two packages as well.
>> >
>> > -A
>> >
>> > On 4/5/21 6:41 AM, Kern, Lori wrote:
>> > > We are in process of making some major updates to the caching in
>> BiocFileCache, AnnotationHub, and ExperimentHub. Namely, the default
>> caching location will change from using rappdirs::user_cache_dir to
>> using tools::R_user_dir eventually relieving the dependency on rappdirs.
>> To avoid conflicting default caches, if anyone used an old default caching
>> directory, there will be an error to decide how to deal with the old
>> location before proceeding and documentation in the vignettes for how to
>> resolve. Currently I have update BiocFileCache, the changes were just
>> pushed to the devel branch and should propagate tonight. I plan on doing
>> the same for both AnnotationHub and ExperimentHub within the next few
>> days. We appreciate any feedback or questions with regards to these
>> updates.
>> > >
>> > > This is only relevant to using the default cache location, if a user
>> manually specified a unique location, used environment variables, or
>> created a package specific cache the code/location is not affected. Anyone
>> using package specific caching that utilizes rappdirs is encouraged also to
>> consider changing package code to use the now available function in tools.
>> > >
>> > > Cheers,
>> > >
>> > >
>> > > Lori Shepherd
>> > >
>> > > Bioconductor Core Team
>> > >
>> > > Roswell Park Comprehensive Cancer Center
>> > >
>> > > Department of Biostatistics & Bioinformatics
>> > >
>> > > Elm & Carlton Streets
>> > >
>> > > Buffalo, New York 14263
>> > >
>> > >
>> > > This email message may contain legally privileged and/or confidential
>> information. If you are not the intended recipient(s), or the employee or
>> agent responsible for the delivery of this message to the intended
>> recipient(s), you are hereby notified that any disclosure, copying,
>> distribution, or use of this email message is prohibited. If you have
>> received this message in error, please notify the sender immediately by
>> e-mail and delete this email message from your computer. Thank you.
>> > > [[alternative HTML version deleted]]
>> > >
>> > > _______________________________________________
>> > > Bioc-devel using r-project.org mailing list
>> > >
>> https://secure-web.cisco.com/1xDKtwmD7KF6XN2PGsO5WMb8MUBuVd1_7pAaiY6x4Jc1PgpDH8GiGacn-IzghnM40n60o567y87UdWrsqKF1LJRwGTwXOP598j5D7E6SPBKpWfVoTsB78DxjOEhJN0-wwvxq-4tw3xW2W3I1DpwxlybPtN8tG2zlJiSHn9OBn0hHbo8lo81oVIOopo4mAa_ui2PcrX6zQPeh322U9G813vyhB3BsSA7YM-V70ahEdb6JUf-XZBqeY_mSavleRlfFk4-ROT8ng5D16P9XlU-qk1i64Gjc_Q0dExIV1WKOh5Sr7c05LH0vQ6eRG-IHZTN7O/https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fbioc-devel
>> > >
>> >
>> > _______________________________________________
>> > Bioc-devel using r-project.org mailing list
>> >
>> https://secure-web.cisco.com/1xDKtwmD7KF6XN2PGsO5WMb8MUBuVd1_7pAaiY6x4Jc1PgpDH8GiGacn-IzghnM40n60o567y87UdWrsqKF1LJRwGTwXOP598j5D7E6SPBKpWfVoTsB78DxjOEhJN0-wwvxq-4tw3xW2W3I1DpwxlybPtN8tG2zlJiSHn9OBn0hHbo8lo81oVIOopo4mAa_ui2PcrX6zQPeh322U9G813vyhB3BsSA7YM-V70ahEdb6JUf-XZBqeY_mSavleRlfFk4-ROT8ng5D16P9XlU-qk1i64Gjc_Q0dExIV1WKOh5Sr7c05LH0vQ6eRG-IHZTN7O/https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fbioc-devel
>> >
>> >
>> > This email message may contain legally privileged and/or confidential
>> information. If you are not the intended recipient(s), or the employee or
>> agent responsible for the delivery of this message to the intended
>> recipient(s), you are hereby notified that any disclosure, copying,
>> distribution, or use of this email message is prohibited. If you have
>> received this message in error, please notify the sender immediately by
>> e-mail and delete this email message from your computer. Thank you.
>>
>>
>> This email message may contain legally privileged and/or confidential
>> information. If you are not the intended recipient(s), or the employee or
>> agent responsible for the delivery of this message to the intended
>> recipient(s), you are hereby notified that any disclosure, copying,
>> distribution, or use of this email message is prohibited. If you have
>> received this message in error, please notify the sender immediately by
>> e-mail and delete this email message from your computer. Thank you.
>> [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> Bioc-devel using r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>>
>
--
The information in this e-mail is intended only for the ...{{dropped:18}}
More information about the Bioc-devel
mailing list