[Bioc-devel] "length(url) == 1 is not TRUE" error for vignette only on build system
Martin Morgan
mtmorg@n@b|oc @end|ng |rom gm@||@com
Sat Sep 12 00:22:03 CEST 2020
bfcquery() just searches the cache, so if you've created two resources that match the name then you end up with two rids
> xx = bfcnew(bfc, "foo")
> bfcquery(bfc, "foo", "rname")
# A tibble: 1 x 10
rid rname create_time access_time rpath rtype fpath last_modified_t… etag
<chr> <chr> <chr> <chr> <chr> <chr> <chr> <dbl> <chr>
1 BFC3 foo 2020-09-11… 2020-09-11… /Use… rela… 64ea… NA NA
# … with 1 more variable: expires <dbl>
> xx = bfcnew(bfc, "foo_bar")
> bfcquery(bfc, "foo", "rname")
# A tibble: 2 x 10
rid rname create_time access_time rpath rtype fpath last_modified_t… etag
<chr> <chr> <chr> <chr> <chr> <chr> <chr> <dbl> <chr>
1 BFC3 foo 2020-09-11… 2020-09-11… /Use… rela… 64ea… NA NA
2 BFC4 foo_… 2020-09-11… 2020-09-11… /Use… rela… 64ea… NA NA
# … with 1 more variable: expires <dbl>
and even
> xx = bfcnew(bfc, "foo_bar")
> bfcquery(bfc, "foo", "rname")
# A tibble: 3 x 10
rid rname create_time access_time rpath rtype fpath last_modified_t… etag
<chr> <chr> <chr> <chr> <chr> <chr> <chr> <dbl> <chr>
1 BFC3 foo 2020-09-11… 2020-09-11… /Use… rela… 64ea… NA NA
2 BFC4 foo_… 2020-09-11… 2020-09-11… /Use… rela… 64ea… NA NA
3 BFC5 foo_… 2020-09-11… 2020-09-11… /Use… rela… 64ea… NA NA
# … with 1 more variable: expires <dbl>
If the cache is under netDx control, then you can be careful about creating new resources by checking first whether the resource exists, as outlined in http://bioconductor.org/packages/devel/bioc/vignettes/BiocFileCache/inst/doc/BiocFileCache.html#cache-to-manage-package-data
If one is managing internet resources, just use bfcadd
> bfc <- BiocFileCache()
> path <- bfcrpath(bfc, url)
adding rname 'https://bioconductor.org/index.html'
|======================================================================| 100%
> path <- bfcrpath(bfc, url)
>
where the first time 'url' is used (as a unique key) the resource is downloaded to the cache; the second time it is simply accessed from the cache.
It might be necessary to manually 'clean up' the builders
Martin
On 9/11/20, 4:10 PM, "Bioc-devel on behalf of Pages, Herve" <bioc-devel-bounces using r-project.org on behalf of hpages using fredhutch.org> wrote:
I'm not a BiocFileCache expert, sorry. You will probably get better help
by opening a BiocFileCache issue on GitHub.
Cheers,
H.
On 9/11/20 12:00, Shraddha Pai wrote:
> Hi Herve,
> Sorry - hit shortcut to "send" by mistake.
>
> This is the code snippet that got cut off:
> if (length(rid)>1) {
> x <- as.POSIXct(rid_rec$last_modified_time)
> y <- order(x,decreasing=TRUE)
> rid <- rid[y[1]]
> }
>
> Overkill? I could just take rid[1] but that would not be methodical.
>
> Will file issue once you confirm that the problem is that there should
> be a unique rid associated with each rname.
>
> Thanks again - appreciate it,
> Shraddha
>
> On Fri, Sep 11, 2020 at 2:57 PM Shraddha Pai <shraddha.pai using utoronto.ca
> <mailto:shraddha.pai using utoronto.ca>> wrote:
>
> Hi Herve,
> Thank you for pinpointing this.
> Just so I'm clear: rid should not have two IDs correct?
> For each rname, the cache should only have a unique ID, right? Or is
> that not the case?
>
> To get the package to build I will take the rid with the latest
> "last_modified_time".
> Looks like this can be achieved with:
> if (length(rid)>1) {
> x <- rid_rec$last_modified
>
> On Fri, Sep 11, 2020 at 1:22 PM Pages, Herve <hpages using fredhutch.org
> <mailto:hpages using fredhutch.org>> wrote:
>
> Hi Shraddha,
>
> Seems to be a BiocFileCache issue.
>
> On my laptop the following code (taken from your
> Predict_CaseControl_from_CNV.Rnw vignette):
>
> require(BiocFileCache)
> geneURL <- paste("http://download.baderlab.org/netDx/
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__download.baderlab.org_netDx_&d=DwMFaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=V-k2jGkEksU1mmcALiO9azNiQrGrVrfwlL4vMvJZwHQ&s=DG1UzWYSS8kbCIEYszKms6FBBQ-GZKWplwuOOOIf88c&e=>",
> "supporting_data/refGene.hg18.bed",sep="")
> cache <- rappdirs::user_cache_dir(appname = "netDx")
> bfc <- BiocFileCache::BiocFileCache(cache,ask=FALSE)
> rid_rec <- bfcquery(bfc, "hg18_genes", "rname")
> rid <- rid_rec$rid
>
> returns only one rid:
>
> > rid
> [1] "BFC4"
>
> whereas on malbec1 it returns two:
>
> > rid
> [1] "BFC6" "BFC11"
>
> In both cases, I'm using the latest version of BiocFileCache
> (1.13.1).
>
> I suspect this has to do with the cache being in different
> states on the
> two machines.
>
> The fact that 'rid' has a length of 2 causes the failure a few
> lines
> later when bfcdownload() is called:
>
> rid <- rid_rec$rid
> if (!length(rid)) {
> rid <- names(bfcadd(bfc, "hg18_genes", geneURL))
> }
> if (!isFALSE(bfcneedsupdate(bfc, rid))){
> bfcdownload(bfc, rid,ask=FALSE)
> }
>
> I suggest that you open a BiocFileCache issue on GitHub.
>
> Thanks,
> H.
>
>
> On 9/11/20 08:51, Shraddha Pai wrote:
> > Hello BioC-devel team,
> > Any idea why netDx 1.1.4 fails on the bioc devel build system?
> >
> > This is the error:
> >
> > --- re-building 'Predict_CaseControl_from_CNV.Rmd' using knitr
> > Quitting from lines 173-195 (Predict_CaseControl_from_CNV.Rmd)
> > Error: processing vignette 'Predict_CaseControl_from_CNV.Rmd'
> failed
> > with diagnostics:
> > length(url) == 1 is not TRUE
> > --- failed re-building 'Predict_CaseControl_from_CNV.Rmd'
> >
> >
> > I just set up the Bioc-devel docker, cloned the repo using
> > git clone git using git.bioconductor.org:packages/netDx
> > and ran the vignette, and it runs through successfully.
> >
> > However the build error occurs across all platforms so
> something needs to
> > change.
> >
> > Thank you,
> > Shraddha
> >
> > [[alternative HTML version deleted]]
> >
> > _______________________________________________
> > Bioc-devel using r-project.org <mailto:Bioc-devel using r-project.org>
> mailing list
> >
> https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_bioc-2Ddevel&d=DwICAg&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=zwoVT5i-NLz7FqpRg0pSLIqNJBJiI_Ac8FigI43cwRc&s=YhavBW80Elhyr-R7RuyrA2bwYPfI08HAsraBEs2FRU0&e=
> >
>
> --
> Hervé Pagès
>
> Program in Computational Biology
> Division of Public Health Sciences
> Fred Hutchinson Cancer Research Center
> 1100 Fairview Ave. N, M1-B514
> P.O. Box 19024
> Seattle, WA 98109-1024
>
> E-mail: hpages using fredhutch.org <mailto:hpages using fredhutch.org>
> Phone: (206) 667-5791
> Fax: (206) 667-1319
>
--
Hervé Pagès
Program in Computational Biology
Division of Public Health Sciences
Fred Hutchinson Cancer Research Center
1100 Fairview Ave. N, M1-B514
P.O. Box 19024
Seattle, WA 98109-1024
E-mail: hpages using fredhutch.org
Phone: (206) 667-5791
Fax: (206) 667-1319
_______________________________________________
Bioc-devel using r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel
More information about the Bioc-devel
mailing list