[Bioc-devel] "length(url) == 1 is not TRUE" error for vignette only on build system
Shepherd, Lori
Lor|@Shepherd @end|ng |rom Ro@we||P@rk@org
Mon Sep 14 18:20:56 CEST 2020
To follow up on this:
Please check the package code for adding/accessing resources in the netDx package. If the BiocFileCache options were used correctly there should not have been multiple entries added to the builders cache as Martin demonstrated below.
Once the package has been updated to ensure duplicate entries will not be added, we can clean up the builders. The package code should be adjusted first to ensure that users will not encounter the same situation.
Lori Shepherd
Bioconductor Core Team
Roswell Park Comprehensive Cancer Center
Department of Biostatistics & Bioinformatics
Elm & Carlton Streets
Buffalo, New York 14263
________________________________
From: Bioc-devel <bioc-devel-bounces using r-project.org> on behalf of Martin Morgan <mtmorgan.bioc using gmail.com>
Sent: Friday, September 11, 2020 6:22 PM
To: Pages, Herve <hpages using fredhutch.org>; Shraddha Pai <shraddha.pai using utoronto.ca>
Cc: bioc-devel using r-project.org <Bioc-devel using r-project.org>
Subject: Re: [Bioc-devel] "length(url) == 1 is not TRUE" error for vignette only on build system
bfcquery() just searches the cache, so if you've created two resources that match the name then you end up with two rids
> xx = bfcnew(bfc, "foo")
> bfcquery(bfc, "foo", "rname")
# A tibble: 1 x 10
rid rname create_time access_time rpath rtype fpath last_modified_t� etag
<chr> <chr> <chr> <chr> <chr> <chr> <chr> <dbl> <chr>
1 BFC3 foo 2020-09-11� 2020-09-11� /Use� rela� 64ea� NA NA
# � with 1 more variable: expires <dbl>
> xx = bfcnew(bfc, "foo_bar")
> bfcquery(bfc, "foo", "rname")
# A tibble: 2 x 10
rid rname create_time access_time rpath rtype fpath last_modified_t� etag
<chr> <chr> <chr> <chr> <chr> <chr> <chr> <dbl> <chr>
1 BFC3 foo 2020-09-11� 2020-09-11� /Use� rela� 64ea� NA NA
2 BFC4 foo_� 2020-09-11� 2020-09-11� /Use� rela� 64ea� NA NA
# � with 1 more variable: expires <dbl>
and even
> xx = bfcnew(bfc, "foo_bar")
> bfcquery(bfc, "foo", "rname")
# A tibble: 3 x 10
rid rname create_time access_time rpath rtype fpath last_modified_t� etag
<chr> <chr> <chr> <chr> <chr> <chr> <chr> <dbl> <chr>
1 BFC3 foo 2020-09-11� 2020-09-11� /Use� rela� 64ea� NA NA
2 BFC4 foo_� 2020-09-11� 2020-09-11� /Use� rela� 64ea� NA NA
3 BFC5 foo_� 2020-09-11� 2020-09-11� /Use� rela� 64ea� NA NA
# � with 1 more variable: expires <dbl>
If the cache is under netDx control, then you can be careful about creating new resources by checking first whether the resource exists, as outlined in http://secure-web.cisco.com/1t9-aFwD7wK-0S-MuqPB5uIzT5hQ2gyAD8KMGooS3HThd0k45RZ-frPKl-6Yl2EwZFFcmutTlgEODe0vTjmGHdvrtnhcf5beUQTSS3KiUAl4eWVWv4m3kjsTCUxTVQ2Ll5cGeus1QPlv2osdgHIycOGo6WKDjPqb6PoCxqWOMpEB_x8cSenUxF2N0OkDSEKuwlgJzHwqGjVOwqZQ_odXuqKEnKXz5moLlJO0XSz5XXkTj4kTGwtMAjqKAk_tR8DujPGfYr6Cc70ReKMZUWNVIOc4JgCOAmkILBW94Rp0x_3CX-8DEDeaRhlrgoM7MxkqHGyB0UxHyCXGepxXY196Uglz7WmH1uY02-XQsQ_6e8og/http%3A%2F%2Fbioconductor.org%2Fpackages%2Fdevel%2Fbioc%2Fvignettes%2FBiocFileCache%2Finst%2Fdoc%2FBiocFileCache.html%23cache-to-manage-package-data
If one is managing internet resources, just use bfcadd
> bfc <- BiocFileCache()
> path <- bfcrpath(bfc, url)
adding rname 'https://secure-web.cisco.com/1lUwUbgjSoXsv6f7JfdXhRG2f7_ISpNGSvQkNDjD3wQSEEMkTAVHfopgsowEo1GgsXrbByzyoKcAqYa3Kw2bxocuFRTwCIF5UxwWY7fw_FCeNByJcX4L4A9pGyPD1Il4OWJRoYewqI9SOxbFsK6XqqYQmOxhzALdAwDrJw1cOOpppzIIoz5unFJOO2Ihuca8QuIswQZGQqxSBXbeuu8WxSl9QoenolmV7PHovZu_sUMd1DYSflBtliuDz8hRJHSdp-N4wLsvl5GZXxTcJH4ZruhecvOJPR7aGrHCRLZln3wWoMKctL_Oc2xnUYu0ZYrCrVD5MQ8Wf7fgFEl7RPTt5RbTMxrbLO0zXxZZaEwhL5AU/https%3A%2F%2Fbioconductor.org%2Findex.html'
|======================================================================| 100%
> path <- bfcrpath(bfc, url)
>
where the first time 'url' is used (as a unique key) the resource is downloaded to the cache; the second time it is simply accessed from the cache.
It might be necessary to manually 'clean up' the builders
Martin
On 9/11/20, 4:10 PM, "Bioc-devel on behalf of Pages, Herve" <bioc-devel-bounces using r-project.org on behalf of hpages using fredhutch.org> wrote:
I'm not a BiocFileCache expert, sorry. You will probably get better help
by opening a BiocFileCache issue on GitHub.
Cheers,
H.
On 9/11/20 12:00, Shraddha Pai wrote:
> Hi Herve,
> Sorry - hit shortcut to "send" by mistake.
>
> This is the code snippet that got cut off:
> if (length(rid)>1) {
> x <- as.POSIXct(rid_rec$last_modified_time)
> y <- order(x,decreasing=TRUE)
> rid <- rid[y[1]]
> }
>
> Overkill? I could just take rid[1] but that would not be methodical.
>
> Will file issue once you confirm that the problem is that there should
> be a unique rid associated with each rname.
>
> Thanks again - appreciate it,
> Shraddha
>
> On Fri, Sep 11, 2020 at 2:57 PM Shraddha Pai <shraddha.pai using utoronto.ca
> <mailto:shraddha.pai using utoronto.ca>> wrote:
>
> Hi Herve,
> Thank you for pinpointing this.
> Just so I'm clear: rid should not have two IDs correct?
> For each rname, the cache should only have a unique ID, right? Or is
> that not the case?
>
> To get the package to build I will take the rid with the latest
> "last_modified_time".
> Looks like this can be achieved with:
> if (length(rid)>1) {
> x <- rid_rec$last_modified
>
> On Fri, Sep 11, 2020 at 1:22 PM Pages, Herve <hpages using fredhutch.org
> <mailto:hpages using fredhutch.org>> wrote:
>
> Hi Shraddha,
>
> Seems to be a BiocFileCache issue.
>
> On my laptop the following code (taken from your
> Predict_CaseControl_from_CNV.Rnw vignette):
>
> require(BiocFileCache)
> geneURL <- paste("http://secure-web.cisco.com/1q6dletAEEmECRlpAh7w_w4AmdNAhzos1ijnImWIdzsuXHz3kMIav7Urj3RzYO0S8BzLMKG_0Zcp4ytMlPfA9Ht6IFyJIoCQY-9mRDq-BQDDX_zer8B-bkuIy0hvpOFRkd4a0lSKnviwzssrArVnDcHSCh7EQZsCSMgdEeFsojc3WO3wt8cG5GJztsLKILl9NFGnr2g19zjUcuvqtOBAyec62CVmCfonqPnOWy_VEUPd4WcDd2XwFUW7vEzrFcpC3V6K5pAc0bWjjqEz-3RLU0cWFYgNOGa7zIBwHCX8XMu1JcpLCVoXB-W_RXWMiOwtaHwy8vGei3fBckztfRFgZnA/http%3A%2F%2Fdownload.baderlab.org%2FnetDx%2F
> <https://secure-web.cisco.com/1pBAaf568Sh8T9WwtZnai1icCdNVYUY8FRTQfjYtlK3gI5radqfCsKfzfJz-0BFk_9sTpehBBo60575Jx0MEnZeJRJIKZxiwx2eQ0X90O_CRwVziGst9BfByxNmW5LhvoYTWWSYm1G7ILPw13lka_Q4K2MiNJUX9s0gU6JDcCjBZ6000t2wqGJ_YEBcAGdt30Uv9ElTiYcAXps8tQzLNc9DGwx8ErTvnVmMjvbBVbBRQEBe3Gvo7OcE4-TS_nIEzsZhyvsT1V8xJ3KU9bTb_0J2S1AWC3ETe48ipc0WcJ_cA0OkCYMR7v5dIhE58VGsox/https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttp-3A__download.baderlab.org_netDx_%26d%3DDwMFaQ%26c%3DeRAMFD45gAfqt84VtBcfhQ%26r%3DBK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA%26m%3DV-k2jGkEksU1mmcALiO9azNiQrGrVrfwlL4vMvJZwHQ%26s%3DDG1UzWYSS8kbCIEYszKms6FBBQ-GZKWplwuOOOIf88c%26e%3D>",
> "supporting_data/refGene.hg18.bed",sep="")
> cache <- rappdirs::user_cache_dir(appname = "netDx")
> bfc <- BiocFileCache::BiocFileCache(cache,ask=FALSE)
> rid_rec <- bfcquery(bfc, "hg18_genes", "rname")
> rid <- rid_rec$rid
>
> returns only one rid:
>
> > rid
> [1] "BFC4"
>
> whereas on malbec1 it returns two:
>
> > rid
> [1] "BFC6" "BFC11"
>
> In both cases, I'm using the latest version of BiocFileCache
> (1.13.1).
>
> I suspect this has to do with the cache being in different
> states on the
> two machines.
>
> The fact that 'rid' has a length of 2 causes the failure a few
> lines
> later when bfcdownload() is called:
>
> rid <- rid_rec$rid
> if (!length(rid)) {
> rid <- names(bfcadd(bfc, "hg18_genes", geneURL))
> }
> if (!isFALSE(bfcneedsupdate(bfc, rid))){
> bfcdownload(bfc, rid,ask=FALSE)
> }
>
> I suggest that you open a BiocFileCache issue on GitHub.
>
> Thanks,
> H.
>
>
> On 9/11/20 08:51, Shraddha Pai wrote:
> > Hello BioC-devel team,
> > Any idea why netDx 1.1.4 fails on the bioc devel build system?
> >
> > This is the error:
> >
> > --- re-building 'Predict_CaseControl_from_CNV.Rmd' using knitr
> > Quitting from lines 173-195 (Predict_CaseControl_from_CNV.Rmd)
> > Error: processing vignette 'Predict_CaseControl_from_CNV.Rmd'
> failed
> > with diagnostics:
> > length(url) == 1 is not TRUE
> > --- failed re-building 'Predict_CaseControl_from_CNV.Rmd'
> >
> >
> > I just set up the Bioc-devel docker, cloned the repo using
> > git clone git using git.bioconductor.org:packages/netDx
> > and ran the vignette, and it runs through successfully.
> >
> > However the build error occurs across all platforms so
> something needs to
> > change.
> >
> > Thank you,
> > Shraddha
> >
> > [[alternative HTML version deleted]]
> >
> > _______________________________________________
> > Bioc-devel using r-project.org <mailto:Bioc-devel using r-project.org>
> mailing list
> >
> https://secure-web.cisco.com/1Q1gRqlh1AbjlYoYoW-Tao_IKz59ZWyREYyVSz_yAJWP75JqkTLfdvqwGnEwk-lQEa0GIswGLe7jTmscS0aE9zb844BYJvAdTWLUneOW1-5ZUjtTOWpbCs88HMYRlKLIrA9cP4HevAvVxueol9bqn0j0LHw-UZX-k72wAWe5tjjd54VbNWOM5kKAPNuti9OB4nxoSRxmrVXUKKx2oQB0wT_yROTrgBTc1EAjjJ1P3fI85SfYyYA2SwxLeltUGcEkWoxzDXL1y7atMucMCnMJ2fv8KN41qlhh0uuv8VaHXyIXcxcg76La78NT4MVLsfX2D/https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__stat.ethz.ch_mailman_listinfo_bioc-2Ddevel%26d%3DDwICAg%26c%3DeRAMFD45gAfqt84VtBcfhQ%26r%3DBK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA%26m%3DzwoVT5i-NLz7FqpRg0pSLIqNJBJiI_Ac8FigI43cwRc%26s%3DYhavBW80Elhyr-R7RuyrA2bwYPfI08HAsraBEs2FRU0%26e%3D
> >
>
> --
> Herv� Pag�s
>
> Program in Computational Biology
> Division of Public Health Sciences
> Fred Hutchinson Cancer Research Center
> 1100 Fairview Ave. N, M1-B514
> P.O. Box 19024
> Seattle, WA 98109-1024
>
> E-mail: hpages using fredhutch.org <mailto:hpages using fredhutch.org>
> Phone: (206) 667-5791
> Fax: (206) 667-1319
>
--
Herv� Pag�s
Program in Computational Biology
Division of Public Health Sciences
Fred Hutchinson Cancer Research Center
1100 Fairview Ave. N, M1-B514
P.O. Box 19024
Seattle, WA 98109-1024
E-mail: hpages using fredhutch.org
Phone: (206) 667-5791
Fax: (206) 667-1319
_______________________________________________
Bioc-devel using r-project.org mailing list
https://secure-web.cisco.com/1e58U71LbG0FMxJdQhxAXm6tEq4DjFc60_hb90-31Y7uIV0IbSMcYaN13j95FzoALCkV6H1go-g1bCAFCzKjDJtyWH1ZTXg2zMKy1xg74eX9mvyC7enPrBgvEDMpkrKaaUuqVPrSkFhAO4BikyNDvl6a_whzJzZXuRi2q5nKEr1JFdIv2pNXCZnMVgARPrdL8AT7dhjPrEPv1r7zAUJ1FMF7h7UsPWABFV1myjK_Thqh0_oXgySzvERAN7bKu5QStc-kXMzxLnj16qYsIycsLBpUZ4b5xvt2xnQfTcJxYfo_zSC2Pgrw2c3wYXBTi69XCs1jdxZRu7qId_lNuFbZ_tg/https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fbioc-devel
_______________________________________________
Bioc-devel using r-project.org mailing list
https://secure-web.cisco.com/1e58U71LbG0FMxJdQhxAXm6tEq4DjFc60_hb90-31Y7uIV0IbSMcYaN13j95FzoALCkV6H1go-g1bCAFCzKjDJtyWH1ZTXg2zMKy1xg74eX9mvyC7enPrBgvEDMpkrKaaUuqVPrSkFhAO4BikyNDvl6a_whzJzZXuRi2q5nKEr1JFdIv2pNXCZnMVgARPrdL8AT7dhjPrEPv1r7zAUJ1FMF7h7UsPWABFV1myjK_Thqh0_oXgySzvERAN7bKu5QStc-kXMzxLnj16qYsIycsLBpUZ4b5xvt2xnQfTcJxYfo_zSC2Pgrw2c3wYXBTi69XCs1jdxZRu7qId_lNuFbZ_tg/https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fbioc-devel
This email message may contain legally privileged and/or confidential information. If you are not the intended recipient(s), or the employee or agent responsible for the delivery of this message to the intended recipient(s), you are hereby notified that any disclosure, copying, distribution, or use of this email message is prohibited. If you have received this message in error, please notify the sender immediately by e-mail and delete this email message from your computer. Thank you.
[[alternative HTML version deleted]]
More information about the Bioc-devel
mailing list