[Bioc-devel] EXTERNAL: Re: SPB inconsistencies
Shepherd, Lori
Lor|@Shepherd @end|ng |rom Ro@we||P@rk@org
Tue Mar 5 13:17:04 CET 2019
I will clear the cache on the build system.
Hopefully with the work I've been doing on converting to BiocFileCache there should be some file level control and re-download in these instances in the very near future but it isn't up yet.
In the current system that is why would we would readd items to the database to get a new AH_id for redownload - There is an option for hub[[AH_ID, force=TRUE]] that is currently implemented that would force redownload but you wouldn't want this flagged all the time as it defeats the purpose of the cache.
Lori Shepherd
Bioconductor Core Team
Roswell Park Cancer Institute
Department of Biostatistics & Bioinformatics
Elm & Carlton Streets
Buffalo, New York 14263
________________________________
From: Bioc-devel <bioc-devel-bounces using r-project.org> on behalf of Ramos Perez, Marcel <Marcel.RamosPerez using RoswellPark.org>
Sent: Monday, March 4, 2019 5:31:56 PM
To: Martin Morgan; bioc-devel using r-project.org
Subject: Re: [Bioc-devel] EXTERNAL: Re: SPB inconsistencies
Ah yes, I overwrite the existing resource.
I will look into a temporary workaround to downloading the newer GISTIC
resource when the cached one is stale.
Best,
Marcel
On 3/4/19 3:50 PM, Martin Morgan wrote:
> Thanks Marcel -- when you update this data, do you add a new resource or over-write an existing resource? I think the best practice would be to create a new resource, with updated curateTCGAData package & version bump. Martin
>
> On 3/4/19, 2:31 PM, "Bioc-devel on behalf of Ramos Perez, Marcel" <bioc-devel-bounces using r-project.org on behalf of Marcel.RamosPerez using RoswellPark.org> wrote:
>
> Hi Martin,
>
> I suspect that this is due to caching because I'm unable to reproduce
> including after deleting my hub cache.
>
> hb <- ExperimentHub::ExperimentHub()
> #> snapshotDate(): 2019-01-25
>
> unlink(
> AnnotationHub::hubCache(hb),
> recursive = TRUE,
> force = TRUE
> )
>
> suppressMessages({
> gbm <- curatedTCGAData::curatedTCGAData("GBM",
> assays=c("GISTIC_Peaks", "CNVSNP", "RNASeq2GeneNorm"),
> dry.run=FALSE)
> })
>
> gbm
> #> A MultiAssayExperiment object of 3 listed
> #> experiments with user-defined names and respective classes.
> #> Containing an ExperimentList class object of length 3:
> #> [1] GBM_CNVSNP-20160128: RaggedExperiment with 146852 rows and 1104 columns
> #> [2] GBM_GISTIC_Peaks-20160128: RangedSummarizedExperiment with 68 rows and 577 columns
> #> [3] GBM_RNASeq2GeneNorm-20160128: SummarizedExperiment with 20501 rows and 166 columns
> #> Features:
> #> experiments() - obtain the ExperimentList instance
> #> colData() - the primary/phenotype DataFrame
> #> sampleMap() - the sample availability DataFrame
> #> `$`, `[`, `[[` - extract colData columns, subset, or experiment
> #> *Format() - convert into a long or wide DataFrame
> #> assays() - convert ExperimentList to a SimpleList of matrices
>
> sessionInfo()
> #> R Under development (unstable) (2019-03-04 r76198)
> #> Platform: x86_64-pc-linux-gnu (64-bit)
> #> Running under: Ubuntu 18.10
> #>
> #> Matrix products: default
> #> BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.8.0
> #> LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.8.0
> #>
> #> locale:
> #> [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
> #> [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
> #> [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
> #> [7] LC_PAPER=en_US.UTF-8 LC_NAME=C
> #> [9] LC_ADDRESS=C LC_TELEPHONE=C
> #> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
> #>
> #> attached base packages:
> #> [1] parallel stats4 stats graphics grDevices utils datasets
> #> [8] methods base
> #>
> #> other attached packages:
> #> [1] RaggedExperiment_1.7.4 curatedTCGAData_1.5.8
> #> [3] MultiAssayExperiment_1.9.15 SummarizedExperiment_1.13.0
> #> [5] DelayedArray_0.9.8 BiocParallel_1.17.17
> #> [7] matrixStats_0.54.0 Biobase_2.43.1
> #> [9] GenomicRanges_1.35.1 GenomeInfoDb_1.19.2
> #> [11] IRanges_2.17.4 S4Vectors_0.21.10
> #> [13] BiocGenerics_0.29.1 nvimcom_0.9-72
> #>
> #> loaded via a namespace (and not attached):
> #> [1] Rcpp_1.0.0 XVector_0.23.0
> #> [3] compiler_3.6.0 BiocManager_1.30.4.9003
> #> [5] later_0.8.0 highr_0.7
> #> [7] AnnotationHub_2.15.7 zlibbioc_1.29.0
> #> [9] bitops_1.0-6 tools_3.6.0
> #> [11] digest_0.6.18 bit_1.1-14
> #> [13] lattice_0.20-38 RSQLite_2.1.1
> #> [15] evaluate_0.13 memoise_1.1.0
> #> [17] pkgconfig_2.0.2 Matrix_1.2-15
> #> [19] shiny_1.2.0 DBI_1.0.0
> #> [21] curl_3.3 yaml_2.2.0
> #> [23] xfun_0.5 GenomeInfoDbData_1.2.0
> #> [25] httr_1.4.0 stringr_1.4.0
> #> [27] knitr_1.21 grid_3.6.0
> #> [29] bit64_0.9-7 R6_2.4.0
> #> [31] AnnotationDbi_1.45.0 rmarkdown_1.11
> #> [33] ExperimentHub_1.9.1 blob_1.1.1
> #> [35] magrittr_1.5 promises_1.0.1
> #> [37] htmltools_0.3.6 mime_0.6
> #> [39] interactiveDisplayBase_1.21.0 xtable_1.8-3
> #> [41] httpuv_1.4.5.1 stringi_1.3.1
> #> [43] RCurl_1.95-4.12
>
> Created on 2019-03-04 by the reprex package<https://reprex.tidyverse.org> (v0.2.1)
>
>
>
> Best,
>
> Marcel
>
>
>
> On 2/28/19 4:28 PM, Martin Morgan wrote:
>
> In today's R-devel, I install packages required for the vignette, including CNVRanger's Suggests: packages, and ensure that my installation is valid (modulo some irrelevant packages) via BiocManager::valid().
>
> I then Stangle'd the vignette
>
> cd CNVRanger/vignettes
> R CMD Stange CNVRanger.Rmd
>
> and then in a new R session sourced the vignette code
>
> source("CNVRanger.R", echo=TRUE, max=Inf)
>
> it runs until
>
>
>
> library(curatedTCGAData)
>
>
>
>
>
> suppressMessages(
>
>
> + gbm <- curatedTCGAData("GBM",
> + assays=c("GISTIC_Peaks", "CNVSNP", "RNASeq2GeneNorm"),
> + dry.run=FALSE)
> + )
> |======================================================================| 100%
>
> |======================================================================| 100%
>
> |======================================================================| 100%
>
>
>
>
> gbm
>
>
> A MultiAssayExperiment object of 2 listed
> experiments with user-defined names and respective classes.
> Containing an ExperimentList class object of length 2:
> [1] GBM_CNVSNP-20160128: RaggedExperiment with 146852 rows and 1104 columns
> [2] GBM_RNASeq2GeneNorm-20160128: SummarizedExperiment with 20501 rows and 166 columns
> Features:
> experiments() - obtain the ExperimentList instance
> colData() - the primary/phenotype DataFrame
> sampleMap() - the sample availability DataFrame
> `$`, `[`, `[[` - extract colData columns, subset, or experiment
> *Format() - convert into a long or wide DataFrame
> assays() - convert ExperimentList to a SimpleList of matrices
>
>
>
> ## ----gisticPeaks---------------------------------------------------------
> peaks <- granges(rowRanges(gbm[["GBM_GISTIC_Peaks-20160128"]]))
>
>
> Error in matrixStats::rowRanges(x, ...) :
> Argument 'x' must be a matrix or a vector.
> In addition: Warning message:
> call dbDisconnect() when finished working with a connection
>
> where the problem is apparent
>
>
>
> gbm[["GBM_GISTIC_Peaks-20160128"]]
>
>
> NULL
>
> The MultiAssayExperiment doesn't contain GBM_GISTIC_Peaks-20160128.
>
> If I try something simpler it also fails...
>
>
>
> curatedTCGAData("GBM", assays = "GISTIC_Peaks", dry.run=FALSE)
>
>
> snapshotDate(): 2019-01-25
> see ?curatedTCGAData and browseVignettes('curatedTCGAData') for documentation
> downloading 0 resources
> loading from cache
> '/Users/ma38727//.ExperimentHub/2143'
> see ?curatedTCGAData and browseVignettes('curatedTCGAData') for documentation
> downloading 0 resources
> loading from cache
> '/Users/ma38727//.ExperimentHub/671'
> see ?curatedTCGAData and browseVignettes('curatedTCGAData') for documentation
> downloading 0 resources
> loading from cache
> '/Users/ma38727//.ExperimentHub/674'
> see ?curatedTCGAData and browseVignettes('curatedTCGAData') for documentation
> downloading 0 resources
> loading from cache
> '/Users/ma38727//.ExperimentHub/685'
> harmonizing input:
> removing 6149 sampleMap rows not in names(experiments)
> removing 599 colData rownames not in sampleMap 'primary'
> A MultiAssayExperiment object of 0 listed
> experiments with no user-defined names and respective classes.
> Containing an ExperimentList class object of length 0:
> Features:
> experiments() - obtain the ExperimentList instance
> colData() - the primary/phenotype DataFrame
> sampleMap() - the sample availability DataFrame
> `$`, `[`, `[[` - extract colData columns, subset, or experiment
> *Format() - convert into a long or wide DataFrame
> assays() - convert ExperimentList to a SimpleList of matrices
>
> So perhaps the problem is with the curated* package?
>
> Here's my sessionInfo() (yikes!)
>
>
>
> sessionInfo()
>
>
> R Under development (unstable) (2019-02-28 r76173)
> Platform: x86_64-apple-darwin17.7.0 (64-bit)
> Running under: macOS High Sierra 10.13.6
>
> Matrix products: default
> BLAS: /Users/ma38727/bin/R-devel/lib/libRblas.dylib
> LAPACK: /Users/ma38727/bin/R-devel/lib/libRlapack.dylib
>
> locale:
> [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
>
> attached base packages:
> [1] parallel stats4 stats graphics grDevices utils datasets
> [8] methods base
>
> other attached packages:
> [1] ensembldb_2.7.9
> [2] AnnotationFilter_1.7.0
> [3] GenomicFeatures_1.35.7
> [4] AnnotationDbi_1.45.0
> [5] TCGAutils_1.3.20
> [6] curatedTCGAData_1.5.8
> [7] MultiAssayExperiment_1.9.15
> [8] SummarizedExperiment_1.13.0
> [9] DelayedArray_0.9.8
> [10] BiocParallel_1.17.17
> [11] matrixStats_0.54.0
> [12] Biobase_2.43.1
> [13] BSgenome.Btaurus.UCSC.bosTau6.masked_1.3.99
> [14] BSgenome.Btaurus.UCSC.bosTau6_1.4.0
> [15] BSgenome_1.51.0
> [16] rtracklayer_1.43.2
> [17] Biostrings_2.51.2
> [18] XVector_0.23.0
> [19] regioneR_1.15.2
> [20] AnnotationHub_2.15.7
> [21] CNVRanger_0.99.18
> [22] RaggedExperiment_1.7.4
> [23] GenomicRanges_1.35.1
> [24] GenomeInfoDb_1.19.2
> [25] IRanges_2.17.4
> [26] S4Vectors_0.21.10
> [27] BiocGenerics_0.29.1
>
> loaded via a namespace (and not attached):
> [1] httr_1.4.0 edgeR_3.25.3
> [3] splines_3.6.0 bit64_0.9-7
> [5] jsonlite_1.6 shiny_1.2.0
> [7] assertthat_0.2.0 statmod_1.4.30
> [9] interactiveDisplayBase_1.21.0 BiocManager_1.30.4
> [11] blob_1.1.1 GenomeInfoDbData_1.2.0
> [13] Rsamtools_1.99.2 yaml_2.2.0
> [15] progress_1.2.0 pillar_1.3.1
> [17] RSQLite_2.1.1 lattice_0.20-38
> [19] limma_3.39.12 glue_1.3.0
> [21] digest_0.6.18 promises_1.0.1
> [23] rvest_0.3.2 htmltools_0.3.6
> [25] httpuv_1.4.5.1 Matrix_1.2-15
> [27] XML_3.98-1.17 pkgconfig_2.0.2
> [29] biomaRt_2.39.3 zlibbioc_1.29.0
> [31] purrr_0.3.0 xtable_1.8-3
> [33] later_0.8.0 tibble_2.0.1
> [35] lazyeval_0.2.1 magrittr_1.5
> [37] crayon_1.3.4 mime_0.6
> [39] memoise_1.1.0 xml2_1.2.0
> [41] tools_3.6.0 prettyunits_1.0.2
> [43] hms_0.4.2 stringr_1.4.0
> [45] locfit_1.5-9.1 compiler_3.6.0
> [47] rlang_0.3.1 grid_3.6.0
> [49] GenomicDataCommons_1.7.3 RCurl_1.95-4.11
> [51] rappdirs_0.3.1 bitops_1.0-6
> [53] ExperimentHub_1.9.1 DBI_1.0.0
> [55] curl_3.3 R6_2.4.0
> [57] GenomicAlignments_1.19.1 dplyr_0.8.0.1
> [59] bit_1.1-14 ProtGenerics_1.15.0
> [61] readr_1.3.1 stringi_1.3.1
> [63] Rcpp_1.0.0 tidyselect_0.2.5
>
>
> On 2/28/19, 2:19 PM, "Bioc-devel on behalf of Ludwig Geistlinger" <bioc-devel-bounces using r-project.org on behalf of Ludwig.Geistlinger using sph.cuny.edu><mailto:bioc-devel-bounces using r-project.orgonbehalfofLudwig.Geistlinger@sph.cuny.edu> wrote:
>
> Hi,
>
>
> According to
>
>
> http://bioconductor.org/checkResults/devel/bioc-LATEST/index.html
>
>
> all SPB machines run different snapshots of R-devel with the most recent one (2019-01-22 r76000,
>
> already >1 month outdated) installed on celaya2.
>
>
> I observe two things:
>
>
> 1. My package builds on celaya2, but fails to build on the other SPB machines running older R-devel snapshots.
>
>
> http://bioconductor.org/checkResults/devel/bioc-LATEST/CNVRanger/
>
>
> 2. Further, I can't reproduce the SPB built error using a recent R-devel snapshot (2019-02-24 r76155) through the bioconductor/devel_base2 docker container.
>
>
> Can you please advise on how to best proceed from my side? Just wait for the SPB R-devel being updated?
>
>
> > sessionInfo()
>
> R Under development (unstable) (2019-02-24 r76155)
> Platform: x86_64-pc-linux-gnu (64-bit)
> Running under: Debian GNU/Linux 9 (stretch)
>
> Matrix products: default
> BLAS/LAPACK: /usr/lib/libopenblasp-r0.2.19.so
>
> locale:
> [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
> [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
> [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=C
> [7] LC_PAPER=en_US.UTF-8 LC_NAME=C
> [9] LC_ADDRESS=C LC_TELEPHONE=C
> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
>
> attached base packages:
> [1] stats4 parallel stats graphics grDevices utils datasets
> [8] methods base
>
> other attached packages:
> [1] TCGAutils_1.3.19
> [2] BSgenome.Btaurus.UCSC.bosTau6.masked_1.3.99
> [3] BSgenome.Btaurus.UCSC.bosTau6_1.4.0
> [4] BSgenome_1.51.0
> [5] rtracklayer_1.43.1
> [6] Biostrings_2.51.2
> [7] XVector_0.23.0
> [8] regioneR_1.15.2
> [9] BiocStyle_2.11.0
> [10] CNVRanger_0.99.7
> [11] curatedTCGAData_1.5.8
> [12] edgeR_3.25.3
> [13] limma_3.39.12
> [14] RaggedExperiment_1.7.4
> [15] MultiAssayExperiment_1.9.15
> [16] SummarizedExperiment_1.13.0
> [17] DelayedArray_0.9.8
> [18] BiocParallel_1.17.16
> [19] matrixStats_0.54.0
> [20] ensembldb_2.7.9
> [21] AnnotationFilter_1.7.0
> [22] GenomicFeatures_1.35.7
> [23] AnnotationDbi_1.45.0
> [24] Biobase_2.43.1
> [25] GenomicRanges_1.35.1
> [26] GenomeInfoDb_1.19.2
> [27] IRanges_2.17.4
> [28] S4Vectors_0.21.10
> [29] AnnotationHub_2.15.7
> [30] BiocGenerics_0.29.1
>
> loaded via a namespace (and not attached):
> [1] ProtGenerics_1.15.0 bitops_1.0-6
> [3] bit64_0.9-7 progress_1.2.0
> [5] httr_1.4.0 GenomicDataCommons_1.7.3
> [7] tools_3.6.0 R6_2.4.0
> [9] DBI_1.0.0 lazyeval_0.2.1
> [11] tidyselect_0.2.5 prettyunits_1.0.2
> [13] bit_1.1-14 curl_3.3
> [15] compiler_3.6.0 rvest_0.3.2
> [17] xml2_1.2.0 readr_1.3.1
> [19] rappdirs_0.3.1 stringr_1.4.0
> [21] digest_0.6.18 Rsamtools_1.99.2
> [23] rmarkdown_1.11 pkgconfig_2.0.2
> [25] htmltools_0.3.6 rlang_0.3.1
> [27] rstudioapi_0.9.0 RSQLite_2.1.1
> [29] shiny_1.2.0 jsonlite_1.6
> [31] dplyr_0.8.0.1 RCurl_1.95-4.11
> [33] magrittr_1.5 GenomeInfoDbData_1.2.0
> [35] Matrix_1.2-15 Rcpp_1.0.0
> [37] stringi_1.3.1 yaml_2.2.0
> [39] zlibbioc_1.29.0 grid_3.6.0
> [41] blob_1.1.1 promises_1.0.1
> [43] ExperimentHub_1.9.1 crayon_1.3.4
> [45] lattice_0.20-38 hms_0.4.2
> [47] locfit_1.5-9.1 knitr_1.21
> [49] pillar_1.3.1 biomaRt_2.39.2
> [51] XML_3.98-1.17 glue_1.3.0
> [53] evaluate_0.13 BiocManager_1.30.4
> [55] httpuv_1.4.5.1 purrr_0.3.0
> [57] assertthat_0.2.0 xfun_0.5
> [59] mime_0.6 xtable_1.8-3
> [61] later_0.8.0 tibble_2.0.1
> [63] GenomicAlignments_1.19.1 memoise_1.1.0
> [65] interactiveDisplayBase_1.21.0
>
> [[alternative HTML version deleted]]
>
> _______________________________________________
> Bioc-devel using r-project.org<mailto:Bioc-devel using r-project.org> mailing list
> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>
> _______________________________________________
> Bioc-devel using r-project.org<mailto:Bioc-devel using r-project.org> mailing list
> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>
>
>
> This email message may contain legally privileged and/or confidential information. If you are not the intended recipient(s), or the employee or agent responsible for the delivery of this message to the intended recipient(s), you are hereby notified that any disclosure, copying, distribution, or use of this email message is prohibited. If you have received this message in error, please notify the sender immediately by e-mail and delete this email message from your computer. Thank you.
> _______________________________________________
> Bioc-devel using r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>
This email message may contain legally privileged and/or confidential information. If you are not the intended recipient(s), or the employee or agent responsible for the delivery of this message to the intended recipient(s), you are hereby notified that any disclosure, copying, distribution, or use of this email message is prohibited. If you have received this message in error, please notify the sender immediately by e-mail and delete this email message from your computer. Thank you.
_______________________________________________
Bioc-devel using r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel
This email message may contain legally privileged and/or confidential information. If you are not the intended recipient(s), or the employee or agent responsible for the delivery of this message to the intended recipient(s), you are hereby notified that any disclosure, copying, distribution, or use of this email message is prohibited. If you have received this message in error, please notify the sender immediately by e-mail and delete this email message from your computer. Thank you.
[[alternative HTML version deleted]]
More information about the Bioc-devel
mailing list