[Bioc-devel] GenomicScores: irreproducible build error caused by gwascat

Robert Castelo robert@c@@te|o @end|ng |rom up|@edu
Tue Jun 9 18:52:32 CEST 2020


Martin, i think this is a very good solution, with your script one 
identifies automatically the broken dependencies and then the 
installation via git avoid having to clone the corresponding repos. i 
tried it on another package with broken dependencies right now and it 
works very well:

deps_broken("GGtools")
# A tibble: 2 x 5
   pkg     dep          install buildsrc checksrc
   <fct>   <chr>        <chr> <chr>    <chr>
1 GGtools GGBase       OK      ERROR    skipped
2 GGtools BiocParallel OK      OK       TIMEOUT

i'd say the 'deps_broken()' function would be a very useful addition to 
BiocPkgTools ;)

robert.

On 09/06/2020 15:57, Martin Morgan wrote:
> I wrote a small gist
>
>    https://gist.github.com/mtmorgan/21e18991c6ebdb388e8828bcc0fe72f6
>
> that queries for broken dependencies. Using two packages that have been mentioned recently on this mailing list
>
>    > tbl <- deps_broken(c("EnrichmentBrowser", "GenomicScores"))
>    > tbl
>    # A tibble: 1 x 5
>      pkg               dep   install buildsrc checksrc
>      <fct>             <chr> <chr>   <chr>    <chr>
>    1 EnrichmentBrowser safe  OK      OK       ERROR
>
>    > tbl %>% count(pkg, .drop=FALSE)
>    # A tibble: 2 x 2
>      pkg                   n
>      <fct>             <int>
>    1 EnrichmentBrowser     1
>    2 GenomicScores         0
>
> Trying to install 'safe' from public repositories fails
>
>    > BiocManager::install("safe")
>    Bioconductor version 3.12 (BiocManager 1.30.10), R 4.0.0 Patched (2020-05-14
>      r78465)
>    Installing package(s) 'safe'
>    Warning message:
>    package 'safe' is not available (for R version 4.0.0 Patched)
>
> but it can be installed directly from git using the 'remotes' package
>
>    remotes::install_git(
>        "https://git.bioconductor.org/packages/safe",
>        repos = BiocManager::repositories()
>    )
>
> I do not think it would be a good idea for BiocManager::install(version = "devel") to use a repository that included packages that installed but did not build or check. One interpretation of the GenomicScores exchange is that it helped alert the maintainer of gwascat about an error in their package, and encouraged a prompt fix.  Also BiocManager is really meant to help users follow best practices, rather than to make Bioconductor developer's lives easier.
>
> Perhaps there could be some other easy developer-oriented approach to installing broken packages?
>
> Martin
>
> On 6/8/20, 10:20 PM, "Hervé Pagès" <hpages using fredhutch.org> wrote:
>
>      On 6/5/20 07:52, Martin Morgan wrote:
>      > no, the build system should only propagate packages that have passed build and check -- the goal is to make the life of the user easier and more reliable, us developers have to sweat the details!
>
>      This makes a lot of sense for release but for devel maybe the criteria
>      for propagating a package could be relaxed? It doesn't sound too insane
>      to do that but I realize it does introduce a difference between release
>      and devel that could also cause its own confusion.
>
>      Another approach that is currently under consideration is to make
>      BiocManager::install() capable of installing/updating packages directly
>      from git.bioconductor.org in BioC devel. This could be achieved by
>      making the source tarballs resulting from
>
>         git checkout master https://git.bioconductor.org/packages/<pkg> &&
>           R CMD build --no-build-vignettes <pkg>
>
>      available in their own repo (e.g.
>      https://bioconductor.org/packages/devel) and add this repo to
>      BiocManager::repositories() in devel. Note that these no-vignettes
>      source tarballs are actually those used by the build system during the
>      INSTALL stage.
>
>      How do people feel about that?
>
>      H.
>
>      >
>      > Usually there is a single package that has changed and causes problems, and like in the present case a little detective work (once one knows what to look for) leads quickly to the suspect.
>      >
>      > Martin
>      >
>      > On 6/5/20, 10:18 AM, "Robert Castelo" <robert.castelo using upf.edu> wrote:
>      >
>      >      Thanks Martin, i wasn't aware about that fact. i've cloned the gwascat
>      >      repo and been able to directly install from the directory and reproduce
>      >      the error. in this case, this was easy because it involved just one
>      >      package but with multiple broken package dependencies, i'd have to
>      >      manually clone each of them and install them to reproduce the error.
>      >
>      >      would it be a good idea in the devel build system to propagate the tar
>      >      ball resulting of
>      >
>      >      R CMD build --no-build-vignettes
>      >
>      >      so that we get automatically updates for at least those whose problem is
>      >      in the vignettes only?
>      >
>      >      robert.
>      >
>      >      On 05/06/2020 15:37, Martin Morgan wrote:
>      >      > R CMD INSTALL <foo> or install.packages("foo", repos = NULL) on a clone of the repository. A build might fail because the examples or vignette fail to build, whereas for installation the only requirement is that the package is syntactically correct.
>      >      >
>      >      > Martin
>      >      >
>      >      > On 6/5/20, 7:58 AM, "Robert Castelo" <robert.castelo using upf.edu> wrote:
>      >      >
>      >      >      never thought about it this way, but how can the system install
>      >      >      something that does not build?
>      >      >
>      >      >      how should *i* install something that does not build to reproduce the error?
>      >      >
>      >      >      sorry if these are very naive questions!!
>      >      >
>      >      >      robert.
>      >      >
>      >      >      On 05/06/2020 13:34, Martin Morgan wrote:
>      >      >      > The build system installs the version of gwascat that is available from git checkout (anticipating that this will propagate). gwascat installs, but fails to pass check -- it is likely broken
>      >      >      >
>      >      >      >    https://urldefense.proofpoint.com/v2/url?u=https-3A__bioconductor.org_checkResults_devel_bioc-2DLATEST_gwascat_&d=DwIGaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=uJnw6vW_sTWKb9l6DPFY7NEqPq_zkEzDFbIrir-Xis8&s=r_GJJMDdjgc6nXTDSC0OhJW6gho041eSiWwKrw3h00w&e=
>      >      >      >
>      >      >      > in a way that causes your package to fail
>      >      >      >
>      >      >      > Martin
>      >      >      >
>      >      >      > On 6/5/20, 6:27 AM, "Bioc-devel on behalf of Robert Castelo" <bioc-devel-bounces using r-project.org on behalf of robert.castelo using upf.edu> wrote:
>      >      >      >
>      >      >      >      hi,
>      >      >      >
>      >      >      >      my package GenomicScores is not building, see:
>      >      >      >
>      >      >      >      https://urldefense.proofpoint.com/v2/url?u=http-3A__bioconductor.org_checkResults_devel_bioc-2DLATEST_GenomicScores_malbec1-2Dbuildsrc.html&d=DwIGaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=uJnw6vW_sTWKb9l6DPFY7NEqPq_zkEzDFbIrir-Xis8&s=-TxA290vxOPuMjHcxETokMk-jPL8H_YJpKMIr3UL50E&e=
>      >      >      >
>      >      >      >      apparently, it is breaking in the following lines of its vignette:
>      >      >      >
>      >      >      >      library(gwascat)
>      >      >      >      data(ebicat37)
>      >      >      >
>      >      >      >      which in the report from the bioc build machine says:
>      >      >      >
>      >      >      >      gwascat loaded.  Use makeCurrentGwascat() to extract current image.
>      >      >      >        from EBI.  The data folder of this package has some legacy extracts.
>      >      >      >      Quitting from lines 404-408 (GenomicScores.Rmd)
>      >      >      >      Error: processing vignette 'GenomicScores.Rmd' failed with diagnostics:
>      >      >      >      object 'ebicat37' not found
>      >      >      >      --- failed re-building ‘GenomicScores.Rmd’
>      >      >      >
>      >      >      >      however, in my installation of current bioc-devel on R-4.0 with all
>      >      >      >      packages up to date, GenomicScores builds fine and i cannot reproduce
>      >      >      >      this error. below you can find my session information after the previous
>      >      >      >      two instructions. the logs of 'gwascat' show changes in May 2nd that
>      >      >      >      could be potentially responsible for this but the fact is that 'gwascat'
>      >      >      >      is not building either and it does not seem that the changes propagate
>      >      >      >      through the build system, its version is still 2.21.0, on which
>      >      >      >      GenomicScores built without problems for the current release.
>      >      >      >
>      >      >      >      i'm cc'ing this email to Vince, as maintainer of 'gwascat', in case he
>      >      >      >      has some more specific suggestion about this but any hint will be
>      >      >      >      greatly appreciated.
>      >      >      >
>      >      >      >      thanks!!
>      >      >      >
>      >      >      >      sessionInfo()
>      >      >      >      R version 4.0.0 (2020-04-24)
>      >      >      >      Platform: x86_64-pc-linux-gnu (64-bit)
>      >      >      >      Running under: CentOS Linux 7 (Core)
>      >      >      >
>      >      >      >      Matrix products: default
>      >      >      >      BLAS:   /opt/R/R-4.0.0/lib64/R/lib/libRblas.so
>      >      >      >      LAPACK: /opt/R/R-4.0.0/lib64/R/lib/libRlapack.so
>      >      >      >
>      >      >      >      locale:
>      >      >      >        [1] LC_CTYPE=en_US.UTF8       LC_NUMERIC=C
>      >      >      >        [3] LC_TIME=en_US.UTF8        LC_COLLATE=en_US.UTF8
>      >      >      >        [5] LC_MONETARY=en_US.UTF8    LC_MESSAGES=en_US.UTF8
>      >      >      >        [7] LC_PAPER=en_US.UTF8       LC_NAME=C
>      >      >      >        [9] LC_ADDRESS=C              LC_TELEPHONE=C
>      >      >      >      [11] LC_MEASUREMENT=en_US.UTF8 LC_IDENTIFICATION=C
>      >      >      >
>      >      >      >      attached base packages:
>      >      >      >      [1] stats     graphics  grDevices utils     datasets methods   base
>      >      >      >
>      >      >      >      other attached packages:
>      >      >      >      [1] gwascat_2.21.0 colorout_1.2-2
>      >      >      >
>      >      >      >      loaded via a namespace (and not attached):
>      >      >      >        [1] Rcpp_1.0.4.6 lattice_0.20-41
>      >      >      >        [3] prettyunits_1.1.1 Rsamtools_2.5.1
>      >      >      >        [5] Biostrings_2.57.1 assertthat_0.2.1
>      >      >      >        [7] digest_0.6.25 BiocFileCache_1.13.0
>      >      >      >        [9] R6_2.4.1 GenomeInfoDb_1.25.1
>      >      >      >      [11] stats4_4.0.0 RSQLite_2.2.0
>      >      >      >      [13] httr_1.4.1 ggplot2_3.3.1
>      >      >      >      [15] pillar_1.4.4 zlibbioc_1.35.0
>      >      >      >      [17] rlang_0.4.6 GenomicFeatures_1.41.0
>      >      >      >      [19] progress_1.2.2 curl_4.3
>      >      >      >      [21] blob_1.2.1 S4Vectors_0.27.11
>      >      >      >      [23] Matrix_1.2-18 BiocParallel_1.23.0
>      >      >      >      [25] stringr_1.4.0 RCurl_1.98-1.2
>      >      >      >      [27] bit_1.1-15.2 biomaRt_2.45.0
>      >      >      >      [29] munsell_0.5.0 DelayedArray_0.15.1
>      >      >      >      [31] compiler_4.0.0 rtracklayer_1.49.3
>      >      >      >      [33] pkgconfig_2.0.3 askpass_1.1
>      >      >      >      [35] BiocGenerics_0.35.3 openssl_1.4.1
>      >      >      >      [37] tidyselect_1.1.0 SummarizedExperiment_1.19.4
>      >      >      >      [39] tibble_3.0.1 GenomeInfoDbData_1.2.3
>      >      >      >      [41] IRanges_2.23.7 matrixStats_0.56.0
>      >      >      >      [43] XML_3.99-0.3 crayon_1.3.4
>      >      >      >      [45] dplyr_1.0.0 dbplyr_1.4.4
>      >      >      >      [47] GenomicAlignments_1.25.1 bitops_1.0-6
>      >      >      >      [49] rappdirs_0.3.1 grid_4.0.0
>      >      >      >      [51] gtable_0.3.0 lifecycle_0.2.0
>      >      >      >      [53] DBI_1.1.0 magrittr_1.5
>      >      >      >      [55] scales_1.1.1 stringi_1.4.6
>      >      >      >      [57] XVector_0.29.1 ellipsis_0.3.1
>      >      >      >      [59] generics_0.0.2 vctrs_0.3.0
>      >      >      >      [61] tools_4.0.0 bit64_0.9-7
>      >      >      >      [63] Biobase_2.49.0 glue_1.4.1
>      >      >      >      [65] purrr_0.3.4 hms_0.5.3
>      >      >      >      [67] parallel_4.0.0 colorspace_1.4-1
>      >      >      >      [69] AnnotationDbi_1.51.0 GenomicRanges_1.41.3
>      >      >      >      [71] memoise_1.1.0
>      >      >      >
>      >      >      >
>      >      >      >
>      >      >      >      	[[alternative HTML version deleted]]
>      >      >      >
>      >      >      >      _______________________________________________
>      >      >      >      Bioc-devel using r-project.org mailing list
>      >      >      >      https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_bioc-2Ddevel&d=DwIGaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=uJnw6vW_sTWKb9l6DPFY7NEqPq_zkEzDFbIrir-Xis8&s=TODpGxZp3CBBjK_BWiLYHgQMHKzOOSe3Yj7Viu3w6TI&e=
>      >      >
>      >      >
>      >
>      > _______________________________________________
>      > Bioc-devel using r-project.org mailing list
>      > https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_bioc-2Ddevel&d=DwIGaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=uJnw6vW_sTWKb9l6DPFY7NEqPq_zkEzDFbIrir-Xis8&s=TODpGxZp3CBBjK_BWiLYHgQMHKzOOSe3Yj7Viu3w6TI&e=
>      >
>
>      --
>      Hervé Pagès
>
>      Program in Computational Biology
>      Division of Public Health Sciences
>      Fred Hutchinson Cancer Research Center
>      1100 Fairview Ave. N, M1-B514
>      P.O. Box 19024
>      Seattle, WA 98109-1024
>
>      E-mail: hpages using fredhutch.org
>      Phone:  (206) 667-5791
>      Fax:    (206) 667-1319



More information about the Bioc-devel mailing list