[Bioc-devel] Cannot reproduce a regionReport 1.21.7 error on the Bioc 3.11 Linux builder

Fri Apr 24 06:52:16 CEST 2020

Hi Bioc-devel again,

I believe that I have found the source of my regionReport problem and
a workaround. We'll know tomorrow if it goes through the Bioc builds
but it works at
https://github.com/leekgroup/regionReport/actions/runs/86364716.

The problem is related to
https://github.com/ropensci/magick/issues/171 and similar issues. The
workaround is to use `crop = NULL` on R Markdown code chunks to
disable the default image cropping behavior by
BiocStyle::html_document, and thus avoid cache issues from magick (R)
and ImageMagick (not R).

Both the Bioc-devel Linux machine and GitHub Actions using
Bioconductor devel run into a problem where there's not enough
resources in the cache for cropping the images produced by
regionReport::derfinderReport(). Either (a) the Bioc-devel Windows and
macOS machines don't have magick installed (so no cropping gets
attempted), (b) use a compiled version of magick that ships with the
right version of ImageMagick, or (c) they have more resources than the
bioc Linux build machine and the bioc devel docker running on GHA
Linux

In any case, disabling cropping circumvents this issue.

Here's the full investigative report.

## A new lead

I recently stumbled upon [1] and re-reading it now I see that they
link to [2] which mentions that BiocStyle crops by default and that
this can be turned off using `crop = NULL`.

regionReport uses BiocStyle::html_document both for its vignettes as
well as the reports it makes by default. Actually at [3], which was
built using the Bioconductor devel docker & GitHub Actions (GHA), I
see tons of messages that are similar to:

## The magick package is required to crop
"basicExploration_files/figure-html/pvals-1.png" but not available.

For reference, [4] is how that example from
regionReport::derfinderReport() looks built using GHA. I copied the
log of messages to a gist at [5]. From [6] I found [7] where Mike
Smith explains to Aaron Lun the default behavior of BiocStyle and
again suggests to use `crop = NULL` just like Yihui Xie mentioned
earlier at [2]. I see at [8] that Kevin Rue-Albrecht also ran into a
`crop = NULL` problem in iSEE.

So I have a good lead now!

## Adding magick to Suggests

We can see clearly that magick is not available on my GitHub Actions
run using the Biocondutor devel docker: regionReport 1.21.7 doesn't
mention magick on it's DESCRIPTION so it's never installed on GHA.
However, magick is likely installed on the Bioconductor 3.11 Linux
build machine and thus it is used for cropping. At [9] I bumped
regionReport to 1.21.8 after adding magick to suggests. Actually, this
is what Yihui Xie recommends BiocStyle could/should do in [2]. That
crashed and burned on GHA Linux with Bioc-docker at [10], thus
reproducing the Linux build machine error.

Quitting from lines 224-229 (basicExploration.Rmd)
35Error: processing vignette 'regionReport.Rmd' failed with diagnostics:
36R: cache resources exhausted
`/tmp/RtmpfaoqUE/Rbuild74c3a6bf655/regionReport/vignettes/report/html/basicExploration_files/figure-html/plotRegions-17.png'
@ error/cache.c/OpenPixelCache/4083

Actually, this error message is more informative than the one from the
Bioc Linux builder as it gives the actual image name where it all
burned down.

## Moving problematic code from the vignette to an example

At [11] I moved the problematic code to an example instead of the
vignette. On GHA, it failed again at [12] predictably on another image
than in [10]. Also predictably, R CMD build worked this time, but R
CMD check failed. However, now the I get the full logs that help me
see the details much better and link it to code [13] (so if that was
the bug source, I would know where to look).

Quitting from lines 399-400 (basicExploration.Rmd)
355Error in magick_image_trim(image, fuzz) :
356 R: cache resources exhausted
`/__w/regionReport/regionReport/check/regionReport.Rcheck/derfinderReport-example/basicExploration/basicExploration_files/figure-html/genomeOverview3-1.png'
@ error/cache.c/OpenPixelCache/4083
357Calls: derfinderReport ... <Anonymous> -> assert_image ->
<Anonymous> -> magick_image_trim
358Execution halted

## Using crop = NULL

At [14] I dropped magick from the DESCRIPTION and added crop = NULL to
the template Rmd files in regionReport. On GHA the Linux build ran
well [15] and the resulting example report [overwrote 4] looks well =)

## Other OS?

What about the other OS? Well, I guess that they have some extra
resources than the Linux one. For example, locally on my macOS laptop
regionReport 1.2.8 did build [16] with magick being available.

Alternatively, the other Bioc-devel build machines (Windows, macOS)
might not have magick installed, hence no cropping was attempted and
no errors would have been detected, like in GHA's Linux regionReport
1.2.7 build.

I replied to Thomas Lumley about the lead idea at [17] before doing
all these tests.

Best,
Leo

[1] https://github.com/yihui/knitr/issues/1785#issuecomment-574723631
[2] https://github.com/yihui/knitr/issues/1796#issuecomment-575371248
[3] http://leekgroup.github.io/regionReport/articles/regionReport.html#derfinder-single-base-level-case-1
[4] http://leekgroup.github.io/regionReport/reference/derfinderReport-example/basicExploration/basicExploration.html
[5] https://stat.ethz.ch/pipermail/bioc-devel/2020-April/016645.html
[6] https://github.com/Bioconductor/BiocStyle/issues?q=crop
[7] https://github.com/Bioconductor/BiocStyle/issues/65#issuecomment-552832630
[8] https://github.com/iSEE/iSEE/commit/817d24d0aa092d04cbdfeb5daf5bfe60a711435a
[9] https://github.com/leekgroup/regionReport/commit/14e7b714b7598add2b67d797668cecd121a726fe
[10] https://github.com/leekgroup/regionReport/runs/614073194#step:8:36
[11] https://github.com/leekgroup/regionReport/commit/c63ec2c737d8514f521ec64c66189ed8a8a772a2
[12] https://github.com/leekgroup/regionReport/runs/614135852?check_suite_focus=true#step:8:354
[13] https://github.com/leekgroup/regionReport/blob/c63ec2c737d8514f521ec64c66189ed8a8a772a2/inst/basicExploration/basicExploration.Rmd#L399
[14] https://github.com/leekgroup/regionReport/commit/c7b2fc8dc086befd0539a8425cab81cc61baeb02
[15] https://github.com/leekgroup/regionReport/runs/614159356?check_suite_focus=true#step:8:30
[16] https://gist.github.com/lcolladotor/73db141aea3c7d9540ecd18b9b253e40
[17] https://twitter.com/tslumley/status/1253505155523178496?s=20

On Thu, Apr 23, 2020 at 10:03 PM Leonardo Collado Torres

<lcolladotor using gmail.com> wrote:
>
> Hi,
>
> I went to google again and maybe
> https://github.com/phw/peek/issues/112#issuecomment-296690302 is the
> answer? It would require someone with access to the Linux builder to
> check this particular ImageMagick config XML file.
>
> I had linked to this thread earlier
> https://stat.ethz.ch/pipermail/bioc-devel/2020-April/016538.html
> though back then I mentioned that it could be a disk space issue. That
> was before the more recent thread about that ended with ggbio being
> bumpted to 1.35.2 (for which using Bioconductor's devel docker
> revealed the problem).
>
> I also just asked on Twitter
> https://twitter.com/fellgernon/status/1253502433231343617?s=20 to see
> if anyone has seen a similar error.
>
> Best,
> Leo
>
> On Thu, Apr 23, 2020 at 9:36 PM Leonardo Collado Torres
> <lcolladotor using gmail.com> wrote:
> >
> > Hi Bioc-devel,
> >
> > I'm having trouble with my regionReport package on the Bioc-devel
> > (3.11) Linux builds. Two weeks ago, using the Bioconductor devel
> > docker I was able to reproduce an error that was solved by ggbio
> > bumping their version number to 1.35.2 as described in [1]. I've tried
> > my best to reproduce the error shown at [2] which is:
> >
> > Quitting from lines 224-229 (basicExploration.Rmd)
> > Error: processing vignette 'regionReport.Rmd' failed with diagnostics:
> > R: unable to get registry ID `cache:hosts' @
> > error/registry.c/GetImageRegistry/202
> >
> > Recently [3], I bumped the regionReport version to 1.21.7 where I
> > explicitly asked for ggbio 1.35.2 to be available. I suspected that
> > the Linux builder still had version ggbio 1.35.1 installed. But that
> > did not resolve the issue as the current build report is for
> > regionReport version 1.21.7 and there's no error about ggbio 1.21.7
> > not being available. Hm... the required versions for the dependencies
> > does get checked at R CMD build (or even install), right?
> >
> > At [4] you can see that regionReport 1.21.7 can be successfully
> > checked against Bioconductor devel's docker. This uses GitHub actions
> > that has 7 GB of RAM available [5]. The memory resources (7GB vs 2.5
> > GB? [6]), disk space (14 GB vs ?) and some system dependencies are the
> > only differences I can think of (installed through [7], none seem to
> > me to be related from what I see at [8]) besides the Linux builder
> > differences vs the Bioconductor devel docker (dunno what they are).
> >
> > If you have any suggestions, please let me know.
> >
> > Best,
> > Leo
> >
> > [1] https://stat.ethz.ch/pipermail/bioc-devel/2020-April/016538.html
> > [2] https://master.bioconductor.org/checkResults/3.11/bioc-LATEST/regionReport/malbec2-buildsrc.html
> > [3] https://github.com/leekgroup/regionReport/commit/5e7d956f71011f8551f2431f5af0e212d04be9f1
> > [4] https://github.com/leekgroup/regionReport/runs/604161822?check_suite_focus=true#step:8:98
> > [5] https://help.github.com/en/actions/reference/virtual-environments-for-github-hosted-runners#supported-runners-and-hardware-resources
> > [6] https://github.com/Bioconductor/Contributions/issues/1389#issuecomment-605024369
> > 2.5 GB is used for the 32-bit windows Single Package Builder.
> > [7] https://github.com/leekgroup/regionReport/runs/604161822?check_suite_focus=true#step:6:2
> > [8] https://github.com/leekgroup/regionReport/runs/604161822?check_suite_focus=true#step:6:167
> >
> > Leonardo Collado Torres, Ph. D., Research Scientist
> > Lieber Institute for Brain Development
> > 855 N Wolfe St, Suite 300
> > Baltimore, MD 21205
> > http://lcolladotor.github.io