[Bioc-devel] HDF5Array failure on windows

[Hervé Pagès]

On 4/7/20 06:15, Kasper Daniel Hansen wrote:
> Thanks for the two pointers.
> 
> Herve: we are using a number of unexpected functions from HDF5Array:
>    ‘HDF5Array:::.create_dir’, ‘HDF5Array:::.replace_dir’, 
> ‘HDF5Array:::.shorten_assay2h5_links’

tss tss... naughty!

> That might be relevant in light of that specific test error.

OK, always good to know, thanks! The error seems to be related to the 
code I added recently for handling on-disk dimnames. I only started to 
look at this and will report here when I find something.

H.

> 
> 
> On Mon, Apr 6, 2020 at 1:13 PM Hervé Pagès <hpages using fredhutch.org 
> <mailto:hpages using fredhutch.org>> wrote:
> 
>     Hi Kasper, Martin,
> 
>     About bsseq's timeout: An important recent change to
>     DelayedArray/HDF5Array is that, starting with DelayedArray 0.13.8,
>     nothing in these packages uses parallel evaluation **by default**.
>     Concretely this means that getAutoBPPARAM() now returns NULL on a fresh
>     session instead of one of the parallelization backends defined in
>     BiocParallel (e.g. MulticoreParam on Linux/Mac, SnowParam on Windows).
>     This could slow down some code in bsseq or other packages that were
>     benefiting from the automatic parallel evaluation. Now it's the
>     responsibility of the user to set the parallelization backend (with
>     setAutoBPPARAM) if they wish things like matrix multiplication,
>     rowsum()
>     or rowSums() use parallel evaluation again.
> 
>     Also BiocParallel has been moved from Depends to Suggests.
> 
>     About bsseq error on Windows: That seems indeed to be related to the
>     HDF5Array error on the same platform. The unit tests in bsseq and
>     HDF5Array both fail with the same error ("HDF5 dataset
>     './.HDF5ArrayAUTO00014_dimnames/2' does not exist"). I'll take a look.
>     What's puzzling is that we see this error on both Windows archs for
>     bsseq (i.e. on 64-bit and 32-bit) while we only see it on 64-bit
>     Windows
>     for HDF5Array. This suggests something nasty and hard to troubleshoot..
>     sigh!
> 
>     Cheers,
>     H.
> 
> 
>     On 4/6/20 06:59, Martin Morgan wrote:
>      > Timeouts are often related to parallel evaluation, either
>     competing for resources or underlying limitations in the robustness
>     of the (parallel) code. Something close to best practice is to limit
>     or eliminate parallel evaluation in examples, vignettes, and tests.
>      >
>      > Documentation issues are usually related to the use of [] in
>     \link[]{}, as described at
>     https://urldefense.proofpoint.com/v2/url?u=https-3A__cran.r-2Dproject.org_doc_manuals_r-2Drelease_R-2Dexts.html-23Cross-5F002dreferences&d=DwIGaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=pNn2sHW_Vo7GJjsEgdNz2QJq4GrASIW3z6QSRefcLkY&s=2r59fOKiJQ_AJBSaOXUDB1j3dD1txuMA-cJzCnnIE9k&e= 
>     . The important point is that the [] information is the NAME OF THE
>     HTML FILE of the documentation module, not the 'name' of the
>     documentation module. The solution (from my perspective) is usually
>     to simply omit the [] and let the R help system resolve the link
>     dynamically (sometimes prompting the user to choose, if there
>     multiple man pages).
>      >
>      > Martin Morgan
>      >
>      >
>      > On 4/6/20, 9:55 AM, "Bioc-devel on behalf of Kasper Daniel
>     Hansen" <bioc-devel-bounces using r-project.org
>     <mailto:bioc-devel-bounces using r-project.org> on behalf of
>     kasperdanielhansen using gmail.com <mailto:kasperdanielhansen using gmail.com>>
>     wrote:
>      >
>      >      We currently (and for a while) have had various errors in
>     bsseq that seems
>      >      to have come and go. We now have a failure on Windows which
>     is related to
>      >      HDF5. I see that HDF5Array also fails on Windows, which
>     makes me believe
>      >      the error could be upstream. There is also a warning about
>     hep page links
>      >      which HDF5Array has as well. Any comments on this is
>     appreciated.
>      >
>      >      Perhaps relatedly: we are getting a timeout on linux. On my
>     own OS X setup,
>      >      it completes in a timely fashion. Do we sometimes have
>     unexplained timeouts
>      >      related to HDF5?
>      >
>      >      --
>      >      Best,
>      >      Kasper
>      >
>      >       [[alternative HTML version deleted]]
>      >
>      >      _______________________________________________
>      > Bioc-devel using r-project.org <mailto:Bioc-devel using r-project.org>
>     mailing list
>      >
>     https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_bioc-2Ddevel&d=DwIGaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=pNn2sHW_Vo7GJjsEgdNz2QJq4GrASIW3z6QSRefcLkY&s=28DlcVps0EVU-QaF8utnvHUFuj1DpJIZrrXrlPqIb28&e=
>      >
>      > _______________________________________________
>      > Bioc-devel using r-project.org <mailto:Bioc-devel using r-project.org>
>     mailing list
>      >
>     https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_bioc-2Ddevel&d=DwIGaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=pNn2sHW_Vo7GJjsEgdNz2QJq4GrASIW3z6QSRefcLkY&s=28DlcVps0EVU-QaF8utnvHUFuj1DpJIZrrXrlPqIb28&e=
>      >
> 
>     -- 
>     Hervé Pagès
> 
>     Program in Computational Biology
>     Division of Public Health Sciences
>     Fred Hutchinson Cancer Research Center
>     1100 Fairview Ave. N, M1-B514
>     P.O. Box 19024
>     Seattle, WA 98109-1024
> 
>     E-mail: hpages using fredhutch.org <mailto:hpages using fredhutch.org>
>     Phone:  (206) 667-5791
>     Fax:    (206) 667-1319
> 
> 
> 
> -- 
> Best,
> Kasper

-- 
Hervé Pagès

Program in Computational Biology
Division of Public Health Sciences
Fred Hutchinson Cancer Research Center
1100 Fairview Ave. N, M1-B514
P.O. Box 19024
Seattle, WA 98109-1024

E-mail: hpages using fredhutch.org
Phone:  (206) 667-5791
Fax:    (206) 667-1319