[Bioc-devel] Error in HDF5 - Package submission - Not detected locally

[Peter Hickey]

Hi Tiago,

The following will create a similarly sized subset of the PBMC3k
dataset with the counts in-memory as a sparse matrix:

pbmc3k <- TENxPBMCData::TENxPBMCData("pbmc3k")
mini_pbmc3k <- pbmc3k[1:1700, 1:600]
assay(mini_pbmc3k) <- as(assay(mini_pbmc3k), "dgCMatrix")

Cheers,
Pete

On Mon, 16 Sep 2019 at 23:38, Tiago Lubiana Alves
<tiago.lubiana.alves using usp.br> wrote:
>
> Hello Mike,
>
> Thank you for the detailed explanation.
>
> You are right, for the vignette, I can download it from the ExperimentHub
> subset the pbmc3k dataset in the first few lines. The main point of having
> a new dataset was to use it in the examples of functions. The full dataset
> is too big and the examples would take too long otherwise.
>
> Still, if someone knows how to "transform from an HDF5 back
> SingleCellExperiment to using the data in memory", it would be of great
> help.
> Best,
> Tiago
>
>
> Tiago Lubiana
> Mestrando em Bioinformática, FCF/USP
> *Computational Systems Biology Laboratory (CSBL)*
> Telefone (laboratório): +55 (11) 2648-0240
> Telefone (pessoal): +55 (11) 954258000
>
>
> On Mon, Sep 16, 2019 at 5:39 AM Mike Smith <grimbough using gmail.com> wrote:
>
> > Hi Tiago,
> >
> > I suspect what has happened here is that when create the mini_pbmc3k
> > object, you're doing this by subsetting the PBMC, 3k scRNA-seq data from
> > ExperimenHub. The assay data for that are are actually stored in an HDF5
> > file which will be downloaded and stored in your ExperimentHub cache on
> > your local machine. When you take the subset for your new object it still
> > points to the same HDF5 file, and when you save mini_pbmc3k all it actually
> > saves is the location of the HDF5 rather than putting the data in the
> > object. This works locally, but naturally the BioC build server doesn't
> > have the original HDF5 in exactly the same location, and so it fails.
> >
> > Is there a reason you don't want to use the whole dataset and make the
> > first section of the vignette demonstrate how to download it via
> > ExperimentHub? Alternatively there is a way to transform from an HDF5 back
> > SingleCellExperiment to using the data in memory, but I can't remember it
> > right now - hopefully someone else will be along shortly.
> >
> > This also seems like the perfect opportunity to point out that our next
> > BioC Developers' Forum (
> > https://stat.ethz.ch/pipermail/bioc-devel/2019-September/015499.html) will
> > have a discussion on object serilaisation which should cover exactly this
> > type of issue.
> >
> > Best,
> > Mike
> >
> >
> >
> > On Sat, 14 Sep 2019 at 23:26, Tiago Lubiana Alves <
> > tiago.lubiana.alves using usp.br> wrote:
> >
> >> Hello,
> >>
> >> I am having a problem with a package submission build.
> >>
> >> This is the package issue:
> >> https://github.com/Bioconductor/Contributions/issues/1241
> >> And this is the ERROR:
> >>
> >> HDF5-DIAG: Error detected in HDF5 (1.10.5) thread 0:
> >>   #000: C:/hdf5_build/CMake-hdf5-1.10.5/hdf5-1.10.5/src/H5F.c line 509
> >> in H5Fopen(): unable to open file
> >>     major: File accessibilty
> >>     minor: Unable to open file
> >>   #001: C:/hdf5_build/CMake-hdf5-1.10.5/hdf5-1.10.5/src/H5Fint.c line
> >> 1498 in H5F_open(): unable to open file: time = Fri Sep 13 15:05:35
> >> 2019
> >> , name = '/home/lubianat/.cache/ExperimentHub/5486ffbe0e3_1605',
> >> tent_flags = 0
> >> (...)
> >>
> >> Quitting from lines 46-51 (fcoex.Rmd)
> >> Error: processing vignette 'fcoex.Rmd' failed with diagnostics:
> >> failed to open file '/home/lubianat/.cache/ExperimentHub/5486ffbe0e3_1605'
> >> --- failed re-building 'fcoex.Rmd'
> >>
> >>
> >> It looks like something is pointing in the wrong direction, but I have not
> >> been able to figure out exactly what. I've tried re-saving the data file
> >> ("mini_pbmc3k"), which is loaded in the vignette chunk of the error, but
> >> that did not help.
> >>
> >> Does anyone have a suggestion of what might be happening?
> >>
> >> Thank you very much for your time,
> >> Best,
> >> Tiago
> >>
> >> Tiago Lubiana
> >> Mestrando em Bioinformática, FCF/USP
> >> *Computational Systems Biology Laboratory (CSBL)*
> >> Telefone (laboratório): +55 (11) 2648-0240
> >> Telefone (pessoal): +55 (11) 954258000
> >>
> >>         [[alternative HTML version deleted]]
> >>
> >> _______________________________________________
> >> Bioc-devel using r-project.org mailing list
> >> https://stat.ethz.ch/mailman/listinfo/bioc-devel
> >>
> >
>
>         [[alternative HTML version deleted]]
>
> _______________________________________________
> Bioc-devel using r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/bioc-devel