[Bioc-devel] BiocParallel and AnnotationDbi: database disk image is malformed

[Vincent Carey]

good question

some of the discussion on

http://sqlite.1065341.n5.nabble.com/Parallel-access-to-read-only-in-memory-database-td91814.html

seems relevant.

converting the relatively small annotation package content to pure R
read-only tables on the master before parallelizing
might be very simple?

On Fri, Jan 19, 2018 at 11:43 AM, Ludwig Geistlinger <
Ludwig.Geistlinger at sph.cuny.edu> wrote:

> Hi,
>
> Within a package I am developing, I would like to enable parallel probe to
> gene mapping for a compendium of microarray datasets.
>
> This accordingly makes use of annotation packages such as hgu133a.db,
> which in turn connect to the SQLite database via AnnotationDbi.
>
> When running in multi-core mode (i.e. using a MulticoreParam with
> BiocParallel) using more than 2 cores, this causes the error:
>
> database disk image is malformed
>
>
> In a very similar problem:
>
> https://support.bioconductor.org/p/38541/
>
> Adi Tarca and Dan Tenenbaum identified and resolved this problem by
> ensuring that each process has its own unique database connection, i.e.
> AnnotationDbi is not loaded before sending the job to the workers.
>
> This solution was easily realized as this analysis was carried out within
> a script and not a package.
>
> However, within my package, AnnotationDbi is loaded as a dependency of my
> package's imports.
>
> How to resolve this here?
> I am not sure whether I perfectly understand the underlying mechanisms,
> but is there a way to make my workers load their own version of
> AnnotationDbi instead of using the one of the parent process?
> Or am I supposed to unload all packages depending on AnnotationDbi, and
> AnnotationDbi itself, before sending the job to the workers (and reload all
> of them after the job has finished?)
>
> Thanks a lot,
> Ludwig
>
>
>
> --
> Dr. Ludwig Geistlinger
> CUNY School of Public Health
>
>         [[alternative HTML version deleted]]
>
> _______________________________________________
> Bioc-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>

	[[alternative HTML version deleted]]