[Bioc-devel] check fails on arch 'i386'

Fri Oct 18 03:51:15 CEST 2019

Many thanks Kasper, this would be much appreciated.

Best,
Tim

On 18/10/19 12:58 am, Kasper Daniel Hansen wrote:
I see these packages may have something to do with mini. There is a a (very old, nothing new) issue with the annotation objects in minfi where loading them uses a _lot_ of ram. For a long time I couldn't understand it, but now I get it. I should really fix this, and probably _very_soon. It won't have any real user-visible changes (unless you're using the SNP functionality which is very nice but not used by many people I think), but it will require a new set of annotation packages.

Best,
Kasper

On Thu, Oct 17, 2019 at 4:08 AM Tim Peters <t.peters using garvan.org.au<mailto:t.peters using garvan.org.au>> wrote:
Hi Hervé (and Reka),

I am having much the same problem on tokay1 with DMRcate
(https://bioconductor.org/checkResults/3.10/bioc-LATEST/DMRcate/tokay1-checksrc.html).

I've gone through a number of iterations trying to minimise the amount
of memory run by the examples (including your suggestions such as using
GRanges as a container for everything in a CpGannotated object, and
restricting analysis to 100 CpG sites) but the memory allocation error
is persistent, failing to allocate a quite modest ~500Mb.

I suspect it's being caused by minfi::makeGenomicRatioSetFromMatrix()
which is called by cpg.annotate() and DMR.plot() - I started a thread
here around the time of the April release concerning the problem but it
wasn't responded to:

https://stat.ethz.ch/pipermail/bioc-devel/2019-April/015012.html

This leaves me in a bind as minfi::makeGenomicRatioSetFromMatrix() has
exactly the functionality cpg.annotate() and DMR.plot() needs, plus it
provides minimal code and I don't really want to reinvent the wheel here!

Would it be possible to waive the 32-bit Windows build for DMRcate?

Best,

Tim

On 15/10/19 4:34 am, Pages, Herve wrote:
> Hi Toth,
>
> Not clear to me what's going on but I kind of suspect this might have
> something to do with the use of data.table.
>
> A few things to keep in mind:
>
> - 'R CMD check' runs all the example in the same R session. This means
> that the outcome of the examples of a given man page can be affected by
> the examples from other man pages.
>
> - A process cannot allocate more than 3 Gb of memory on 32-bit Windows.
> There is no such limit on 64-bit Windows or on the other platforms we
> support (Linux and Mac). On these platforms the maximum amount of memory
> a process can allocate only depends on what's available on the system.
>
> - Generally speaking, sticking to the Bioconductor way of doing things
> (e.g use of GRanges objects instead of data.table objects, use of
> findOverlaps() instead of data.table::foverlaps()) will lead to faster
> code and reduced memory usage. Therefore I would strongly encourage you
> to consider switching to GRanges/findOverlaps() and to avoid the use of
> data.table in methrix.
>
> Best,
> H.
>
>
> On 10/11/19 09:36, Toth, Reka wrote:
>> Dear BioC developers,
>>
>> The most recent build of methrix fails on Tokay1 server:
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__bioconductor.org_spb-5Freports_methrix-5Fbuildreport-5F20191009174210.html&d=DwIF-g&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=doO3ej8vD58ZBQWVxkHTaCLk-xipbskav8uofJL2syI&s=zs3uDnFqyAl-hr_7StB3Yr-tGZW8VPfclbPY_JKtSt8&e=
>> The error message is the following:
>>
>> ** running examples for arch 'i386' ... ERROR
>> Running examples in 'methrix-Ex.R' failed
>> The error most likely occurred in:
>>
>>> base::assign(".ptime", proc.time(), pos = "CheckExEnv")
>>> ### Name: get_region_summary
>>> ### Title: Extract and summarize methylation or coverage info by regions of
>>> ###   interest
>>> ### Aliases: get_region_summary
>>>
>>> ### ** Examples
>>>
>>> data("methrix_data")
>>> get_region_summary(m = methrix_data, regions = data.table(chr = "chr21", start = 27867971, end =  27868103), type = "M", how = "mean")
>> -Checking for overlaps..
>> Error: cannot allocate vector of size 1.4 Gb
>> Execution halted
>>
>> I am a bit puzzled what the reasons could be, since it passed this step during the previous builds and I didn’t change any of the functions in my last commit.
>> When I run the check on our local Windows, I don’t get any error. We use a very small dataset as example, it definitely should not use a vector of size 1.4 Gb.
>>
>> I would appreciate every input or idea on how to tackle this error.
>> Thanks a lot!
>>
>> Best,
>> Reka
>>
>> Dr. Reka Toth
>> Computational Epigenomics Group
>> Division of Epigenomics and Cancer Risk Factors /C010
>> Postdoctoral Fellow
>>
>> German Cancer Research Center (DKFZ)
>> Foundation under Public Law
>> Im Neuenheimer Feld 280
>> 69120 Heidelberg
>> Germany
>>
>> r.toth using dkfz-heidelberg.de<mailto:r.toth using dkfz-heidelberg.de>
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.dkfz.de&d=DwIF-g&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=doO3ej8vD58ZBQWVxkHTaCLk-xipbskav8uofJL2syI&s=U7YGBonX7WmBqfyEneCtEsuRUKl1fbTS6Esi3e9XGAI&e=
>>
>> _______________________________________________
>> Bioc-devel using r-project.org<mailto:Bioc-devel using r-project.org> mailing list
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_bioc-2Ddevel&d=DwIF-g&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=doO3ej8vD58ZBQWVxkHTaCLk-xipbskav8uofJL2syI&s=2BO1kFpXuH1gpxwLxHae_ELnR7g_k9JV-qtslgowQMQ&e=
>>
--

===================

Tim Peters, PhD

Bioinformatics Research Officer | Immunogenomics Laboratory | Immunology Division

Garvan Institute of Medical Research

384 Victoria St., Darlinghurst, NSW, Australia 2010

E: t.peters using garvan.org.au<mailto:t.peters using garvan.org.au> | W: http://www.garvan.org.au | P: +612 9295 8325

NOTICE
Please consider the environment before printing this email. This message and any attachments are intended for the addressee named and may contain legally privileged/confidential/copyright information. If you are not the intended recipient, you should not read, use, disclose, copy or distribute this communication. If you have received this message in error please notify us at once by return email and then delete both messages. We accept no liability for the distribution of viruses or similar in electronic communications. This notice should not be removed.

_______________________________________________
Bioc-devel using r-project.org<mailto:Bioc-devel using r-project.org> mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel

--
Best,
Kasper

--

===================

Tim Peters, PhD

Bioinformatics Research Officer | Immunogenomics Laboratory | Immunology Division

Garvan Institute of Medical Research

384 Victoria St., Darlinghurst, NSW, Australia 2010

E: t.peters using garvan.org.au<mailto:t.peters using garvan.org.au> | W: http://www.garvan.org.au | P: +612 9295 8325

NOTICE
Please consider the environment before printing this email. This message and any attachments are intended for the addressee named and may contain legally privileged/confidential/copyright information. If you are not the intended recipient, you should not read, use, disclose, copy or distribute this communication. If you have received this message in error please notify us at once by return email and then delete both messages. We accept no liability for the distribution of viruses or similar in electronic communications. This notice should not be removed.

	[[alternative HTML version deleted]]