[Bioc-devel] build machines
Kasper Daniel Hansen
kasperdanielhansen at gmail.com
Fri Apr 27 16:29:50 CEST 2018
Thanks.
I used
/usr/bin/time -v R CMD check ...
to record the max memory usage of the check, which for minfi suggests
around 5Gb. That's a lot.
Best,
Kasper
On Thu, Apr 26, 2018 at 3:02 PM, Hervé Pagès <hpages at fredhutch.org> wrote:
> Hi,
>
> The Linux and Windows builders have 32 GB of RAM, the Mac
> builders 64 Gb.
>
> We also run concurrent R CMD check's.
>
> Here is a summary:
>
> platform RAM nb of nb of concurrent
> (Gb) cores R CMD check's
> ---------------------------------------------------
> Linux (malbecs) 32 20 10
> Windows (tokays) 32 40 24
> Mac (meridas) 64 24 18
>
> That's a lot of concurrency. And there is actually more
> concurrency than that if you consider the fact that many
> packages run things in parallel during 'R CMD check'.
> Some packages are good citizens and limit the number of
> cores to 1 or 2 only during 'R CMD check' but some packages
> try to use all the cores that are available. This will have
> a strong impact on the overall progress of the builds. We
> don't have an easy way to identify those packages right now.
>
> In average, based on our monitoring of the build machines
> things seem to work ok i.e. the concurrent R CMD check's
> don't seem to be competing too much to access resources.
>
> But occasionally there could be too much competition. The
> crazy big elapsed time compared to the relatively short user
> and system times that you observed Kasper are likely to reflect
> that. They could be the sign that the machine ran out of memory
> and started swapping. Not because it happens to your package
> means that your package uses too much memory. The swapping is
> the result of the **cumulated** memory usage of all the
> R CMD check's running at that moment. It could be worth checking
> how much memory R CMD check'ing your package uses though.
>
> The exact set of packages that are being R CMD check'ed at any
> given time is in constant fluctuation and will also vary from
> one day to the other. This would explain why some days you see
> timeouts on some platforms and some days not. We don't have
> an easy way to know which packages were competing with yours
> during the 40 min window that 'R CMD check' was running on your
> package until the build system declared a timeout. It's possible
> (by looking at the BBS logs) but is time consuming.
>
> We should probably add some memory at some point to the Windows
> builders. 32 Gb is not enough to smoothly run 24 R CMD check's
> concurrently.
>
> H.
>
>
> On 04/26/2018 08:48 AM, Diogo FT Veiga wrote:
>
>> Hi Daniel,
>>
>> I have the same issue with my package (new contribution). I just finish
>> reviewing the package with the modifications requested.
>>
>> I am having a warning because R CMD check is exceeding 5 min, but this is
>> happening only in the Windows machine.
>>
>> In Linux and OSX the check finishes in <= 4min, while in Windows takes
>> ~6min.
>>
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__biocondu
>> ctor.org_spb-5Freports_maser-5Fbuildreport-5F20180425114748
>> .html&d=DwICAg&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY
>> _wJYbW0WYiZvSXAJJKaaPhzWA&m=JwiMI-3BEUJlonlihLD_mDkPuEIalQbk
>> rQPSGahzfsg&s=1aMitB3PnVLoojx1lnj_UT_ZeKlJ_OcJDFT4D6BPXow&e=
>>
>>
>> Not sure how to proceed from here.
>>
>> Thanks,
>> Diogo
>>
>>
>> On Thu, Apr 26, 2018 at 9:52 AM, Kasper Daniel Hansen <
>> kasperdanielhansen at gmail.com> wrote:
>>
>> We have been working on the minfi package lately, with a move to a
>>> DelayedArray backend.
>>>
>>> Right now there are some weird issues regarding timings in R CMD check.
>>> Leaving aside the issue that the tests (now disabled) and examples are
>>> too
>>> slow, we get some very weird behaviour.
>>>
>>> An example is the current (soon to be replace) build report of minfi
>>> 1.25.2
>>> which prints
>>>
>>> Examples with CPU or elapsed time > 5s
>>> user system elapsed
>>> preprocessFunnorm 99.388 0.632 148.554
>>> combineArrays 64.104 2.120 68.329
>>> bumphunter 62.540 1.392 64.107
>>> preprocessNoob 43.944 0.016 44.955
>>> preprocessQuantile 33.968 0.064 36.547
>>> getAnnotation 31.072 0.024 31.126
>>> compartments 18.668 0.188 18.871
>>> minfiQC 10.124 6.628 1102.929
>>> getSex 10.536 0.012 10.561
>>> read.metharray 7.504 2.116 82.713
>>> read.metharray.exp 9.076 0.032 10.592
>>> mapToGenome-methods 4.648 0.548 163.648
>>> mdsPlot 0.340 0.204 14.901
>>>
>>>
>>> on Tokay (Linux). Note minfiQC which has an elapsed time which is crazy
>>> high compared to user+system. Previous build report (which I didn't
>>> save)
>>> had a timeout on all platforms with a semingly similar behaviour but with
>>> the getSex function. The code did not change in the meantime. For
>>> today's
>>> build we only see this on Linux, but yesterday all platforms were
>>> affected.
>>>
>>> This is likely to be very hard to debug. But I am thinking memory
>>> issues:
>>> this example requires loading an annotation package and a data package,
>>> both of which are "big". How much RAM does the machines have and are
>>> multiple R CMD check's run concurrently?
>>>
>>> Best,
>>> Kasper
>>>
>>> [[alternative HTML version deleted]]
>>>
>>> _______________________________________________
>>> Bioc-devel at r-project.org mailing list
>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.et
>>> hz.ch_mailman_listinfo_bioc-2Ddevel&d=DwICAg&c=eRAMFD45gAfqt
>>> 84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=Jw
>>> iMI-3BEUJlonlihLD_mDkPuEIalQbkrQPSGahzfsg&s=R1DGN1kNpBZ4ZRBC
>>> TQzDPQlNYapuBNSYB4JTM6tO60w&e=
>>>
>>>
>> [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> Bioc-devel at r-project.org mailing list
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.et
>> hz.ch_mailman_listinfo_bioc-2Ddevel&d=DwICAg&c=eRAMFD45gAfqt
>> 84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=Jw
>> iMI-3BEUJlonlihLD_mDkPuEIalQbkrQPSGahzfsg&s=R1DGN1kNpBZ4ZRBC
>> TQzDPQlNYapuBNSYB4JTM6tO60w&e=
>>
>>
> --
> Hervé Pagès
>
> Program in Computational Biology
> Division of Public Health Sciences
> Fred Hutchinson Cancer Research Center
> 1100 Fairview Ave. N, M1-B514
> P.O. Box 19024
> Seattle, WA 98109-1024
>
> E-mail: hpages at fredhutch.org
> Phone: (206) 667-5791
> Fax: (206) 667-1319
>
[[alternative HTML version deleted]]
More information about the Bioc-devel
mailing list