[Bioc-devel] Should GenomicFeatures really depend pn RMySQL ? Is it time to migrate to RMariaDB ?

Vincent Carey stvjc at channing.harvard.edu
Fri Feb 2 16:22:51 CET 2018 

It is important but not completely clear how to address.

I think it has been written that bioc package checking should make minimal
use of the internet, and most
checking concerning manipulation of net-derived objects should use "mocks".

One possible approach is to test with calls to the external service that
are minimally sufficient to assess
whether key expectations are met.  If not, fail or warn with detailed
indication.  All other tests are devised to
use stored objects that represent service products.   These products are
presumed valid on the basis
of the tests of expectations.

On Fri, Feb 2, 2018 at 9:16 AM, Wolfgang Huber <wolfgang.huber at embl.de>
wrote:

> Thanks Mike and Hervé!
>
> Somehow, errors in examples that are caused by the state (or absence) of
> things on the internet should have a different status in my view than ones
> that reflect local state of R and packages - and distinguishing this could
> make maintenance of the package corpus & builds more modular.
>
> But I'm not entirely confident whether this view is coherent or
> practicable, what do others think?
>
>         Wolfgang
>
> 2.2.18 12:19, Hervé Pagès scripsit:
>
> Hi Wolfgang, Mike,
>>
>> We didn't have a successful build of GenomicFeatures in devel
>> for many days because of all kinds of problems with the examples
>> that try to access the Ensembl marts. The latest of which being:
>>
>>    https://support.bioconductor.org/p/105565/
>>
>> (which also affects GenomicFeatures in release).
>>
>> Thanks Mike for looking at the timeout issue (FWIW I can reproduce
>> it). I thought this timeout was a new setting on the Ensembl mart
>> server side :-)
>>
>> In GenomicFeatures 1.31.7 I replaced RMySQL with RMariaDB:
>>
>>
>> https://github.com/Bioconductor/GenomicFeatures/commit/08dd2
>> 4296d94ef31b5f5685240b871c79a160e91
>>
>> I also made another small speed improvement to makeTxDbFromUCSC().
>>
>> H.
>>
>>
>> On 02/02/2018 02:33 AM, Mike Smith wrote:
>>
>>> The error TxDbFromBiomart looks like it might be related to a biomaRt
>>> change I made recently to submit queries using httr rather than RCurl.
>>> Others have reported something similar (e.g
>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__support
>>> .bioconductor.org_p_104502_&d=DwIGaQ&c=eRAMFD45gAfqt84VtBcfh
>>> Q&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=nTaIFim9PZ
>>> DsGaU0xtK3D4cw4UKBrPzen-jCDV-ppDE&s=k3wnoKsf0v7Biv9DKASzF3h
>>> SAzhO02fR9UMIiznM2EA&e=) and I raised the timeout from
>>> 10 to 60 seconds.  I guess with the old version it was even longer than
>>> that.
>>>
>>> I haven't been able to recreate the problem at my end, I think the time
>>> taken is related as much to the internet connection as to the query, but
>>> I'll take a look at the failing example to see if I can shed any more
>>> light
>>> on it.
>>>
>>> Mike
>>>
>>> On 2 February 2018 at 10:41, Wolfgang Huber <wolfgang.huber at embl.de>
>>> wrote:
>>>
>>> Thanks Hervé!
>>>>
>>>> This seems to take a long time to propagate. As of now,
>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__biocond
>>>> uctor.org_packages_devel_bioc_html_GenomicFeatures.html&d=DwIGaQ&c=
>>>> eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJ
>>>> KaaPhzWA&m=nTaIFim9PZDsGaU0xtK3D4cw4UKBrPzen-jCDV-ppDE&s=
>>>> 09LGf-6SKIrEdIR2hwC4Vq1YOBj-3PrsuCanwxv9AMs&e=
>>>> still shows 1.31.3.
>>>>
>>>> ( Btw, there's also an error in the build report which seems to come
>>>> from
>>>> a rather excessive-looking example in the makeTxDbFromBiomart man page -
>>>> that maybe could better live in a vignette, not least for reducing
>>>> brittleness? )
>>>>
>>>>          Best wishes
>>>>                  Wolfgang
>>>>
>>>> 30.1.18 19:00, Hervé Pagès scripsit:
>>>>
>>>> This is done in GenomicFeatures 1.31.6.
>>>>
>>>>>
>>>>> Note that I also made a few changes to makeTxDbFromUCSC() to make it
>>>>> a little bit faster (about 2x).
>>>>>
>>>>> @Kasper: moving the makeTxDb* functions to a GenomicFeaturesBuildTools
>>>>> or GenomicFeaturesForge package is maybe an idea to explore...
>>>>>
>>>>> H.
>>>>>
>>>>> On 01/26/2018 06:09 PM, Kasper Daniel Hansen wrote:
>>>>>
>>>>> As an alternative to Suggests, perhaps make a
>>>>>> GenomicFeaturesBuildTools.
>>>>>>    Not sure if it is better or worse, just different
>>>>>>
>>>>>> On Fri, Jan 26, 2018 at 2:39 PM, Wolfgang Huber <
>>>>>> wolfgang.huber at embl.de>
>>>>>> wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>> 26.1.18 14:59, Martin Morgan scripsit:
>>>>>>>
>>>>>>> On 01/24/2018 03:38 PM, Wolfgang Huber wrote:
>>>>>>>
>>>>>>>>
>>>>>>>> GenomicFeatures_1.31.3 imports RMySQL.
>>>>>>>>
>>>>>>>>>
>>>>>>>>> I'm having great trouble installing RMySQL from source on a recent
>>>>>>>>> MacOS
>>>>>>>>> (10.13.3) with homebrew.
>>>>>>>>>
>>>>>>>>> The package's homepage says "The 'RMySQL' package contains an old
>>>>>>>>> implementation based on legacy code from S-PLUS which being phased
>>>>>>>>> out. A
>>>>>>>>> modern 'MySQL' client based on 'Rcpp' is available from the
>>>>>>>>> 'RMariaDB'
>>>>>>>>> package" https://urldefense.proofpoint.
>>>>>>>>> com/v2/url?u=https-3A__cran.r-
>>>>>>>>> 2Dproject.org_web_packages_RMySQL_index.html&d=DwICAg&c=eRAM
>>>>>>>>> FD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaP
>>>>>>>>> hzWA&m=jUD3XGyxapAK8nfWvxbR_9Rz7RBxP592NO0nWz_wyr0&s=0neTB2B
>>>>>>>>> Jghm4wF-LnEZ9V9sV_AbF1botcOeTb_bWk-w&e=
>>>>>>>>>
>>>>>>>>> So is it time to heed that advice and migrate GenomicFeatures to
>>>>>>>>> RMariaDB ?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Out of curiosity, is MariaDB easier to install on your system? Its
>>>>>>>> system
>>>>>>>> dependencies are described at https://urldefense.proofpoint.
>>>>>>>> com/v2/url?u=https-3A__urldefense.proofpoint&d=DwIGaQ&c=eRAM
>>>>>>>> FD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaP
>>>>>>>> hzWA&m=nTaIFim9PZDsGaU0xtK3D4cw4UKBrPzen-jCDV-ppDE&s=t1JSruT
>>>>>>>> yZxTGystW_bcaK5XfEnAVD0dTfxdLOO3boT4&e=.
>>>>>>>> com/v2/url?u=https-3A__CRAN.R-2Dproject.org_package-3DRMaria
>>>>>>>> DB&d=DwICAg&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJ
>>>>>>>> YbW0WYiZvSXAJJKaaPhzWA&m=jUD3XGyxapAK8nfWvxbR_9Rz7RBxP592NO0
>>>>>>>> nWz_wyr0&s=55tLfrRpaigMug-pW7L1AhpsD1A830xw1f9d1FZEANo&e=
>>>>>>>>
>>>>>>>>
>>>>>>>> I have no problems installing RMariaDB on MacOS (10.13.3) on an
>>>>>>> R-devel
>>>>>>> from source, after installing mariadb-connector-c with homebrew.
>>>>>>>
>>>>>>> OTOH, I have not figured out a way to install RMySQL neither on my
>>>>>>> R-devel
>>>>>>> from source (various complaints about missing .h files) nor on a
>>>>>>> binary
>>>>>>> R
>>>>>>> 3.4.2 with the binary package download (complaints about missing
>>>>>>> system
>>>>>>> libraries / wrong versions).
>>>>>>>
>>>>>>> Thanks and kind regards
>>>>>>>                   Wolfgang
>>>>>>>
>>>>>>> FWIW MySQL is a relatively recent addition as a dependency to
>>>>>>>
>>>>>>> GenomicFeatures; it enables `makeTxDbFromEnsembl()`, which is
>>>>>>>> probably
>>>>>>>> a
>>>>>>>> much more stable solution than `makeTxDbFromBiomart()`. On the other
>>>>>>>> had
>>>>>>>> Johannes does an excellent job on the ensembldb packages, so perhaps
>>>>>>>> this
>>>>>>>> code could really be conditional with the RMySQL dependency moved to
>>>>>>>> Suggests:
>>>>>>>>
>>>>>>>> Martin
>>>>>>>>
>>>>>>>>
>>>>>>>> With thanks in advance-
>>>>>>>>
>>>>>>>>> Wolfgang
>>>>>>>>>
>>>>>>>>> -------
>>>>>>>>> Wolfgang Huber
>>>>>>>>> Principal Investigator, EMBL Senior Scientist
>>>>>>>>> European Molecular Biology Laboratory (EMBL)
>>>>>>>>> Heidelberg, Germany
>>>>>>>>>
>>>>>>>>> wolfgang.huber at embl.de
>>>>>>>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.hube
>>>>>>>>> r.embl.de&d=DwICAg&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWd
>>>>>>>>> GbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=jUD3XGyxapAK8nfWvxbR_9Rz7RBx
>>>>>>>>> P592NO0nWz_wyr0&s=Vp0j7sKJXiZSWHn3LVWSECRS-f-AFZZSSh0mq5rvJ-0&e=
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Bioc-devel at r-project.org mailing list
>>>>>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.et
>>>>>>>>> hz.ch_mailman_listinfo_bioc-2Ddevel&d=DwICAg&c=eRAMFD45gAfqt
>>>>>>>>> 84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=jU
>>>>>>>>> D3XGyxapAK8nfWvxbR_9Rz7RBxP592NO0nWz_wyr0&s=0h8ajRHX9SrbTr3P
>>>>>>>>> P9GKMvt8hoBxj6c4Op-wxmvtyLA&e=
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>> This email message may contain legally privileged and/or
>>>>>>>> confidential
>>>>>>>> information.  If you are not the intended recipient(s), or the
>>>>>>>> employee or
>>>>>>>> agent responsible for the delivery of this message to the intended
>>>>>>>> recipient(s), you are hereby notified that any disclosure, copying,
>>>>>>>> distribution, or use of this email message is prohibited.  If you
>>>>>>>> have
>>>>>>>> received this message in error, please notify the sender
>>>>>>>> immediately by
>>>>>>>> e-mail and delete this email message from your computer. Thank you.
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>> With thanks in advance-
>>>>>>> Wolfgang
>>>>>>>
>>>>>>> -------
>>>>>>> Wolfgang Huber
>>>>>>> Principal Investigator, EMBL Senior Scientist
>>>>>>> European Molecular Biology Laboratory (EMBL)
>>>>>>> Heidelberg, Germany
>>>>>>>
>>>>>>> wolfgang.huber at embl.de
>>>>>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.hube
>>>>>>> r.embl.de&d=DwICAg&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWd
>>>>>>> GbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=jUD3XGyxapAK8nfWvxbR_9Rz7RBx
>>>>>>> P592NO0nWz_wyr0&s=Vp0j7sKJXiZSWHn3LVWSECRS-f-AFZZSSh0mq5rvJ-0&e=
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Bioc-devel at r-project.org mailing list
>>>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.et
>>>>>>> hz.ch_mailman_listinfo_bioc-2Ddevel&d=DwICAg&c=eRAMFD45gAfqt
>>>>>>> 84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=jU
>>>>>>> D3XGyxapAK8nfWvxbR_9Rz7RBxP592NO0nWz_wyr0&s=0h8ajRHX9SrbTr3P
>>>>>>> P9GKMvt8hoBxj6c4Op-wxmvtyLA&e=
>>>>>>>
>>>>>>>
>>>>>>>      [[alternative HTML version deleted]]
>>>>>>
>>>>>> _______________________________________________
>>>>>> Bioc-devel at r-project.org mailing list
>>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.et
>>>>>> hz.ch_mailman_listinfo_bioc-2Ddevel&d=DwICAg&c=eRAMFD45gAfqt
>>>>>> 84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=jU
>>>>>> D3XGyxapAK8nfWvxbR_9Rz7RBxP592NO0nWz_wyr0&s=0h8ajRHX9SrbTr3P
>>>>>> P9GKMvt8hoBxj6c4Op-wxmvtyLA&e=
>>>>>>
>>>>>>
>>>>>>
>>>>> --
>>>> With thanks in advance-
>>>> Wolfgang
>>>>
>>>> -------
>>>> Wolfgang Huber
>>>> Principal Investigator, EMBL Senior Scientist
>>>> European Molecular Biology Laboratory (EMBL)
>>>> Heidelberg, Germany
>>>>
>>>> wolfgang.huber at embl.de
>>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.hube
>>>> r.embl.de&d=DwIGaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWd
>>>> GbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=nTaIFim9PZDsGaU0xtK3D4cw4UKB
>>>> rPzen-jCDV-ppDE&s=y0PiNkZtZmjkVXUcY4uOoz6Dism0uJaGHU8tDbtbqXU&e=
>>>>
>>>>
>>>> _______________________________________________
>>>> Bioc-devel at r-project.org mailing list
>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.et
>>>> hz.ch_mailman_listinfo_bioc-2Ddevel&d=DwIGaQ&c=eRAMFD45gAfqt
>>>> 84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=nT
>>>> aIFim9PZDsGaU0xtK3D4cw4UKBrPzen-jCDV-ppDE&s=by9IWiqpk7ksvyc9
>>>> 5kowdGzkGOjTkQpOQqU4esDz_Dk&e=
>>>>
>>>>
>>>     [[alternative HTML version deleted]]
>>>
>>> _______________________________________________
>>> Bioc-devel at r-project.org mailing list
>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.et
>>> hz.ch_mailman_listinfo_bioc-2Ddevel&d=DwIGaQ&c=eRAMFD45gAfqt
>>> 84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=nT
>>> aIFim9PZDsGaU0xtK3D4cw4UKBrPzen-jCDV-ppDE&s=by9IWiqpk7ksvyc9
>>> 5kowdGzkGOjTkQpOQqU4esDz_Dk&e=
>>>
>>>
>>
> --
> With thanks in advance-
> Wolfgang
>
> -------
> Wolfgang Huber
> Principal Investigator, EMBL Senior Scientist
> European Molecular Biology Laboratory (EMBL)
> Heidelberg, Germany
>
> wolfgang.huber at embl.de
> http://www.huber.embl.de
>
>
> _______________________________________________
> Bioc-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>

	[[alternative HTML version deleted]]

More information about the Bioc-devel mailing list