[Bioc-devel] Should GenomicFeatures really depend pn RMySQL ? Is it time to migrate to RMariaDB ?

Fri Feb 2 13:19:41 CET 2018

Hi Wolfgang, Mike,

We didn't have a successful build of GenomicFeatures in devel
for many days because of all kinds of problems with the examples
that try to access the Ensembl marts. The latest of which being:

   https://support.bioconductor.org/p/105565/

(which also affects GenomicFeatures in release).

Thanks Mike for looking at the timeout issue (FWIW I can reproduce
it). I thought this timeout was a new setting on the Ensembl mart
server side :-)

In GenomicFeatures 1.31.7 I replaced RMySQL with RMariaDB:

https://github.com/Bioconductor/GenomicFeatures/commit/08dd24296d94ef31b5f5685240b871c79a160e91

I also made another small speed improvement to makeTxDbFromUCSC().

H.

On 02/02/2018 02:33 AM, Mike Smith wrote:
> The error TxDbFromBiomart looks like it might be related to a biomaRt
> change I made recently to submit queries using httr rather than RCurl.
> Others have reported something similar (e.g
> https://urldefense.proofpoint.com/v2/url?u=https-3A__support.bioconductor.org_p_104502_&d=DwIGaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=nTaIFim9PZDsGaU0xtK3D4cw4UKBrPzen-jCDV-ppDE&s=k3wnoKsf0v7Biv9DKASzF3hSAzhO02fR9UMIiznM2EA&e=) and I raised the timeout from
> 10 to 60 seconds.  I guess with the old version it was even longer than
> that.
> 
> I haven't been able to recreate the problem at my end, I think the time
> taken is related as much to the internet connection as to the query, but
> I'll take a look at the failing example to see if I can shed any more light
> on it.
> 
> Mike
> 
> On 2 February 2018 at 10:41, Wolfgang Huber <wolfgang.huber at embl.de> wrote:
> 
>> Thanks Hervé!
>>
>> This seems to take a long time to propagate. As of now,
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__bioconductor.org_packages_devel_bioc_html_GenomicFeatures.html&d=DwIGaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=nTaIFim9PZDsGaU0xtK3D4cw4UKBrPzen-jCDV-ppDE&s=09LGf-6SKIrEdIR2hwC4Vq1YOBj-3PrsuCanwxv9AMs&e=
>> still shows 1.31.3.
>>
>> ( Btw, there's also an error in the build report which seems to come from
>> a rather excessive-looking example in the makeTxDbFromBiomart man page -
>> that maybe could better live in a vignette, not least for reducing
>> brittleness? )
>>
>>          Best wishes
>>                  Wolfgang
>>
>> 30.1.18 19:00, Hervé Pagès scripsit:
>>
>> This is done in GenomicFeatures 1.31.6.
>>>
>>> Note that I also made a few changes to makeTxDbFromUCSC() to make it
>>> a little bit faster (about 2x).
>>>
>>> @Kasper: moving the makeTxDb* functions to a GenomicFeaturesBuildTools
>>> or GenomicFeaturesForge package is maybe an idea to explore...
>>>
>>> H.
>>>
>>> On 01/26/2018 06:09 PM, Kasper Daniel Hansen wrote:
>>>
>>>> As an alternative to Suggests, perhaps make a GenomicFeaturesBuildTools.
>>>>    Not sure if it is better or worse, just different
>>>>
>>>> On Fri, Jan 26, 2018 at 2:39 PM, Wolfgang Huber <wolfgang.huber at embl.de>
>>>> wrote:
>>>>
>>>>
>>>>>
>>>>> 26.1.18 14:59, Martin Morgan scripsit:
>>>>>
>>>>> On 01/24/2018 03:38 PM, Wolfgang Huber wrote:
>>>>>>
>>>>>> GenomicFeatures_1.31.3 imports RMySQL.
>>>>>>>
>>>>>>> I'm having great trouble installing RMySQL from source on a recent
>>>>>>> MacOS
>>>>>>> (10.13.3) with homebrew.
>>>>>>>
>>>>>>> The package's homepage says "The 'RMySQL' package contains an old
>>>>>>> implementation based on legacy code from S-PLUS which being phased
>>>>>>> out. A
>>>>>>> modern 'MySQL' client based on 'Rcpp' is available from the 'RMariaDB'
>>>>>>> package" https://urldefense.proofpoint.com/v2/url?u=https-3A__cran.r-
>>>>>>> 2Dproject.org_web_packages_RMySQL_index.html&d=DwICAg&c=eRAM
>>>>>>> FD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaP
>>>>>>> hzWA&m=jUD3XGyxapAK8nfWvxbR_9Rz7RBxP592NO0nWz_wyr0&s=0neTB2B
>>>>>>> Jghm4wF-LnEZ9V9sV_AbF1botcOeTb_bWk-w&e=
>>>>>>>
>>>>>>> So is it time to heed that advice and migrate GenomicFeatures to
>>>>>>> RMariaDB ?
>>>>>>>
>>>>>>>
>>>>>> Out of curiosity, is MariaDB easier to install on your system? Its
>>>>>> system
>>>>>> dependencies are described at https://urldefense.proofpoint.com/v2/url?u=https-3A__urldefense.proofpoint&d=DwIGaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=nTaIFim9PZDsGaU0xtK3D4cw4UKBrPzen-jCDV-ppDE&s=t1JSruTyZxTGystW_bcaK5XfEnAVD0dTfxdLOO3boT4&e=.
>>>>>> com/v2/url?u=https-3A__CRAN.R-2Dproject.org_package-3DRMaria
>>>>>> DB&d=DwICAg&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJ
>>>>>> YbW0WYiZvSXAJJKaaPhzWA&m=jUD3XGyxapAK8nfWvxbR_9Rz7RBxP592NO0
>>>>>> nWz_wyr0&s=55tLfrRpaigMug-pW7L1AhpsD1A830xw1f9d1FZEANo&e=
>>>>>>
>>>>>>
>>>>> I have no problems installing RMariaDB on MacOS (10.13.3) on an R-devel
>>>>> from source, after installing mariadb-connector-c with homebrew.
>>>>>
>>>>> OTOH, I have not figured out a way to install RMySQL neither on my
>>>>> R-devel
>>>>> from source (various complaints about missing .h files) nor on a binary
>>>>> R
>>>>> 3.4.2 with the binary package download (complaints about missing system
>>>>> libraries / wrong versions).
>>>>>
>>>>> Thanks and kind regards
>>>>>                   Wolfgang
>>>>>
>>>>> FWIW MySQL is a relatively recent addition as a dependency to
>>>>>
>>>>>> GenomicFeatures; it enables `makeTxDbFromEnsembl()`, which is probably
>>>>>> a
>>>>>> much more stable solution than `makeTxDbFromBiomart()`. On the other
>>>>>> had
>>>>>> Johannes does an excellent job on the ensembldb packages, so perhaps
>>>>>> this
>>>>>> code could really be conditional with the RMySQL dependency moved to
>>>>>> Suggests:
>>>>>>
>>>>>> Martin
>>>>>>
>>>>>>
>>>>>> With thanks in advance-
>>>>>>> Wolfgang
>>>>>>>
>>>>>>> -------
>>>>>>> Wolfgang Huber
>>>>>>> Principal Investigator, EMBL Senior Scientist
>>>>>>> European Molecular Biology Laboratory (EMBL)
>>>>>>> Heidelberg, Germany
>>>>>>>
>>>>>>> wolfgang.huber at embl.de
>>>>>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.hube
>>>>>>> r.embl.de&d=DwICAg&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWd
>>>>>>> GbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=jUD3XGyxapAK8nfWvxbR_9Rz7RBx
>>>>>>> P592NO0nWz_wyr0&s=Vp0j7sKJXiZSWHn3LVWSECRS-f-AFZZSSh0mq5rvJ-0&e=
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Bioc-devel at r-project.org mailing list
>>>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.et
>>>>>>> hz.ch_mailman_listinfo_bioc-2Ddevel&d=DwICAg&c=eRAMFD45gAfqt
>>>>>>> 84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=jU
>>>>>>> D3XGyxapAK8nfWvxbR_9Rz7RBxP592NO0nWz_wyr0&s=0h8ajRHX9SrbTr3P
>>>>>>> P9GKMvt8hoBxj6c4Op-wxmvtyLA&e=
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> This email message may contain legally privileged and/or confidential
>>>>>> information.  If you are not the intended recipient(s), or the
>>>>>> employee or
>>>>>> agent responsible for the delivery of this message to the intended
>>>>>> recipient(s), you are hereby notified that any disclosure, copying,
>>>>>> distribution, or use of this email message is prohibited.  If you have
>>>>>> received this message in error, please notify the sender immediately by
>>>>>> e-mail and delete this email message from your computer. Thank you.
>>>>>>
>>>>>>
>>>>> --
>>>>> With thanks in advance-
>>>>> Wolfgang
>>>>>
>>>>> -------
>>>>> Wolfgang Huber
>>>>> Principal Investigator, EMBL Senior Scientist
>>>>> European Molecular Biology Laboratory (EMBL)
>>>>> Heidelberg, Germany
>>>>>
>>>>> wolfgang.huber at embl.de
>>>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.hube
>>>>> r.embl.de&d=DwICAg&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWd
>>>>> GbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=jUD3XGyxapAK8nfWvxbR_9Rz7RBx
>>>>> P592NO0nWz_wyr0&s=Vp0j7sKJXiZSWHn3LVWSECRS-f-AFZZSSh0mq5rvJ-0&e=
>>>>>
>>>>> _______________________________________________
>>>>> Bioc-devel at r-project.org mailing list
>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.et
>>>>> hz.ch_mailman_listinfo_bioc-2Ddevel&d=DwICAg&c=eRAMFD45gAfqt
>>>>> 84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=jU
>>>>> D3XGyxapAK8nfWvxbR_9Rz7RBxP592NO0nWz_wyr0&s=0h8ajRHX9SrbTr3P
>>>>> P9GKMvt8hoBxj6c4Op-wxmvtyLA&e=
>>>>>
>>>>>
>>>>      [[alternative HTML version deleted]]
>>>>
>>>> _______________________________________________
>>>> Bioc-devel at r-project.org mailing list
>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.et
>>>> hz.ch_mailman_listinfo_bioc-2Ddevel&d=DwICAg&c=eRAMFD45gAfqt
>>>> 84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=jU
>>>> D3XGyxapAK8nfWvxbR_9Rz7RBxP592NO0nWz_wyr0&s=0h8ajRHX9SrbTr3P
>>>> P9GKMvt8hoBxj6c4Op-wxmvtyLA&e=
>>>>
>>>>
>>>
>> --
>> With thanks in advance-
>> Wolfgang
>>
>> -------
>> Wolfgang Huber
>> Principal Investigator, EMBL Senior Scientist
>> European Molecular Biology Laboratory (EMBL)
>> Heidelberg, Germany
>>
>> wolfgang.huber at embl.de
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.huber.embl.de&d=DwIGaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=nTaIFim9PZDsGaU0xtK3D4cw4UKBrPzen-jCDV-ppDE&s=y0PiNkZtZmjkVXUcY4uOoz6Dism0uJaGHU8tDbtbqXU&e=
>>
>>
>> _______________________________________________
>> Bioc-devel at r-project.org mailing list
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_bioc-2Ddevel&d=DwIGaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=nTaIFim9PZDsGaU0xtK3D4cw4UKBrPzen-jCDV-ppDE&s=by9IWiqpk7ksvyc95kowdGzkGOjTkQpOQqU4esDz_Dk&e=
>>
> 
> 	[[alternative HTML version deleted]]
> 
> _______________________________________________
> Bioc-devel at r-project.org mailing list
> https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_bioc-2Ddevel&d=DwIGaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=nTaIFim9PZDsGaU0xtK3D4cw4UKBrPzen-jCDV-ppDE&s=by9IWiqpk7ksvyc95kowdGzkGOjTkQpOQqU4esDz_Dk&e=
> 

-- 
Hervé Pagès

Program in Computational Biology
Division of Public Health Sciences
Fred Hutchinson Cancer Research Center
1100 Fairview Ave. N, M1-B514
P.O. Box 19024
Seattle, WA 98109-1024

E-mail: hpages at fredhutch.org
Phone:  (206) 667-5791
Fax:    (206) 667-1319