[Bioc-devel] Package submission with strong external dependency during build/check [take 2]

[Régis Ongaro-Carcy]

Thanks for your reponse Martin. 

I am not testing ElasticSearch (ES) but my package is using it intensely. For a bit more of details: it is a software package that manage, at high level, an easy push/pull mecanism with ES. The end goal is to make a use of ES for fast searches in datasets since ES query language possesses many interesting features. I also included import functions for biological features such as rtracklayer and BioStrings, and base statistical function applicable on massive datasets directly on ES.

I choose to use an ES docker because the ES API has regular breaking changes (reponse structure changes) so it is easy to test against several docker with different ES versions. I want to be sure before adapting because using mocks will imply the need to implement each method multiplied by the number of ES versions (breaking changes), and ES can be pretty verbose depending on the datasets. In that case, I will need to start a webmockr service that contains all stubs mimicing all ES responses. Should it be a correct workaround for examples, vignettes ans tests?

Régis Ongaro

________________________________________
From: Martin Morgan <mtmorgan.bioc using gmail.com>
Sent: Thursday, July 16, 2020 15:31
To: Régis Ongaro-Carcy; bioc-devel using r-project.org
Subject: Re: [Bioc-devel] Package submission with strong external dependency during build/check [take 2]

[Externe UL*]

This set up is not possible in Bioconductor.

I suggest that you explore the use of 'mocks' so that your code thinks, during testing and examples, that it is querying an elasticsearch server, but is actually checking that it can send the appropriate message / interpret the expected response -- you're testing your R code, not the elasticsearch implementation.

There is some more information at

  https://books.ropensci.org/http-testing/

but perhaps others have experience with this approach?

Martin Morgan


On 7/16/20, 10:26 AM, "Bioc-devel on behalf of Régis Ongaro-Carcy" <bioc-devel-bounces using r-project.org on behalf of regis.ongaro-carcy.1 using ulaval.ca> wrote:

    Hello Bioconductor,

    I would like to submit my software package to Bioconductor and I have some questions.

    My package strongly depends on ElasticSearch (ES, https://www.elastic.co/elasticsearch) as database and search engine. It sends data to, and retrieves data from ES.
    Thus, a big part of the build/check is based on the fact that there is an empty ES available for building/checking the examples, manual, vignettes and tests.

    The only thing actually required on the R-side is two variables in the env to target ES IP address and port (user/pwd optionally). I mainly use the docker version of ES on my computer so that is fairly easy to run and interact with it.

    So, my questions are: is it possible to run a disposable container alongside the Bioconductor build/check system run? Do some packages already implemented it?

    The container should be discarded afterward. This setup works fine on my computer and I am wondering how it could be achieved on Bioconductor's side and if there is another easier/more efficient/more suitable solution. I am hitting the time limit on travis when checking tests without longtests, so a ES container with longtests would also be needed.

    Best regards,
    Régis Ongaro

    P.S.: many thanks Martin Morgan, I spent some time trying to find the right option in the university MS outlook to turn the html default mail into plain text. Hoping it will work this time.
    _______________________________________________
    Bioc-devel using r-project.org mailing list
    https://stat.ethz.ch/mailman/listinfo/bioc-devel
*ATTENTION : L’émetteur de ce courriel est externe à l’Université Laval.
Évitez de cliquer sur un hyperlien, d’ouvrir une pièce jointe ou de transmettre des informations si vous ne connaissez pas l’expéditeur du courriel. En cas de doute, contactez l’équipe de soutien informatique de votre unité ou hameconnage using ulaval.ca.
________________________________