[Bioc-devel] shiny, Bioconductor, and reproducible research

[Diego Diez]

Hi,

I have been following this conversation with a lot of interest. I have
an R package with a shiny interface to another package recently
accepted in bioc-devel (rTRM). I was planning to submit it soon, as
the package is finished.

So my question is, should I hold the submission off until the
questions in this topic are decided or on the contrary submit and
adapt the package later accordingly? If asked about which I prefer,
submit it now would be my choice.

One thing to consider is that from the R package developer point of
view, it probably only makes sense to implement methods for unit
test/etc that do not require the use/knowledge of other language that
is not R. For example, for me one of the main motivations to use shiny
was the fact that I could avoid having to compute things in
Javascript- just could use only R. Maybe it was not implied otherwise
in the thread-just wanted to stress this point. At any rate, any
solution that is R-founded I will be more than willing to follow.
However, I am completely ignorant at this moment about how to
implement unit testing for my shiny package.

One additional comment. In the package I will submit all the code for
the shiny application resides in the inst/ directory (including the
ui.R and server.R files, but also images, html and data files). In R/
there is only a runTRM() function that in turns calls to runApp() with
the correct path. This may not be optimal so any other suggestions on
this regard are welcome.

Best wishes,
Diego

On Tue, Jul 30, 2013 at 6:46 AM, Dan Tenenbaum <dtenenba at fhcrc.org> wrote:
> Hi Winston,
>
>
>
> On Mon, Jul 29, 2013 at 2:39 PM, Winston Chang <winston at stdout.org> wrote:
>> Hi everyone -
>>
>> Great to hear from you all on your thoughts about Shiny. I've tried to
>> answer some of Dan's questions below...
>>
>>
>>>>
>>>> 1) Testing shiny apps
>>>>
>>>> Typically, bioconductor packages have man page examples, vignettes,
>>>> and (sometimes) unit tests, so when we build the packages every day on
>>>> our build system, the code in all of these is evaluated and we have
>>>> some idea of whether the package is working as it's supposed to.
>>>>
>>>> I'm not clear on how to do similar testing of a shiny application.
>>>> Since launching a shiny app takes away the R console (until the app is
>>>> closed), shiny apps should probably not be launched in example or test
>>>> code (unless interactive() is TRUE).
>>>>
>>>> Do the shiny folks (or anyone else) have thoughts on testing shiny apps?
>>
>>
>> With regard to testing, we have unit tests for various components, and we're
>> working on end-to-end tests using Selenium.
>>
>> We do have some unit tests in inst/tests/, but they are entirely on the R
>> side - they don't test interaction with the browser. We have some other
>> tests in inst/tests-js/, which exercise the client (Javascript) side, but
>> they don't test interaction with the server.
>>
>> Of course it's important to have tests for client-server communication.
>> We're also in the process of setting up some end-to-end tests using
>> Selenium, but that's just in the beginning stages right now. Hopefully in
>> the future we'll have more to say about these kinds of tests.
>>
>>
>
> My feeling (and I could be wrong) is that Bioconductor package
> developers will be more concerned with testing the basic logic of
> shiny apps, and that therefore testing only in R would be sufficient
> for the most part. For example, if you could simulate a slider in a
> unit test by just changing a reactive value, it would not be necessary
> to make sure it works in the deployed context (with browser and
> server).
>
> Of course those kinds of tests are also important, but to an extent
> they just ensure that shiny itself works, and so they are more the
> province of the shiny developers.
>
>
>>
>>>>
>>>> 2) Reproducible research
>>>>
>>>> Reproducible research is really important in our community, but shiny
>>>> apps are sort of a black box as far as reproducibility is concerned.
>>>>
>>>> If shiny apps are "read-only" (that is, if they are just used to view
>>>> an object) then this is not really a problem. But shiny has the
>>>> ability now to change objects back in your R session, so we need to be
>>>> able to track what is done inside the shiny app.
>>>>
>>>> shiny apps can return some sort of object that tells you what was done
>>>> in the shiny session (insofar as it modifies any objects in the
>>>> users's session) and this object can provide a way to reproduce those
>>>> steps without the shiny app. So maybe the object would consist of
>>>> lines of code. This means that if there is a shiny app, the package
>>>> author must also provide ways to do the same transformations on
>>>> objects without shiny.
>>
>>
>> If the goal is to allow the same transformations in a Shiny app and in a
>> regular R session, this certainly is possible. You could, for example, have
>> a bunch of inputs in a Shiny app that map directly to arguments in a
>> function call, and save those arguments so that you can later call the
>> function with the same settings. This is something that app authors will
>> have to keep in mind when they write their apps.
>>
>>>>
>>>>
>>>> There are probably other approaches as well. But the end goal would be
>>>> a scriptable, reproducible shiny session.
>>>>
>>>> Another thought was that a shiny app could emit some sort of state
>>>> object, and then be restarted with that object.
>>
>>
>> Capturing and restoring the full state of a Shiny session probably isn't
>> possible, since the full state includes the client, server, and interaction
>> between the two. An app author could save/restore the state of particular
>> objects of interest in a Shiny app, but as of now, this would have to be
>> implemented for each app.
>>
>>
>> I hope this helps!
>
> That's very helpful!
> Thanks,
> Dan
>
>
>> -Winston
>
> _______________________________________________
> Bioc-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/bioc-devel