[Bioc-devel] shiny, Bioconductor, and reproducible research

[Tim Triche, Jr.]

So here is something I came across in my code:



--t

On Jul 29, 2013, at 2:08 PM, Kasper Daniel Hansen <kasperdanielhansen at gmail.com> wrote:

> My perspective,
> 
> Clearly we are very interested in supporting reproducible research in
> Bioconductor.  Indeed, it is a core mission of the project.  However, I
> feel that Dan is placing a very high burden of proof on shiny apps in his
> email, since it essentially says "because it is possible to potentially
> conduct non-reproducible research in shiny, we should not allow it".  The
> logical conclusion of such a statement, is that we should never allow GUIs
> nor interactive code.
> 
> Without having a lot of experience with shiny, I am mostly excited because
> of the potential for (semi) interactive graphics, especially for exploring
> your data (here I am like very biased since that is what we are using it
> for right now).  For such an application, there should not be any
> modification of the user's R objects.  It is true that an interactive
> exploration of your data might be considered non-reproducible, but that is
> also the case if I inspects 100's of pdfs.
> 
> In my opinion we should try to get people to adhere to best practices, and
> (once we have something) we could provide example code for these practices.
> Perhaps a starting point would be a function to tag all objects in the
> user environment and keep track of whether they are modified during the
> shiny session.  In the meantime (unless Michael and Tengfei comes up with
> an awesome solution in the next week :), I do believe we should consider
> how to host such apps, including how do we do code checking.  Ultimately,
> this would be better for the research community than to force people to
> host it on their own webpages.  If we cannot find a suitable way to do code
> checking or to package up the app, perhaps we should not do it.  But that
> would be a better reason than the potential for writing non-reproducible
> code.
> 
> Best,
> Kasper
> 
> 
> On Sat, Jul 27, 2013 at 9:17 PM, Levi Waldron <lwaldron at hsph.harvard.edu>wrote:
> 
>> I agree with Dan on the need to be able to "save" analysis done in a Shiny
>> Bioconductor app, allowing someone later to reload the same state, and to
>> inspect a session and see what was done.  Perhaps "save" and "load" buttons
>> should be present in the user interface of any Bioconductor Shiny app that
>> is not "read-only".
>> 
>> If interactive user input can be saved (as an .RData file?), could that
>> file be used as input for unit tests either the usual console way or
>> something like an http POST?
>> 
>> -Levi
>> 
>> 
>> 
>> 
>> 
>> On Fri, Jul 26, 2013 at 5:42 PM, Michael Lawrence <
>> lawrence.michael at gene.com
>>> wrote:
>> 
>>> An answer to both of these issues is the use of R code. It should be
>>> possible to drive shiny apps through the server, without the user's code
>>> knowing the difference. And you've already thought of recording user
>>> actions as R code. This would be an application of the command pattern,
>> as
>>> with undo/redo stacks, except each object boils down to a
>>> function/expression (Tengfei and I have a CRAN package called commandr
>>> related to this). Someone should try to make a prototype to explore these
>>> issues then we can start thinking about common infrastructure.
>>> 
>>> 
>>> On Fri, Jul 26, 2013 at 12:23 PM, Dan Tenenbaum <dtenenba at fhcrc.org>
>>> wrote:
>>> 
>>>> Hi everyone,
>>>> 
>>>> There was a lot of talk about shiny at BioC2013 and it seems like a
>>>> lot of people are using it for lots of different cool stuff.
>>>> 
>>>> Now we are at the point where some contributors want to add shiny apps
>>>> to Bioconductor as packages.
>>>> 
>>>> We had some discussion internally and decided that there are a few
>>>> issues we wanted to discuss with BioC developers as well as some of
>>>> the shiny developers. Please feel free to share your thoughts.
>>>> 
>>>> 1) Testing shiny apps
>>>> 
>>>> Typically, bioconductor packages have man page examples, vignettes,
>>>> and (sometimes) unit tests, so when we build the packages every day on
>>>> our build system, the code in all of these is evaluated and we have
>>>> some idea of whether the package is working as it's supposed to.
>>>> 
>>>> I'm not clear on how to do similar testing of a shiny application.
>>>> Since launching a shiny app takes away the R console (until the app is
>>>> closed), shiny apps should probably not be launched in example or test
>>>> code (unless interactive() is TRUE).
>>>> 
>>>> Do the shiny folks (or anyone else) have thoughts on testing shiny
>> apps?
>>>> 
>>>> 2) Reproducible research
>>>> 
>>>> Reproducible research is really important in our community, but shiny
>>>> apps are sort of a black box as far as reproducibility is concerned.
>>>> 
>>>> If shiny apps are "read-only" (that is, if they are just used to view
>>>> an object) then this is not really a problem. But shiny has the
>>>> ability now to change objects back in your R session, so we need to be
>>>> able to track what is done inside the shiny app.
>>>> 
>>>> shiny apps can return some sort of object that tells you what was done
>>>> in the shiny session (insofar as it modifies any objects in the
>>>> users's session) and this object can provide a way to reproduce those
>>>> steps without the shiny app. So maybe the object would consist of
>>>> lines of code. This means that if there is a shiny app, the package
>>>> author must also provide ways to do the same transformations on
>>>> objects without shiny.
>>>> 
>>>> There are probably other approaches as well. But the end goal would be
>>>> a scriptable, reproducible shiny session.
>>>> 
>>>> Another thought was that a shiny app could emit some sort of state
>>>> object, and then be restarted with that object.
>>>> 
>>>> Not sure if the place for such infrastructure is inside shiny itself,
>>>> or perhaps in a BioC infrastructure package.
>>>> 
>>>> What are your thoughts?
>>>> Dan
>>>> 
>>>> _______________________________________________
>>>> Bioc-devel at r-project.org mailing list
>>>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>>> 
>>>        [[alternative HTML version deleted]]
>>> 
>>> _______________________________________________
>>> Bioc-devel at r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>> 
>> 
>> 
>> --
>> Levi Waldron
>> Post-doctoral fellow
>> Department of Biostatistics, Harvard School of Public Health
>> Department of Biostatistics and Computational Biology, Dana-Farber Cancer
>> Institute
>> Building 1, room 412C
>> 655 Huntington Avenue
>> Boston, Massachusetts 02115
>> mobile: (617) 851-6849
>> fax: (617) 432-5619
>> http://www.hsph.harvard.edu/research/levi-waldron/
>> 
>>        [[alternative HTML version deleted]]
>> 
>> _______________________________________________
>> Bioc-devel at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
> 
>    [[alternative HTML version deleted]]
> 
> _______________________________________________
> Bioc-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/bioc-devel