My perspective,

Clearly we are very interested in supporting reproducible research in
Bioconductor.  Indeed, it is a core mission of the project.  However, I
feel that Dan is placing a very high burden of proof on shiny apps in his
email, since it essentially says "because it is possible to potentially
conduct non-reproducible research in shiny, we should not allow it".  The
logical conclusion of such a statement, is that we should never allow GUIs
nor interactive code.

Without having a lot of experience with shiny, I am mostly excited because
of the potential for (semi) interactive graphics, especially for exploring
your data (here I am like very biased since that is what we are using it
for right now).  For such an application, there should not be any
modification of the user's R objects.  It is true that an interactive
exploration of your data might be considered non-reproducible, but that is
also the case if I inspects 100's of pdfs.

In my opinion we should try to get people to adhere to best practices, and
(once we have something) we could provide example code for these practices.
 Perhaps a starting point would be a function to tag all objects in the
user environment and keep track of whether they are modified during the
shiny session.  In the meantime (unless Michael and Tengfei comes up with
an awesome solution in the next week :), I do believe we should consider
how to host such apps, including how do we do code checking.  Ultimately,
this would be better for the research community than to force people to
host it on their own webpages.  If we cannot find a suitable way to do code
checking or to package up the app, perhaps we should not do it.  But that
would be a better reason than the potential for writing non-reproducible
code.

Best,
Kasper


On Sat, Jul 27, 2013 at 9:17 PM, Levi Waldron <lwaldron@hsph.harvard.edu>wrote:

> I agree with Dan on the need to be able to "save" analysis done in a Shiny
> Bioconductor app, allowing someone later to reload the same state, and to
> inspect a session and see what was done.  Perhaps "save" and "load" buttons
> should be present in the user interface of any Bioconductor Shiny app that
> is not "read-only".
>
> If interactive user input can be saved (as an .RData file?), could that
> file be used as input for unit tests either the usual console way or
> something like an http POST?
>
> -Levi
>
>
>
>
>
> On Fri, Jul 26, 2013 at 5:42 PM, Michael Lawrence <
> lawrence.michael@gene.com
> > wrote:
>
> > An answer to both of these issues is the use of R code. It should be
> > possible to drive shiny apps through the server, without the user's code
> > knowing the difference. And you've already thought of recording user
> > actions as R code. This would be an application of the command pattern,
> as
> > with undo/redo stacks, except each object boils down to a
> > function/expression (Tengfei and I have a CRAN package called commandr
> > related to this). Someone should try to make a prototype to explore these
> > issues then we can start thinking about common infrastructure.
> >
> >
> > On Fri, Jul 26, 2013 at 12:23 PM, Dan Tenenbaum <dtenenba@fhcrc.org>
> > wrote:
> >
> > > Hi everyone,
> > >
> > > There was a lot of talk about shiny at BioC2013 and it seems like a
> > > lot of people are using it for lots of different cool stuff.
> > >
> > > Now we are at the point where some contributors want to add shiny apps
> > > to Bioconductor as packages.
> > >
> > > We had some discussion internally and decided that there are a few
> > > issues we wanted to discuss with BioC developers as well as some of
> > > the shiny developers. Please feel free to share your thoughts.
> > >
> > > 1) Testing shiny apps
> > >
> > > Typically, bioconductor packages have man page examples, vignettes,
> > > and (sometimes) unit tests, so when we build the packages every day on
> > > our build system, the code in all of these is evaluated and we have
> > > some idea of whether the package is working as it's supposed to.
> > >
> > > I'm not clear on how to do similar testing of a shiny application.
> > > Since launching a shiny app takes away the R console (until the app is
> > > closed), shiny apps should probably not be launched in example or test
> > > code (unless interactive() is TRUE).
> > >
> > > Do the shiny folks (or anyone else) have thoughts on testing shiny
> apps?
> > >
> > > 2) Reproducible research
> > >
> > > Reproducible research is really important in our community, but shiny
> > > apps are sort of a black box as far as reproducibility is concerned.
> > >
> > > If shiny apps are "read-only" (that is, if they are just used to view
> > > an object) then this is not really a problem. But shiny has the
> > > ability now to change objects back in your R session, so we need to be
> > > able to track what is done inside the shiny app.
> > >
> > > shiny apps can return some sort of object that tells you what was done
> > > in the shiny session (insofar as it modifies any objects in the
> > > users's session) and this object can provide a way to reproduce those
> > > steps without the shiny app. So maybe the object would consist of
> > > lines of code. This means that if there is a shiny app, the package
> > > author must also provide ways to do the same transformations on
> > > objects without shiny.
> > >
> > > There are probably other approaches as well. But the end goal would be
> > > a scriptable, reproducible shiny session.
> > >
> > > Another thought was that a shiny app could emit some sort of state
> > > object, and then be restarted with that object.
> > >
> > > Not sure if the place for such infrastructure is inside shiny itself,
> > > or perhaps in a BioC infrastructure package.
> > >
> > > What are your thoughts?
> > > Dan
> > >
> > > _______________________________________________
> > > Bioc-devel@r-project.org mailing list
> > > https://stat.ethz.ch/mailman/listinfo/bioc-devel
> > >
> >
> >         [[alternative HTML version deleted]]
> >
> > _______________________________________________
> > Bioc-devel@r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/bioc-devel
> >
>
>
>
> --
> Levi Waldron
> Post-doctoral fellow
> Department of Biostatistics, Harvard School of Public Health
> Department of Biostatistics and Computational Biology, Dana-Farber Cancer
> Institute
> Building 1, room 412C
> 655 Huntington Avenue
> Boston, Massachusetts 02115
> mobile: (617) 851-6849
> fax: (617) 432-5619
> http://www.hsph.harvard.edu/research/levi-waldron/
>
>         [[alternative HTML version deleted]]
>
> _______________________________________________
> Bioc-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>

	[[alternative HTML version deleted]]

