[Bioc-devel] BiocParallel and Shiny

[Martin Morgan]

OK; I made BiocParallel a little easier to use (bpiterate() can now
take as its first argument a vector; it does not need to have an
'iterator' function). One needs to install the 'devel' version of the
package, currently

BiocManager::install("Bioconductor/BiocParallel")

Here's the complete example

library(shiny)
library(BiocParallel)

server <- function(input, output) {
    output$plot <- renderPlot({
        input$goPlot # Re-run when button is clicked

        withProgress(message = 'Making plot', value = 0, {
            ## Number of times we'll go through the loop
            n <- 100
            dat <- bpiterate(
                seq_len(n),
                function(i) {
                    Sys.sleep(.1)
                    data.frame(x = rnorm(1), y = rnorm(1))
                },
                REDUCE = function(x, y) {
                    incProgress(1/n)
                    rbind(x, y)
                },
                BPPARAM = SnowParam(tasks = n)
            )
        })

        plot(dat$x, dat$y)
    })
}

ui <- shinyUI(basicPage(
    plotOutput('plot', width = "300px", height = "300px"),
    actionButton('goPlot', 'Go plot')
))

shinyApp(ui = ui, server = server)

One thing about Henrik's suggestion is that it uses a number of
packages in addition to BiocParallel; if going that route maybe it
makes sense just to use those different packages rather than trying to
use them AND BiocParallel...

Martin

On Thu, Jul 7, 2022 at 8:32 AM Giulia Pais <giuliapais1 using gmail.com> wrote:
>
> Thanks for the reply,
>
> By just playing a little with some test code it seems like the approach suggested by @Henrik Bengtsson is pretty straightforward so I’ll try with this one,
>
> Thanks again to everyone and have a nice day
>
>
>
> Giulia
>
>
>
> From: Martin Morgan <mtmorgan.bioc using gmail.com>
> Date: Thursday, July 7, 2022 at 14:28
> To: Giulia Pais <giuliapais1 using gmail.com>, Henrik Bengtsson <henrik.bengtsson using gmail.com>
> Cc: bioc-devel using r-project.org <bioc-devel using r-project.org>
> Subject: Re: [Bioc-devel] BiocParallel and Shiny
>
> I think it should be straight-forward to use bpiterate() where the REDUCE function is evaluated locally. It might take a bit of care to make sure that the progress bar is actually informative – generally, increase the `tasks=` argument to the number of time you’d like the progress bar to be updated.
>
>
>
> iter <- function(n) {
>
>     ## return the numbers n, n-1, n-2, … until n < 0, and then return NULL
>
>     force(n)
>
>     function() {
>
>         if (n < 0)
>
>             NULL
>
>         else {
>
>             n <<- n - 1
>
>             n + 1
>
>         }
>
>     }
>
> }
>
>
>
> bpiterate(
>
>     iter(10),
>
>     function(i) { Sys.sleep(i/10); i },
>
>     REDUCE = function(x, y) {
>
>         message(y)
>
>         c(x, y)
>
>     },
>
>     BPPARAM = SnowParam(workers = 2, tasks = 10)
>
> )
>
>
>
> Replace `message()` to do whatever is required to iterate the progress bar in shiny. There is a `reduce.in.order=` argument that might be necessary to obtain results in order; I’m not exactly sure how this would impact a progress bar when, for instance, the early tasks take a long time…
>
>
>
> Martin
>
>
>
>
>
> From: Bioc-devel <bioc-devel-bounces using r-project.org> on behalf of Giulia Pais <giuliapais1 using gmail.com>
> Date: Thursday, July 7, 2022 at 6:11 AM
> To: Henrik Bengtsson <henrik.bengtsson using gmail.com>
> Cc: bioc-devel using r-project.org <bioc-devel using r-project.org>
> Subject: Re: [Bioc-devel] BiocParallel and Shiny
>
> Thank you so much! I'll look into it, have a nice day
>
> Giulia Pais
> ________________________________
> From: Henrik Bengtsson <henrik.bengtsson using gmail.com>
> Sent: Thursday, July 7, 2022 11:59:36 AM
> To: Giulia Pais <giuliapais1 using gmail.com>
> Cc: Vincent Carey <stvjc using channing.harvard.edu>; bioc-devel using r-project.org <bioc-devel using r-project.org>
> Subject: Re: [Bioc-devel] BiocParallel and Shiny
>
> Hi.
>
> If you use the DoparParam backend for BiocParallel with
> doFuture::registerDoFuture(), then you have access to everything
> available in the 'future' ecosystem.  This means you can use any of
> the available future parallel backends, plus near-live progress
> updates via the 'progressr' package. Progress signaled by 'progressr'
> can be rendered in Shiny.
>
> See https://progressr.futureverse.org/#biocparallelbplapply---parallel-lapply,
> for an example how to use BiocParallel on top of the future framework
> together with progress updates signaled by 'progressr'.
>
> See https://progressr.futureverse.org/reference/withProgressShiny.html
> for progressr::withProgressShiny(), which is a plug-in replacement for
> shiny::withProgress().  Run example("withProgressShiny", package =
> "progressr") for an example.
>
> I gave a short presentation on progressr, with a focus on its use
> together with future, at eRum 2020. The talk is available at
> https://www.jottr.org/2020/07/04/progressr-erum2020-slides/.  It might
> give some more insight on how it all works.
>
> Hope this helps,
>
> Henrik
>
> On Thu, Jul 7, 2022 at 11:51 AM Giulia Pais <giuliapais1 using gmail.com> wrote:
> >
> > Hi, thanks for the reply.
> > Yes I�ve looked at this article, it seems to me like I have to access to the �status� of a worker in order to do that, correct?
> > I mean, is there a way through BiocParallel interface in which I can receive a notification when a worker has finished a task? It is not clear to me how BiocParallel manages to update the progress bar internally � I guess the mechanism is similar to the one I described: distribute workload � when a worker has finished a task notify and update progress bar by one tick.
> > I�m honestly not too practical with asynch task management in R, sorry.
> >
> > Thank you
> >
> > From: Vincent Carey <stvjc using channing.harvard.edu>
> > Date: Thursday, July 7, 2022 at 11:40
> > To: Giulia Pais <giuliapais1 using gmail.com>
> > Cc: bioc-devel using r-project.org <bioc-devel using r-project.org>
> > Subject: Re: [Bioc-devel] BiocParallel and Shiny
> > Interesting question.  Have you looked at https://shiny.rstudio.com/articles/progress.html ...?  There is
> > also a file called progress.R in BiocParallel/R folder.  Probably some new code needs to be written
> > to connect information from the BiocParallel activities to the server so incProgress is run as needed.
> >
> > On Thu, Jul 7, 2022 at 4:48 AM Giulia Pais <giuliapais1 using gmail.com<mailto:giuliapais1 using gmail.com>> wrote:
> > Hello,
> > I have a question on the use of BiocParallel with Shiny: I would like to show a progress bar on the UI much like the standard progress bar that can be set in functions like bplapply() � is it possible to do it and how? I haven�t found anything on the topic in the documentation unfortunately.
> >
> > Thanks in advance,
> > Giulia Pais
> >
> >         [[alternative HTML version deleted]]
> >
> > _______________________________________________
> > Bioc-devel using r-project.org<mailto:Bioc-devel using r-project.org> mailing list
> > https://stat.ethz.ch/mailman/listinfo/bioc-devel
> >
> > The information in this e-mail is intended only for th...{{dropped:21}}