In this case study, we’ll work through an application of reasonable complexity, turning its slowest operations into futures/promises and modifying all the downstream reactive expressions and outputs to deal with promises.
As a web service increases in popularity, so does the number of rogue scripts that abuse it for no apparent reason.
—Cheng’s Law of Why We Can’t Have Nice Things
I first noticed this in 2011, when the then-new RStudio IDE was starting to gather steam. We had a dashboard that tracked how often RStudio was being downloaded, and the numbers were generally tracking smoothly upward. But once every few months, we’d have huge spikes in the download counts, ten times greater than normal—and invariably, we’d find that all of the unexpected increase could be tracked to one or two IP addresses.
For hours or days we’d be inundated with thousands of downloads per
hour, then just as suddenly, they’d cease. I didn’t know what was
happening then, and I still don’t know today. Was it the world’s least
competent denial-of-service attempt? Did someone write a download script
with an accidental while (TRUE)
around it?
Our application will let us examine downloads from CRAN for this kind of behavior. For any given day on CRAN, we’ll see what the top downloaders are and how they’re behaving.
RStudio maintains the popular 0-Cloud
CRAN mirror, and
the log files it generates are freely available at http://cran-logs.rstudio.com/. Each day is a separate
gzipped CSV file, and each row is a single package download. For
privacy, IP addresses are anonymized by substituting each day’s IP
addresses with unique integer IDs.
Here are the first few lines of http://cran-logs.rstudio.com/2018/2018-05-26.csv.gz :
"date","time","size","r_version","r_arch","r_os","package","version","country","ip_id"
"2018-05-26","20:42:23",450377,"3.4.4","x86_64","linux-gnu","lubridate","1.7.4","NL",1
"2018-05-26","20:42:30",484348,NA,NA,NA,"homals","0.9-7","GB",2
"2018-05-26","20:42:21",98484,"3.3.1","x86_64","darwin13.4.0","miniUI","0.1.1.1","NL",1
"2018-05-26","20:42:27",518,"3.4.4","x86_64","linux-gnu","RCurl","1.95-4.10","US",3
Fortunately for our purposes, there’s no need to analyze these logs at a high level to figure out which days are affected by badly behaved download scripts. These CRAN mirrors are popular enough that, according to Cheng’s Law, there should be plenty of rogue scripts hitting it every day of the year.
The app I built to explore this data, cranwhales, let us examine the behavior of the top downloaders (“whales”) for any given day, at varying levels of detail. You can view this app live at https://gallery.shinyapps.io/cranwhales/, or download and run the code yourself at https://github.com/rstudio/cranwhales.
When the app starts, the “All traffic” tab shows you the number of package downloads per hour for all users vs. whales. In this screenshot, you can see the proportion of files downloaded by the top six downloaders on May 28, 2018. It may not look like a huge fraction at first, but keep in mind, we are only talking about six downloaders out of 52,815 total!
The “Biggest whales” tab simply shows the most prolific downloaders, with their number of downloads performed. Each anonymized IP address has been assigned an easier-to-remember name, and you can also see the country code of the original IP address.
The “Whales by hour” tab shows the hourly download counts for each
whale individually. In this screenshot, you can see that the
Netherlands’ relieved_snake
downloaded at an extremely
consistent rate during the whole day, while the American
curly_capabara
was active only during business hours in
Eastern Standard Time. Still others, like colossal_chicken
out of Hong Kong, was busy all day but at varying rates.
The “Detail View” has perhaps the most illuminating information. It
lets you view every download made by a given whale on the day in
question. The x dimension is time and the y dimension is what package
they downloaded, so you can see at a glance exactly how many packages
were downloaded, and how their various package downloads relate to each
other. In this case, relieved_snake
downloaded 104
different packages, in the same order, continuously, for the entire
day.
Others behave very differently, like freezing_tapir
, who
downloaded devtools
–and only
devtools
–for the whole day, racking up 19,180 downloads
totalling 7.9 gigabytes for that one package alone!
Sadly, the app can’t tell us any more than that–it can’t explain why these downloaders are behaving this way, nor can it tell us their street addresses so that we can send ninjas in black RStudio helicopters to make them stop.
Now that you’ve seen what the app does, let’s talk about how it was implemented, then convert it from sync to async.
The user interface is a pretty typical shinydashboard. It’s important to note that the UI part of the app is entirely agnostic to whether the server is written in the sync or async style; when we port the app to async, we won’t touch the UI at all.
There are two major pieces of input we need from users: what date to examine (this app only lets us look at one day at a time) and how many of the most prolific downloaders to look at. We’ll put these two controls in the dashboard sidebar.
dashboardSidebar(
dateInput("date", "Date", value = Sys.Date() - 2),
numericInput("count", "Show top N downloaders:", 6)
)
(We set date
to two days ago by default, because there’s
some lag between when a day ends and when its logs are published.)
The rest of the UI code is just typical shinydashboard scaffolding,
plus some shinydashboard::valueBoxOutput
s and
plotOutputs
. These are so trivial that they’re hardly worth
talking about, but I’ll include the code here for completeness. Finally,
there’s detailViewUI
, a Shiny
module that just contains more of the same (value boxes and
plots).
dashboardBody(
fluidRow(
tabBox(width = 12,
tabPanel("All traffic",
fluidRow(
valueBoxOutput("total_size", width = 4),
valueBoxOutput("total_count", width = 4),
valueBoxOutput("total_downloaders", width = 4)
),plotOutput("all_hour")
),tabPanel("Biggest whales",
plotOutput("downloaders", height = 500)
),tabPanel("Whales by hour",
plotOutput("downloaders_hour", height = 500)
),tabPanel("Detail view",
detailViewUI("details")
)
)
) )
Based on these inputs and outputs, we’ll write a variety of reactive expressions and output renderers to download, manipulate, and visualize the relevant log data.
The reactive expressions:
data
(eventReactive
): Whenever
input$date
changes, the data
reactive
downloads the full log for that day from http://cran-logs.rstudio.com, and parses it.whales
(reactive
): Reads from
data()
, tallies the number of downloads performed by each
unique IP, and returns a data frame of the top input$count
most prolific downloaders, along with their download counts.whale_downloads
(reactive
): Joins the
data()
and whales()
data frames, to return all
of the details of the cetacean downloads.The whales
reactive expression depends on
data
, and whale_downloads
depends on
data
and whales
.
The outputs in this app are mostly either renderPlot
s
that we populate with ggplot2
, or
shinydashboard::renderValueBox
es. They all rely on one or
more of the reactive expressions we just described. We won’t catalog
them all here, as they’re not individually interesting, but we will look
at some archetypes below.
While this article is specifically about async, this is a good time to remind you that there are lots of ways to improve the performance of a Shiny app. Async is just one tool in the toolbox, and before reaching for that hammer, take a moment to consider your other options:
These options are more generally useful than using async techniques because they can dramatically speed up the performance of an app even if only a single user is using it. While it obviously depends on the particulars of the app itself, a few lines of precomputation or caching logic can often lead to 10X-100X better performance. Async, on the other hand, generally doesn’t help make a single session faster. Instead, it helps a single Shiny process support more concurrent sessions without getting bogged down.
Async can be an essential tool when there is no way around performing expensive tasks (i.e. taking multiple seconds) while the user waits. For example, an app that analyzes any user-specified Twitter profile may get too many unique queries (assuming most people specify their own Twitter handle) for caching to be much help. And applications that invite users to upload their own datasets won’t have an opportunity to do any offline summarizing in advance. If you need to run apps like that and support lots of concurrent users, async can be a huge help.
In that sense, the cranwhales app isn’t a perfect example, because it has lots of opportunities for precomputation and caching that we’ll willfully ignore today so that I can better illustrate the points I want to make about async. When you’re working on your own app, though, please think carefully about all of the different techniques you have for improving performance.
To quote the article Using promises with Shiny, async programming with Shiny boils down to following a few steps:
In this case, the slow operations are easy to identify: the
downloading and parsing that takes place in the data
reactive expression can each take several long seconds.
Converting the download and parsing operations into futures turns out to be the most complicated part of the process, for reasons we’ll get into later.
Assuming we do that successfully, the data
reactive
expression will no longer return a data frame, but a
promise
object (that resolves to a data frame). Since the
whales
and whale_downloads
reactive
expressions both rely on data
, those will both also need to
be converted to read and return promise
objects. And
therefore, because the outputs all rely on one or more reactive
expressions, they will all need to know how to deal with
promise
objects.
Async code is infectious like that; once you turn the heart of your app into a promise, everything downstream must become promise-aware as well, all the way through to the observers and outputs.
With that overview out of the way, let’s dive into the code.
In the sections below, we’ll take a look at the code behind some outputs and reactive expressions. For each element, we’ll look first at the sync version, then the async version.
In some cases, these code snippets may be slightly abridged. See the GitHub repository for the full code.
Until you’ve received an introduction to the %...>%
operator, the async code below will make no sense, so if you haven’t
read An
informal intro to async programming and/or Working
with promises in R, I highly recommend doing so before
continuing!
promises
and future
The first thing we’ll do is load the basic libraries of async programming.
library(promises)
library(future)
plan(multisession)
I originally used multiprocess
but file downloading
inside a future seemed to fail on Mac. (I’ve found that it’s usually not
worth spending a lot of time trying to figure out why
multiprocess
doesn’t work for some specific code; instead,
just use multisession
, since that’s probably going to be
the solution anyway.)
data
reactive: future_promise() all the thingsThe next thing we’ll do is convert the data
event
reactive to use future
for the expensive bits. The original
code looks lke this:
# SYNCHRONOUS version
<- eventReactive(input$date, {
data <- input$date # Example: 2018-05-28
date <- lubridate::year(date) # Example: "2018"
year
<- glue("http://cran-logs.rstudio.com/{year}/{date}.csv.gz")
url <- file.path("data_cache", paste0(date, ".csv.gz"))
path
withProgress(value = NULL, {
if (!file.exists(path)) {
setProgress(message = "Downloading data...")
download.file(url, path)
}
setProgress(message = "Parsing data...")
read_csv(path, col_types = "Dti---c-ci", progress = FALSE)
}) })
(Earlier, I said we wouldn’t take advantage of precomputation or
caching. That wasn’t entirely true; in the code above, we cache the log
files we download in a data_cache
directory. I couldn’t
bring myself to put my internet connection through that level of abuse,
as I knew I’d be running this code thousands of times as I load tested
it.)
For now, we’ll lose the
withProgress
/setProgress
reporting, since
doing that correctly requires some more advanced techniques that we’ll
talk about later. We’ll come back and fix this code later, but for
now:
# ASYNCHRONOUS version
<- eventReactive(input$date, {
data <- input$date
date <- lubridate::year(date)
year
<- glue("http://cran-logs.rstudio.com/{year}/{date}.csv.gz")
url <- file.path("data_cache", paste0(date, ".csv.gz"))
path
future_promise({
if (!file.exists(path)) {
download.file(url, path)
}read_csv(path, col_types = "Dti---c-ci", progress = FALSE)
}) })
Pretty straightforward. This reactive now returns a future (which counts as a promise), not a data frame.
Remember that we must read any reactive values
(including input
) and reactive expressions from
outside the future. (You will get an error if you
attempt to read one from inside the future.)
At this point, since there are no other long-running operations we
want to make asynchronous, we’re actually done interacting directly with
the future
package. The rest of the reactive expressions
will deal with the future returned by data
using general
async functions and operators from promises
.
whales
reactive: simple pipelines are simpleThe whales
reactive takes the data frame from
data
, and uses dplyr to find the top
input$count
most prolific downloaders.
# SYNCHRONOUS version
<- reactive({
whales data() %>%
count(ip_id) %>%
arrange(desc(n)) %>%
head(input$count)
})
Since data()
now returns a promise, the whole function
needs to be modified to deal with promises.
This is basically a best-case scenario for working with
promises
. The whole expression consists of a single
magrittr pipeline. There’s only one object (data()
) that’s
been converted to a promise. The promise object only appears once, at
the head of the pipeline.
When the stars align like this, converting this code to async is
literally as easy as replacing each %>%
with
%...>%
:
# ASYNCHRONOUS version
<- reactive({
whales data() %...>%
count(ip_id) %...>%
arrange(desc(n)) %...>%
head(input$count)
})
The input (data()
) is a promise, the resulting output
object is a promise, each stage of the pipeline returns a promise; but
we can read and write this code almost as easily as the synchronous
version!
An example this simple may seem reductive, but this best-case
scenario happens surprisingly often, if your coding style is influenced
by the tidyverse. In this example app, 59% of the
reactives, observers, and outputs were converted using nothing more than
replacing %>%
with %...>%
.
One last thing before we move on. In the last section, I emphasized
that reactive values cannot be read from inside a future. Here, we’re
using head(input$count)
inside a promise-pipeline; since
data()
is written using a future, doesn’t that mean… well…
isn’t this wrong?
Nope—this code is just fine. The prohibition is against reading reactive values/expressions from inside a future, because code inside a future is executed in a totally different R process. The steps in a promise-pipeline aren’t futures, but promise handlers. These aren’t executed in a different process; rather, they’re executed back in the original R process after a promise is resolved. We’re allowed and expected to access reactive values and expressions from these handlers.
whale_downloads
reactive: reading from multiple
promisesThe whale_downloads
reactive is a bit more complicated
case.
# SYNCHRONOUS version
<- reactive({
whale_downloads data() %>%
inner_join(whales(), "ip_id") %>%
select(-n)
})
Looks simple, but we can’t just do a simple replacement this time. Can you see why?
# BAD VERSION DOESN'T WORK
<- reactive({
whale_downloads data() %...>%
inner_join(whales(), "ip_id") %...>%
select(-n)
})
Remember, both data()
and whales()
now
return a promise object, not a data frame. None of the dplyr verbs know
how to deal with promises natively (and the same is true for almost
every other R function, anywhere in the R universe).
We’re able to use %...>%
with promises on the
left-hand side and regular dplyr calls on the right-hand side, only
because the %...>%
operator “unwraps” the promise object
for us, yielding a regular object (data frame or whatever) to be passed
to dplyr. But in this case, we’re passing whales()
, which a
promise object, directly to inner_join
, and
inner_join
has no idea what to do with it.
The fundamental thing to pattern-match on here, is that we
have a block of code that relies on more than one promise
object, and that means %...>%
won’t be enough.
This is a pretty common situation as well, and occurs in
12% of reactives and outputs in this example app.
Here’s what the real solution looks like:
# ASYNCHRONOUS version
<- reactive({
whale_downloads promise_all(data_df = data(), whales_df = whales()) %...>% with({
%>%
data_df inner_join(whales_df, "ip_id") %>%
select(-n)
}) })
This solution uses the promise
gathering pattern, which combines promises_all
,
%...>%
, and with
.
promise_all
function gathers multiple promise
objects together, and returns a single promise object. This new promise
object doesn’t resolve until all the input promise objects are resolved,
and it yields a list of those results.> promise_all(a = future_promise("Hello"), b = future_promise("World")) %...>% print()
$a
1] "Hello"
[
$b
1] "World" [
%...>%
, as before, “unwraps” the promise object
and passes the result to its right hand side.with
function (from base R) takes a named list, and
makes it into a sort of virtual parent environment while evaluating a
code block you pass it.> x + y
: object 'x' not found
Error
> with(list(x = 1, y = 2), {
+ x + y
+ })
1] 3 [
Let’s once again combine the three, with the simplest possible example of the gathering pattern:
> promise_all(x = future_promise("Hello"), y = future_promise("World")) %...>%
+ with({ paste(x, y) }) %...>%
+ print()
1] "Hello World" [
You can make use of this pattern without remembering exactly how
these pieces combine. Just remember that the arguments to
promise_all
provide the promise objects
(future_promise(1)
and future_promise(2)
),
along with the names you want to use to refer to their yielded values
(x
and y
); and the code block you put in
with()
can refer to those names without worrying about the
fact that they were ever promises to begin with.
total_downloaders
value box: simple pipelines are
for output, tooAll of the value boxes in this app ended up looking a lot like this:
# SYNCHRONOUS version
$total_downloaders <- renderValueBox({
outputdata() %>%
pull(ip_id) %>%
unique() %>%
length() %>%
format(big.mark = ",") %>%
valueBox("unique downloaders")
})
This is structurally no different than the whales
best-case scenario reactive. One thing worth pointing out is that an
async renderValueBox
means you return a promise that
returns a valueBox
; you don’t return a
valueBox
to whom you have passed a promise.
Meaning, you don’t do this:
# BAD VERSION DOESN'T WORK
$total_downloaders <- renderValueBox({
outputvalueBox(
data() %...>%
pull(ip_id) %...>%
unique() %...>%
length() %...>%
format(big.mark = ","),
"unique downloaders"
) })
Instead, you do this:
# ASYNCHRONOUS version
$total_downloaders <- renderValueBox({
outputdata() %...>%
pull(ip_id) %...>%
unique() %...>%
length() %...>%
format(big.mark = ",") %...>%
valueBox("unique downloaders")
})
The other trick worth nothing is the pull
verb, which is
used to retrieve a specific column of a data frame as a vector (similar
to $
or [[
). In this case,
pull(data, ip_id)
is equivalent to
data[["ip_id"]]
. Note that pull
is part of
dplyr and isn’t specific to promises.
biggest_whales
plot: getting untidyIn a cruel twist of API design fate, one of the cornerstone packages
of the tidyverse lacks a tidy API. I’m referring, of course, to
ggplot2
:
# SYNCHRONOUS version
$downloaders <- renderPlot({
outputwhales() %>%
ggplot(aes(ip_name, n)) +
geom_bar(stat = "identity") +
ylab("Downloads on this day")
})
While dplyr
and other tidyverse packages are designed to
link calls together with %>%
, the older
ggplot2
package uses the +
operator. This is
mostly a small aesthetic wart when synchronous code, but it’s a real
problem with async, because the promises
package doesn’t
currently have a promise-aware replacement for +
like it
does for %>%
.
Fortunately, there’s a pretty good escape hatch for
%>%
, and %...>%
inherited it too.
Instead of a pipeline stage being a simple function call, you can put a
{
and }
delimited code block, and inside of
that code block, you can access the “it” value using a period
(.
).
# ASYNCHRONOUS version
$downloaders <- renderPlot({
outputwhales() %...>% {
<- .
whale_df ggplot(whale_df, aes(ip_name, n)) +
geom_bar(stat = "identity") +
ylab("Downloads on this day")
} })
The importance of this pattern cannot be overstated!
Using %...>%
and simple calls alone, you’re restricted
to doing pipeline-compatible operations. But %...>%
together with a curly-brace code block means your handler code can be
any shape or size. Once inside that code block, you have a regular,
non-promise value in .
(if you even want to use
it—sometimes you don’t, as we’ll see later). You can have zero, one, or
more statements. You can use the .
multiple times, in
nested expressions, whatever.
Tip: if you have extensive or complex code to put in a code block,
start the block by creating a properly named variable to store the value
of .
. The reason for this is that .
may
acquire a different meaning than you intend as you add code to the code
block. For example, if a magrittr pipeline starts with .
,
instead of evaluating the pipeline and returning a value, it creates a
function that takes a single argument. So the following code wouldn’t
filter the resolved value of whales()
, but instead, create
an anonymous function that calls filter(n > 1000)
on
whatever you pass it.
whales() %...>% {
%>% filter(n > 1000)
. }
This fixes it:
whales() %...>% {
<- .
whales_df %>% filter(n > 1000)
whales_df }
There are other ways to work around the above problem as well, but I
like this fix because it doesn’t require any thought or care. Just give
the .
value a new name, and forget the .
exists.
For untidy code with a single promise object, just remember: pair a
single %...>%
with a code block and you should be able
to do almost anything.
data
reactive: progress supportNow that we have discussed a few techniques for writing async code,
let’s come back to our original data
event reactive, and
this time do a more faithful async conversion that preserves the
progress reporting functionality of the original.
Again, here’s the original sync code:
# SYNCHRONOUS version
<- eventReactive(input$date, {
data <- input$date # Example: 2018-05-28
date <- lubridate::year(date) # Example: "2018"
year
<- glue("http://cran-logs.rstudio.com/{year}/{date}.csv.gz")
url <- file.path("data_cache", paste0(date, ".csv.gz"))
path
withProgress(value = NULL, {
if (!file.exists(path)) {
setProgress(message = "Downloading data...")
download.file(url, path)
}
setProgress(message = "Parsing data...")
read_csv(path, col_types = "Dti---c-ci", progress = FALSE)
}) })
Progress reporting currently presents two challenges for future.
First, the withProgress({...})
function cannot be used
with async. withProgress
is designed to wrap a slow
synchronous action, and dismisses its progress dialog when the block of
code it wraps is done executing. Since the call to
future_promise()
will return immediately even though the
actual task is far from done, using withProgress
won’t
work; the progress dialog would be dismissed before the download even
got going.
It’s conceivable that withProgress
could gain promise
compatibility someday, but it’s not in Shiny v1.1.0. In the meantime, we
can work around this by using the alternative, object-oriented
progress API that Shiny offers. It’s a bit more verbose and fiddly
than withProgress
/setProgress
, but it is
flexible enough to work with futures/promises.
Second, progress messages can’t be sent from futures. This is simply because futures are executed in child processes, which don’t have direct access to the browser like the main Shiny process does.
It’s conceivable that future
could gain the ability for
child processes to communicate back to their parents, but no good
solution exists at the time of this writing. In the meantime, we can
work around this by taking the one future that does both downloading and
parsing, and splitting it into two separate futures. After the download
future has completed, we can send a progress message that parsing is
beginning, and then start the parsing future.
The regrettably complicated solution is below.
# ASYNCHRONOUS version
<- eventReactive(input$date, {
data <- input$date
date <- lubridate::year(date)
year
<- glue("http://cran-logs.rstudio.com/{year}/{date}.csv.gz")
url <- file.path("data_cache", paste0(date, ".csv.gz"))
path
<- Progress$new()
p $set(value = NULL, message = "Downloading data...")
pfuture_promise({
if (!file.exists(path)) {
download.file(url, path)
}%...>%
}) $set(message = "Parsing data...") } %...>%
{ pfuture_promise(read_csv(path, col_types = "Dti---c-ci", progress = FALSE)) } %>%
{ finally(~p$close())
})
The single future we wrote earlier has now become a pipeline of promises:
Note that neither the R6 call p$set(message = ...)
nor
the second future_promise()
call are tidy, so they use
curly-brace blocks, as discussed in the above section about
biggest_whales
.
The final step of dismissing the progress dialog doesn’t use
%...>%
at all; because we want the progress dialog to
dismiss whether the download and parse operations succeed or fail, we
use the regular pipe %>%
and finally()
function instead. See the relevant section in Working
with promises in R to learn more.
With these changes in place, we’ve now covered all of the changes to the application. You can see the full changes side-by-side via this GitHub diff.
It was a fair amount of work to do the sync-to-async conversion. Now we’d like to know if the conversion to async had the desired effect: improved responsiveness (i.e. lower latency) when the number of simultaneous visitors increases.
At the time of this writing, we are working on a suite of load testing tools for Shiny that is not publicly available yet, but was previewed by Sean Lopp during his epic rstudio::conf 2018 talk about running a Shiny load test with 10,000 simulated concurrent users.
You use these tools to easily record yourself using your Shiny app, which creates a test script; then play back that test script, but multiplied by dozens/hundreds/thousands of simulated concurrent users; and finally, analyze the timing data generated during the playback step to see what kind of latency the simulated users experienced.
To examine the effects of my async refactor, I recorded a simple test script by loading up the app, waiting for the first tab to appear, then clicking through each of the other tabs, pausing for several seconds each time before moving on to the next. When using the app without any other visitors, the homepage fully loads in less than a second, and the initial loading of data and rendering of the plot on the default tab takes about 7 seconds. After that, each tab takes no more than a couple of seconds to load. Overall, the entire test script, including time where the user is thinking, takes about 40 seconds under ideal settings (i.e. only a single concurrent user).
I then used this test script to generate load against the Shiny app running in my local RStudio. With the settings I chose, the playback tool introduced one new “user” session every 5 seconds, until 50 sessions total had been launched; then it waited until all the sessions were complete. I ran this test on both the sync and async versions in turn, which generated the following results.