[R] How to parallelize a process called by a socket connection

[Ivan Krylov]

On Sat, 1 Feb 2020 11:24:51 -0800
James Spottiswoode <james using jsasoc.com> wrote:

>   while(TRUE){
>  	con <- socketConnection(host="localhost", port =
> server_port, blocking=TRUE, server=TRUE, open="r+", timeout =
> 100000000)
>       data <- readLines(con, 1L, skipNul = T, ok = T)
>     	response <- check(data)    
>     	if (!is.null(response)) writeLines(response, con)
>   }

> This all works perfectly except that while check() spends ~50ms doing
> its stuff no more requests can be received and processed.

This poses an interesting challenge.

Normally, a single-threaded server would call listen(socket, backlog)
[1] with backlog > 1, so that other clients attempting to connect() can
wait in the queue while the server calls accept() in a loop and handles
them one by one. Unfortunately, R socketConnection()s are single-client
only, since R closes the listen()ing helper socket immediately after
accept()ing the first client [2].

A quick and dirty hack would be to modify utils::make.socket (and use
read.socket()/write.socket() instead of readLines()/writeLines()),
omitting the check for "localhost" and .Call(C_sockclose, tmp) at the
end of if(server) branch. (Instead, the socket called "tmp" should be
kept and the code should loop on .Call(C_socklisten, tmp) to process all
clients.) I cannot in recommend this in good faith as a long-term
solution, since all involved APIs are private and should not be
depended upon.

Some code from the svSocket package [3] could be repurposed to rely on
the Tcl/Tk event loop to handle multiple clients on a singe server
socket, but implementing that would require knowledge of Tcl. If you
can afford to change the clients to use the HTTP protocol, you can use
the httpuv package [4] to handle the connection management and HTTP
request parsing for you.

Besides that, I don't see a way to handle multiple clients with a single
server socket in R. I must be missing something.

-- 
Best regards,
Ivan

[1] https://beej.us/guide/bgnet/html/#listen

[2]
https://github.com/wch/r-source/blob/07c17042d9e198319a425df726cc80545ae69812/src/modules/internet/sockconn.c#L77

[3] https://cran.r-project.org/package=svSocket

[4] https://cran.r-project.org/package=httpuv