[R-pkg-devel] socketConnection, delay when reading from

[Tomas Kalibera]

On 11/27/21 8:05 PM, Tomas Kalibera wrote:
>
> On 11/27/21 5:24 PM, Ben Engbers wrote:
>> Op 27-11-2021 om 17:03 schreef Jeff Newmiller:
>>> This is a null-terminated message protocol [1]. It has to be 
>>> processed one byte at a time.
>>>
>>> [1] https://docs.basex.org/wiki/Server_Protocol
>>>
>> The message may contain embedded 0x00's. To distinguish these 
>> embedded 0x00's (and 0xFF's) from a terminating 0x00, embedded 0x00's 
>> and 0xFFare prefixed with a 0xFF byte. This means that when you 
>> process one byte at a time you have to perform a check on every byte. 
>> This results in totally unacceptable response times (My first version 
>> of this client was based on this approach)
>>
>> The only alternative solution I can think off is to use C++ to create 
>> a socket and a function that reads from the socket. But since I have 
>> hardly any experience with C++ programming nor using the rcpp 
>> package....
>
>
> I think you could use non-blocking read. Read what is available in 
> chunks (e.g. up to 1024 bytes long). And based on what is read 
> already, figure out whether it is all data or not. Something along the 
> lines as the demo below. The demo, though, is polling too much 
> (re-running readBin to only find out no data is available yet). It 
> should be possible to improve with socketSelect().

This is an extended demo with socketSelect() used to wait on the client 
for some data to be available, to avoid consuming too much CPU by 
polling. To be pasted into two R sessions running on the same computer.
You would have to replace the function done() with something figuring 
out from the data whether it is complete or not, based on the protocol.

Best
Tomas


# the client

con2 <- socketConnection("localhost", port = 6011, open = "rb")
cat("Connected...\n")
total <- 0

done <- function(n) {
   n >= 2e8
}

while(!done(total)) {
    cat("Waiting for data to become ready...\n")
    socketSelect(list(con2))
    cat("Reading data...\n")
    r <- readBin(con2, "raw", 1024)
    total <- total + length(r)
    cat("Read", length(r), "bytes (total ", total, ").\n")
}
close(con2)

# the server

n <- 1e8
w <- as.raw(runif(n, 0, 255))
con1 <- socketConnection(port = 6011, blocking = TRUE, server = TRUE, 
open="a+b")
cat("Connected...\n")
writeBin(w, con1)
cat("Sent data the first time, sleeping...\n")
Sys.sleep(10)
cat("Sending data the second time...\n")
writeBin(w, con1)
cat("Data sent to client...\n")
close(con1)

>
> Best
> Tomas
>
> # the client
>
> con2 <- socketConnection("localhost", port = 6011, open = "rb")
> cat("Connected...\n")
> total <- 0
>
> done <- function(n) {
>   n >= 1e8
> }
>
> while(!done(total)) {
>    r <- readBin(con2, "raw", 1024)
>    total <- total + length(r)
>    cat("Read", length(r), "bytes (total ", total, ").\n")
> }
> close(con2)
>
> # the server
>
> n <- 1e8
> w <- as.raw(runif(n, 0, 255))
> con1 <- socketConnection(port = 6011, blocking = TRUE, server = TRUE, 
> open="a+b")
> cat("Connected...\n")
> writeBin(w, con1)
> cat("Data sent to client...\n")
> close(con1)
>
>>
>> Ben
>>
>> ______________________________________________
>> R-package-devel using r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-package-devel