[R-pkg-devel] socketConnection, delay when reading from

[Ben Engbers]

I'll first finish the current development but then I will see if your 
suggestion is usefull.

Ben

Op 27-11-2021 om 20:19 schreef Tomas Kalibera:
> On 11/27/21 8:05 PM, Tomas Kalibera wrote:
>>
>> On 11/27/21 5:24 PM, Ben Engbers wrote:
>>> Op 27-11-2021 om 17:03 schreef Jeff Newmiller:
>>>> This is a null-terminated message protocol [1]. It has to be 
>>>> processed one byte at a time.
>>>>
>>>> [1] https://docs.basex.org/wiki/Server_Protocol
>>>>
>>> The message may contain embedded 0x00's. To distinguish these 
>>> embedded 0x00's (and 0xFF's) from a terminating 0x00, embedded 0x00's 
>>> and 0xFFare prefixed with a 0xFF byte. This means that when you 
>>> process one byte at a time you have to perform a check on every byte. 
>>> This results in totally unacceptable response times (My first version 
>>> of this client was based on this approach)
>>>
>>> The only alternative solution I can think off is to use C++ to create 
>>> a socket and a function that reads from the socket. But since I have 
>>> hardly any experience with C++ programming nor using the rcpp 
>>> package....
>>
>>
>> I think you could use non-blocking read. Read what is available in 
>> chunks (e.g. up to 1024 bytes long). And based on what is read 
>> already, figure out whether it is all data or not. Something along the 
>> lines as the demo below. The demo, though, is polling too much 
>> (re-running readBin to only find out no data is available yet). It 
>> should be possible to improve with socketSelect().
> 
> This is an extended demo with socketSelect() used to wait on the client 
> for some data to be available, to avoid consuming too much CPU by 
> polling. To be pasted into two R sessions running on the same computer.
> You would have to replace the function done() with something figuring 
> out from the data whether it is complete or not, based on the protocol.
> 
> Best
> Tomas
> 
> 
> # the client
> 
> con2 <- socketConnection("localhost", port = 6011, open = "rb")
> cat("Connected...\n")
> total <- 0
> 
> done <- function(n) {
>    n >= 2e8
> }
> 
> while(!done(total)) {
>     cat("Waiting for data to become ready...\n")
>     socketSelect(list(con2))
>     cat("Reading data...\n")
>     r <- readBin(con2, "raw", 1024)
>     total <- total + length(r)
>     cat("Read", length(r), "bytes (total ", total, ").\n")
> }
> close(con2)
> 
> # the server
> 
> n <- 1e8
> w <- as.raw(runif(n, 0, 255))
> con1 <- socketConnection(port = 6011, blocking = TRUE, server = TRUE, 
> open="a+b")
> cat("Connected...\n")
> writeBin(w, con1)
> cat("Sent data the first time, sleeping...\n")
> Sys.sleep(10)
> cat("Sending data the second time...\n")
> writeBin(w, con1)
> cat("Data sent to client...\n")
> close(con1)
> 
>>
>> Best
>> Tomas
>>
>> # the client
>>
>> con2 <- socketConnection("localhost", port = 6011, open = "rb")
>> cat("Connected...\n")
>> total <- 0
>>
>> done <- function(n) {
>>   n >= 1e8
>> }
>>
>> while(!done(total)) {
>>    r <- readBin(con2, "raw", 1024)
>>    total <- total + length(r)
>>    cat("Read", length(r), "bytes (total ", total, ").\n")
>> }
>> close(con2)
>>
>> # the server
>>
>> n <- 1e8
>> w <- as.raw(runif(n, 0, 255))
>> con1 <- socketConnection(port = 6011, blocking = TRUE, server = TRUE, 
>> open="a+b")
>> cat("Connected...\n")
>> writeBin(w, con1)
>> cat("Data sent to client...\n")
>> close(con1)
>>
>>>
>>> Ben
>>>
>>> ______________________________________________
>>> R-package-devel using r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-package-devel