[R-pkg-devel] socketConnection, delay when reading from

[Jeff Newmiller]

This is a null-terminated message protocol [1]. It has to be processed one byte at a time.

[1] https://docs.basex.org/wiki/Server_Protocol

On November 27, 2021 7:45:31 AM PST, Gabor Grothendieck <ggrothendieck using gmail.com> wrote:
>Whether the length is variable or not isn't relevant. The point is
>whether the message is prefaced by a length or command from which the
>length can be derived.  Maybe it is not and you will have to rely on
>inefficient methods but in many cases protocols are designed to avoid
>such problems.
>
>On Sat, Nov 27, 2021 at 9:40 AM Ben Engbers <Ben.Engbers using be-logical.nl> wrote:
>>
>> No, according to the specification the minimal number of bytes that is
>> returned is 2. There is no maximum. (When querying a database you never
>> know on forehand how many records match the query so by definition you
>> can't calculate the size of the message).
>>
>> In some C, C++ or Java-code I found on internet, it was possible to
>> change the timeout settings so that there would be no delay. Of course
>> this would have as consequence that in your code you have to deal with
>> the possibility that the message has not been completely returned.
>>
>> In R you can set the timeout to 0 but that results in errors (at least
>> on Windows)
>>
>> Op 27-11-2021 om 14:57 schreef Gabor Grothendieck:
>> > Does the message start with a length or a command whose argument length is known
>> > depending on the particular command?
>> > If so first read the length or command and from that the length of the
>> > remainder of
>> > the message can be determined.
>> >
>> > On Sat, Nov 27, 2021 at 4:09 AM Ben Engbers <Ben.Engbers using be-logical.nl> wrote:
>> >>
>> >>
>> >> Hi,
>> >>
>> >> I have been working on a R-client for the BaseX XML-database and version
>> >> 0.9.2 is nearly finished (submitting version 0.9.0 was rejected by CRAN).
>> >> Version 0.3 of RBaseX can be found here
>> >> (https://cran.microsoft.com/web/packages/RBaseX/index.html).
>> >>
>> >> The client-server protocol specifies that the communication between the
>> >> client and the database is based on a socket. The code (below) shows how
>> >> I create that socket.
>> >>
>> >> Writing to the socket works perfect. Reading from the sockets (see
>> >> second codeblock) also produces correct results. The problem however is
>> >> that the timeout, as specified when initializing the socket, causes a 1
>> >> second delay for every read-operation.
>> >> I have experimented a lot with different settings and have been
>> >> searching a lot on internet, but I can't find any method to get rid of
>> >> that delay. (In C or C++ that should be easier but I have never before
>> >> had any need to use those languages).
>> >> The very first version of my client used a block-size of 1 when reading.
>> >> That gave acceptable response times for small query-results but reading
>> >> large responses from the database took very long time.
>> >>
>> >> Do you have any suggestions on how to cope with this problem?
>> >>
>> >> Ben Engbers
>> >>
>> >> -----------------------------
>> >>       CreateSocket = function(host, port = 1984L, username, password) {
>> >>         tryCatch(
>> >>           {conn <- private$conn <- socketConnection(
>> >>             host = "localhost", port,
>> >>             open = "w+b", server = FALSE, blocking = TRUE, encoding =
>> >> "UTF-8", timeout = 1)
>> >>           }, error = function(e) {
>> >>             stop("Cannot open the connection")
>> >>           }
>> >>         )
>> >>
>> >> -----------------------------
>> >>
>> >> readBin_ <- function(conn) {
>> >>     chars_read <- raw(0)
>> >>     rd <- readBin(conn, what = "raw", 1024)
>> >>     while(length(rd) == 1024) {
>> >>       chars_read <- c(chars_read, rd)
>> >>       rd <- readBin(conn, "raw", 1024)
>> >>       }
>> >>     if (length(rd) > 0) chars_read <- c(chars_read, rd)
>> >>     return(chars_read)
>> >> }
>> >>
>> >> ______________________________________________
>> >> R-package-devel using r-project.org mailing list
>> >> https://stat.ethz.ch/mailman/listinfo/r-package-devel
>> >
>> >
>> >
>>
>
>

-- 
Sent from my phone. Please excuse my brevity.