[R] help with sockets in R

[Christopher Bare]

Hi,

Thanks for the advice! My locale and encoding setting follow:

> Sys.getlocale()
[1] "en_US.UTF-8/en_US.UTF-8/C/C/en_US.UTF-8/en_US.UTF-8"
> getOption("encoding")
[1] "native.enc"

I was indeed able to solve my immediate problem by using readLines to
read the whole response and parse it later, following the example in
an old version of OmegaHat's SSOAP package. The httpRequest package
also reads the entire response as a unit, but does so using readBin
and rawToChar in one case and read.socket in another. OmegaHat's RCurl
library deserves further investigation and might be the smartest way
to go.

Thanks also for the pointer to tests/internet.R, which uses read.socket.

The R Data Import/Export guide states that socketConnection is
preferred over the make.socket/read.socket methods. But with several
different options, it's hard to figure out which way to go.

My understanding of HTTP is very limited, but might reading the
response 'til the server closes the socket run afoul of some of the
more advanced uses of HTTP?


Thanks,

-chris



On Tue, Sep 21, 2010 at 11:43 PM, Prof Brian Ripley
<ripley at stats.ox.ac.uk> wrote:
> readLines() is for a text-mode connection; readChar() is for a binary-mode
> connection.  Given that you asked for possible re-encoding by the 'encoding'
> argument, you cannot safely mix them (text-mode access is re-encoded,
> binary-mode is not).  However, we don't know if re-encoding was active in
> your case since we don't know your locale.
>
> Either don't specify an encoding and re-encode the response in R or use
> readLines() to read the complete response and split it up later.
>
> For a different approach with read/write.socket() see tests/internet.R in
> the R sources.
>
> Please do note that the posting guide asked you for 'at a minimum'
> information (which includes the locale) and a reproducible example.
>
> On Tue, 21 Sep 2010, Christopher Bare wrote:
>
>> Hi R gurus,
>>
>> I'm trying to use a ReSTful web service from within R. Specifically, I
>> need to make HTTP PUT requests.
>>
>> I'm able to make the request and that goes well enough, but I'm having
>> trouble properly consuming the HTTP response. The problem comes in
>> when I'm trying to parse out the response body. I get the length of
>> the response body from the Content-Length header. I then try to use
>> readChar(con, nchars=content.length).
>>
>> The result I'm seeing is that the first few characters of the response
>> body are cut off.
>>
>> My code looks like this:
>>
>>
>> http.request <- function(host, path, request, port=80) {
>>
>>        con <- socketConnection(host=host, port=port, open="w+",
>> blocking=TRUE, encoding="UTF-8")
>>        writeLines(request, con)
>>
>>        write("wrote request", stderr())
>>        flush(stderr())
>>
>>        # build response object
>>        response <- list()
>>        response$status <- readLines(con, n=1)
>>        response$headers <- character(0)
>>        repeat{
>>                ss <- readLines(con, n=1)
>>                write(ss, stderr())
>>                flush(stderr())
>>                if (ss == "") break
>>                key.value <- strsplit(ss, ":\\s*")
>>                response$headers[key.value[[1]][1]] <- key.value[[1]][2]
>>        }
>>
>>        if (any(names(response$headers)=='Content-Length')) {
>>                content.length <-
>> as.integer(response$headers['Content-Length'])
>>                response$body <- readChar(con, nchars=content.length)
>>        }
>>        close(con)
>> }
>>
>>
>> response$body ends up with
>>
>> "e,\"id\":\"some_doc\",\"rev\":\"7-906e06a7744780ef93126adc6f8f10ef\"}\n"
>>
>> when it should be:
>>
>>
>> "{\"ok\":true,\"id\":\"some_doc\",\"rev\":\"7-906e06a7744780ef93126adc6f8f10ef\"}"
>>
>>
>> Is mixing readLines and readChars on the same connection causing bad
>> juju? If it is, what's the recommended what to do such a thing?
>>
>>
>> Thanks for any help!!
>>
>> -Chris
>>
>> ______________________________________________
>> R-help at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>
> --
> Brian D. Ripley,                  ripley at stats.ox.ac.uk
> Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
> University of Oxford,             Tel:  +44 1865 272861 (self)
> 1 South Parks Road,                     +44 1865 272866 (PA)
> Oxford OX1 3TG, UK                Fax:  +44 1865 272595
>