[Rd] Serializing many small objects efficiently

[Whit Armstrong]

Here's a snip from r-hcp. You can probably find it in the archive:

From: Michael Spiegel
Date: Thu, Sep 29, 2011 at 11:38 AM
Subject: RE: [R-sig-hpc] [zeromq-dev] rzmq package

Calling serialize/serialize from c/c++ is not too convoluted. You can
find a good example in
https://github.com/mspiegel/PiebaldMPI/blob/master/src/lapply_workers_helpers.c,
look for the function "generateReturnList". I'm doing both
serialization and unserialization in that function, but you'll be able
to tease apart the two calls.




On Thu, Mar 22, 2012 at 12:34 PM, Antonio Piccolboni
<antonio at piccolboni.info> wrote:
> Hi,
> sorry if this question is trivial or unclear, this is my first venture into
> mixed C/R programming (I am reasonably experienced in each separately).
> I am trying to write a serialization function for a format called
> typedbytes, which is used as an interchange format in Hadoop circles. Since
> I would need to serialize according to the internal R format many small R
> objects I looked at the c interface
>
> void R_Serialize(SEXP s, R_outpstream_t ops);
> SEXP R_Unserialize(R_inpstream_t ips);
>
> If I look at the source for e.g. unserialize is see a
>
>  .Call("R_unserialize", connection, refhook, PACKAGE = "base")
>
> which, despite the name of the second argument, accepts as 'connection' a
> raw vector. Is there any way to call that function from C -- without
> calling the R function? Failing that, from what I've read I gather that it
> is not possible to get a C stream from a connection, so unless I am wrong
> using R_serialize directly is not possible. If all else fails I would have
> probably to use a hack requiring knowledge of the serialization format,
> which I'd much rather avoid. Suggestions? Thanks
>
>
> Antonio
>
>        [[alternative HTML version deleted]]
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel