[Rd] RData File Specification?

[Cook, Ian]

It looks like there's a Perl XDR library, but I haven't tested it.
http://search.cpan.org/src/GORD/XDR-0.03/

This might be easy to do all within R if more of the source functions in
serialize.c (ReadItem, MakeReadRefTable, etc.) were callable using
.Internal (i.e. if they were in the R_FunTab table in names.c).

Thanks for your help.

Ian Cook | Advanced Micro Devices, Inc. | ian.cook at amd.com


-----Original Message-----
From: Hin-Tak Leung [mailto:hin-tak.leung at cimr.cam.ac.uk] 
Sent: Friday, August 24, 2007 11:07 AM
To: Cook, Ian
Cc: r-devel at r-project.org
Subject: Re: [Rd] RData File Specification?

I was going to write 'Use the source, Luke', but it seems that you have 
alreday found the relevant source files. I wrote a Python baed Rdata 
writer and a reader sometimes ago just using that info and I am not
away of any file spec, so I know those two files are sufficient. For 
what you want to do, I think you'll have to write some fairly 
substantial code to process the Rdata as just XDR stream (as my python 
scripts do, using the python built-in xdrlib),
because as far as I know the API you are after is not exposed - you'll
have to - and you can - cut and paste a substantial part of saveload.c
and serialize.c for that matter, of course.

I think my python-based Rdata reader would do most of what you want
(it was written for mostly diagnostic purposes as I was 'hand-crafting'
R objects in C and saving them as Rdata then read it tell me what's 
wrong with them, if any) except it dumps a sort of general human 
readable ascii text format rather than csv...

My sugegstion would be to use a lanaguage you are comfortable with which

comes with an xdr library, and just do it by hand...

Cook, Ian wrote:
> Hi,
> 
> I am developing a tool for converting a large data frame stored in an
uncompressed binary (XDR) RData file to a delimited text file.  The data
frame is too large to load() and extract rows from on a typical PC.  I'm
looking to parse through the file and extract individual entries without
loading the whole thing into memory.
> 
> In terms of some C source functions, instead of doing
RestoreToEnv(R_Unserialize(connection)) which is essentially what load()
does, I'm looking to get the documentation I would need to build a
function "SaveToCSV()" so that I could do
SaveToCSV(R_Unserialize(connection)).
> 
> Where can I get documentation on the RData file format?  Does a spec
document exist?
> 
> See details below.
> 
> Thanks,
> Ian
> 
> Ian Cook | Advanced Micro Devices, Inc. | ian.cook at amd.com