[Rd] Implementing a single-precision class with raw (now integer)
Colin A. Smith
colin at colinsmith.org
Thu Aug 25 03:25:39 CEST 2005
On Aug 22, 2005, at 3:38 , Prof Brian Ripley wrote:
>> Unfortunately, R sometimes hits the 2GB memory limit of Win32.
> The rw-FAQ explains why that is _not_ the limit!
Sorry to be uninformed, I see from the Microsoft documentation that the
limit can be increased to 3GB when you change the boot parameters of
Windows and a flag in the R binary.
> Why via "raw"?
> I believe the intention is that this sort of thing be done via
> external references, but as float and int are the same size on all
> current platforms, I would have considered R integers for storage.
> Then for example subsetting would work and you had a 4x larger size
> limit on 64-bit platforms. (You would also have got automatic handling
> of endianness.)
I was concerned that on some platforms (particularly 64 bit ones) an
integer may actually be 64 bits. However, if all current platforms have
standardized on float and int being the same size, then using the
integer data type would be more elegant and solve any endian issues.
(Is that really the case?) Another motivation for using raw was so
other R functions (particularly internal ones) wouldn't assume the data
was integer and produce erroneous results. I'd rather have user-visible
To have the best of both worlds, I'm now using an S4 class with a
single slot (f) that stores the float data as an integer vector. It
uses the extensible conversion mechanism created by Duncan Lang to
allow the .C() function to handle float objects. (The custom converter
also allows proper checking for single-precision NA values, which don't
get caught if passed as integers.)
At some point, I would like to release this for others to use. Before I
get too far into it, I would greatly appreciate it if anyone had any
ideas, unforeseen caveats, or other suggestions about my current
direction. My implementation is available here:
>> I've run into a limitation with the .C() function where it does not
>> handle raw vectors, which it will do in 2.2.0.
> That is just not true!
I should have been a bit more explicit and said that R 2.1.x doesn't
have RAWSXP handlers in either RObjToCPtr() or CPtrToRObj(), so all the
C code gets is a SEXP. R-devel does and passes raw vectors as unsigned
char *. Of course that doesn't matter now that I'm using an extensible
More information about the R-devel