[Rd] Implementing a single-precision class with raw (now integer)

[Colin A. Smith]

On Aug 22, 2005, at 3:38 , Prof Brian Ripley wrote:

>> Unfortunately, R sometimes hits the 2GB memory limit of Win32.
> The rw-FAQ explains why that is _not_ the limit!

Sorry to be uninformed, I see from the Microsoft documentation that the 
limit can be increased to 3GB when you change the boot parameters of 
Windows and a flag in the R binary.

> Why via "raw"?
>
> I believe the intention is that this sort of thing be done via 
> external references, but as float and int are the same size on all 
> current platforms, I would have considered R integers for storage. 
> Then for example subsetting would work and you had a 4x larger size 
> limit on 64-bit platforms. (You would also have got automatic handling 
> of endianness.)

I was concerned that on some platforms (particularly 64 bit ones) an 
integer may actually be 64 bits. However, if all current platforms have 
standardized on float and int being the same size, then using the 
integer data type would be more elegant and solve any endian issues. 
(Is that really the case?) Another motivation for using raw was so 
other R functions (particularly internal ones) wouldn't assume the data 
was integer and produce erroneous results. I'd rather have user-visible 
error thrown.

To have the best of both worlds, I'm now using an S4 class with a 
single slot (f) that stores the float data as an integer vector. It 
uses the extensible conversion mechanism created by Duncan Lang to 
allow the .C() function to handle float objects. (The custom converter 
also allows proper checking for single-precision NA values, which don't 
get caught if passed as integers.)

At some point, I would like to release this for others to use. Before I 
get too far into it, I would greatly appreciate it if anyone had any 
ideas, unforeseen caveats, or other suggestions about my current 
direction. My implementation is available here:

http://abagyan.scripps.edu/~csmith/float.R
http://abagyan.scripps.edu/~csmith/float.c
http://abagyan.scripps.edu/~csmith/float_0.1.0.tar.gz

>> I've run into a limitation with the .C() function where it does not
>> handle raw vectors, which it will do in 2.2.0.
> That is just not true!

I should have been a bit more explicit and said that R 2.1.x doesn't 
have RAWSXP handlers in either RObjToCPtr() or CPtrToRObj(), so all the 
C code gets is a SEXP. R-devel does and passes raw vectors as unsigned 
char *. Of course that doesn't matter now that I'm using an extensible 
converter.

Cheers.

-Colin