[Rd] Read a text file into R with .Call()

[Simon Urbanek]

On Jun 27, 2013, at 9:18 AM, Ge Tan wrote:

> Hi,
> 
> I want to read a text file into R with .Call().
> So I define some NEW_CHARACTER() to store the chracters read and use SET_STRING_ELT to fill the elements.
> 
> e.g.
> PROTECT(qNames = NEW_CHARACTER(10000));
> char *foo; // This foo holds the string I want.
> while(foo = readLine(FN)){
>  SET_STRING_ELT(qNames, i, mkChar(foo)));
> }
> 
> In this way, I can get the desired character from qNames. The only problem is that "mkChar" will make every foo string into a global CHARSXP cache. When I have a huge amount of file to read, the CHARSXP cache use too much memory. I do not know whether there is any other way to SET_STRING_ELT without the mkChar operation.

No. *all* strings in R are in the cache. The whole point of it is that is uses less memory by not duplicating strings - and the overhead for as little as 10000 strings is minuscule. So I suspect that is not your problem since if that was the case, you would not have enough memory to just load the file. Check you code, chances are the issue is elsewhere.

That said, you can always load the file into a raw vector and use accessor function to create strings on demand - but this is only meaningful when you plan to use a very small subset.

Cheers,
Simon


> I know I cam refer to the Biostrings pakcage's way of readDNAStringSet, but that is a bit complicated I have not full understood it.
> 
> Any help will be appreciated!!
> 
> Ge
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
> 
>