[R] [Fwd: adding more columns in big.matrix object of bigmemory package]

[Jay Emerson]

For good reasons (having to do with avoiding copies of massive things)
we leave such merging to the user: create a new filebacking of the
proper size, and fill it (likely a column at a time, assuming you have
enough RAM to support that).

Jay

On Fri, Dec 17, 2010 at 2:16 AM, utkarshsinghal
<utkarsh.singhal at global-analytics.com> wrote:
>
> Hi,
>
> With reference to the mail below, I have large datasets, coming from various different sources, which I can read into filebacked big.matrix using library bigmemory. I want to merge them all into one 'big.matrix' object. (Later, I want to run regression using library 'biglm').
>
> I am unsuccessfully trying to do this from quite some time now. Can you please suggest some way? Am I missing some already available function?
>
> Even a functionality of the following will work for me:
>
> Just appending more columns in an existing big.matrix object (not merging).
> If the individual datasets are small enough to be read in usual R, just the combined dataset is huge.
>
> Any thoughts are welcome.
>
> Thanks,
> Utkarsh
>
>
> -------- Original Message --------
> Subject: adding more columns in big.matrix object of bigmemory package
> Date: Thu, 16 Dec 2010 18:29:38 +0530
> From: utkarshsinghal <utkarsh.singhal at global-analytics.com>
> To: r help <r-help at stat.math.ethz.ch>
>
> Hi all,
>
> Is there any way I can add more columns to an existing filebacked big.matrix object.
>
> In general, I want a way to modify an existing big.matrix object, i.e., add rows/columns, rename colnames, etc.
> I tried the following:
>
> > library(bigmemory)
> > x = read.big.matrix("test.csv",header=T,type="double",shared=T,backingfile="test.backup",descriptorfile="test.desc")
> > x[,"v4"] = "new"
> Error in mmap(j, colnames(x)) :
>   Couldn't find a match to one of the arguments.
> (The above functionality is presently there in usual data.frames in R.)
>
>
> Thanks in advance,
> Utkarsh
>



--
John W. Emerson (Jay)
Associate Professor of Statistics
Department of Statistics
Yale University
http://www.stat.yale.edu/~jay