[R] Construction of a large sparse matrix

[roger koenker]

The dense blocks are too big as Reid has already written --
for smaller instances of this sort of thing  I would suggest that the 
the kronecker
product %x% operator in SparseM,  would be more convenient.


url:	www.econ.uiuc.edu/~roger        	Roger Koenker
email	rkoenker at uiuc.edu			Department of Economics
vox: 	217-333-4558				University of Illinois
fax:   	217-244-6678				Champaign, IL 61820

On Apr 18, 2005, at 3:54 PM, Doran, Harold wrote:

> Dear List:
>
> I'm working to construct a very large sparse matrix and have found
> relief using the SparseM package. I have encountered an issue that is
> confusing to me and wonder if anyone may be able to suggest a smarter
> solution. The matrix I'm creating is a covariance matrix for a larger
> research problem that is subsequently used in a simulation. Below is 
> the
> latex form of the matrix if anyone wants to see the pattern I am trying
> to create.
>
> The core of my problem seems to localize to the last line of the
> following portion of code.
>
> n<-sample.size*4
> k<-n/4
> vl.mat <- as.matrix.csr(0, n, n)
> block <- 1:k #each submatrix size
> for(i in 1:3) vl.mat[i *k + block, i*k + block] <- LE
>
> When the variable LE is 0, the matrix is easily created. For example,
> when sample.size = 10,000 this matrix was created on my machine in 
> about
> 1 second. Here is the object size.
>
>> object.size(vl.mat)
> [1] 160692
>
> However, when LE is any number other than 0, the code generates an
> error. For example, when I try LE <- 2 I get
>
> Error: cannot allocate vector of size 781250 Kb
> In addition: Warning message:
> Reached total allocation of 1024Mb: see help(memory.size)
> Error in as.matrix.coo(as.matrix.csr(value, nrow = length(rw), ncol =
> length(cl))) :
>         Unable to find the argument "x" in selecting a method for
> function "as.matrix.coo"
>
> I'm guessing that single digit integers should occupy the same amount 
> of
> memory. So, I'm thinking that the matrix is "less sparse" and the
> problem is related to the introduction of a non-zero element (seems
> obvious). However, the matrix still retains a very large proportion of
> zeros. In fact, there are still more zeros than non-zero elements.
>
> Can anyone suggest a reason why I am not able to create this matrix? 
> I'm
> at the limit of my experience and could use a pointer if anyone is able
> to provide one.
>
> Many thanks,
> Harold
>
>
> P.S. The matrix above is added to another matrix to create the
> covariance matrix below. The code above is designed to create the
> portion of the matrix \sigma^2_{vle}\bm{J} .
>
>
> \begin{equation}
> \label{vert:cov}
> \bm{\Phi} = var
> \left [
> \begin{array}{c}
> Y^*_{1}\\
> Y^*_{2}\\
> Y^*_{3}\\
> Y^*_{4}\\
> \end{array}
> \right ]
> =
> \left [
> \begin{array}{cccc}
> \sigma^2_{\epsilon}\bm{I}& \sigma^2_{\epsilon}\rho\bm{I} & \bm{0} &
> \bm{0}\\
> \sigma^2_{\epsilon}\rho\bm{I} &
> \sigma^2_{\epsilon}\bm{I}+\sigma^2_{vle}\bm{J} &
> \sigma^2_{\epsilon}\rho^2\bm{I} & \bm{0}\\
> \bm{0} & \sigma^2_{\epsilon}\rho^2\bm{I} &
> \sigma^2_{\epsilon}\bm{I}+\sigma^2_{vle}\bm{J}&
> \sigma^2_{\epsilon}\rho^3\bm{I}\\
> \bm{0} & \bm{0} & \sigma^2_{\epsilon}\rho^3\bm{I}&
> \sigma^2_{\epsilon}\bm{I}+\sigma^2_{vle}\bm{J} \\
> \end{array}
> \right]
> \end{equation}
>
> where $\bm{I}$ is the identity matrix, $\bm{J}$ is the unity matrix, 
> and
> $\rho$ is the autocorrelation.
>
>
>
> 	[[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide! 
> http://www.R-project.org/posting-guide.html