[R] using SVD to get an inverse matrix of covariance matrix

[Spencer Graves]

	  More presicely, if M is singular, then M%*%x = b will have multiple 
solutions only if "b" is in the subspace spanned by columns of M.

	  Example:

 > M <- array(1:2, dim=c(2,2))
 > (svdM <- svd(M))
$d
[1] 3.162278 0.000000

$u
            [,1]       [,2]
[1,] -0.4472136 -0.8944272
[2,] -0.8944272  0.4472136

$v
            [,1]       [,2]
[1,] -0.7071068 -0.7071068
[2,] -0.7071068  0.7071068

	  This "M" is not symmetrical and so cannot be a covariance matrix, but 
you can get the same effect with a symmetrical matrix.  I'm using this 
example because it is the simplest thing that comes to mind to 
illustrate the point.

	  By the definition of the singular value decomposition, M = svdM$u %*% 
diag(svdM$d) %*% t(svdM$v).  Since svdM$d[2] == 0,

	  M%*%x = svdM$u[, 1]*svdM$d[1]*t(svdM$v[,1])%*%x
	    = x1 * svdM$u[,1],
where
	  x1 = svdM$d[1]*t(svdM$v[,1])%*%x

	  Thus, if b is proportional to svdM$u[,1], the system has a solution. 
  That solution will be proportional to svdM$v[,1] + x2*svdM$v[,2], for 
any arbitrary value of x2.

hope this helps.
spencer graves

Thomas Lumley wrote:
> On Fri, 11 Jul 2003, ge yreyt wrote:
> 
> 
>>Dear R-users,
>>
>>I have one question about using SVD to get an inverse
>>matrix of covariance matrix
>>
>>Sometimes I met many singular values d are close to 0:
>>look this example
> 
> 
> <snip>
> 
>>most values of inverse matrix
>>will be huge. This must be not a good way. MOre
>>special case, if a single value is 0, then
>>we can not calculate inverse d based on 1/d.
>>
>>Therefore, my question is how I can calculate inverse
>>d (that is inverse diag(d) more efficiently???
>>
> 
> 
> If singular values are zero the matrix doesn't have an inverse: that is,
> the equation   Mx=b  will have multiple solutions for any given b.
> 
> It is possible to get a pseudoinverse, a matrix M that picks out one of
> the solutions.  One way to do this is to set the diagonal to 1/d where d
> is not (nearly) zero and to 0 when d is (nearly) zero. One place to find a
> discussion of this is `Matrix Computations' by Golub and van Loan.
> 
> 
> 	-thomas
> 
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://www.stat.math.ethz.ch/mailman/listinfo/r-help