[R] Spectral Decomposition

[(Ted Harding)]

On 29-Jun-07 12:29:31, Doran, Harold wrote:
> All of my resources for numerical analysis show that the spectral
> decomposition is
> 
> A = CBC'
> 
> Where C are the eigenvectors and B is a diagonal matrix of eigen
> values. Now, using the eigen function in R
> 
># Original matrix
> aa <- matrix(c(1,-1,-1,1), ncol=2)
> 
> ss <- eigen(aa)
> 
># This results yields back the original matrix according to the
> formula above
> ss$vectors %*% diag(ss$values) %*% t(ss$vectors)
> 
> However, for the following I do not get back the original matrix
> using the general formula for the spectral decomposition:
> 
> set.seed(123)
> 
> aa <- matrix(rnorm(16), ncol=4)
> 
> ss <- eigen(aa)
> 
> ss$vectors %*% diag(ss$values) %*% t(ss$vectors)
> 
> However, if I do the following
> 
> A = CBC^{-1}
> 
> I get back the original matrix A
> 
> ss$vectors %*% diag(ss$values) %*% solve(ss$vectors)

Harold, I think the key to the issue is whether your original
matric is symmetric or not. For your formula

  A = C*B*C'

to work, where B is a diagonal matrix (therefore essentially
symmetric) you have -- bearing in mind the reversal of factors --

  A' = ReverseFactorsIn(C'*B'*C) = C*B*C' = A

so A would have to be symmetric. This was the case for your
first example matrix(c(1,-1,-1,1), ncol=2).

However, your second example will not be symmetric, so the
formula will not work, and you will need A = C*B*C^(-1).

If A is not symmetric, you have "left" eigenvectors:

  x'*A = lambda*x'

and "right" eigenvectors:

  A*x = lambda*x

and the "left" eigenvectors are not the same as the "right"
eigenvectors, though you have the same set of eigenvalues lambda
in each case.

You then have

  A = L'*B*R

Of course the most frequent occurrence of this kind of question
in statistics is where A is a covariance or correlation matrix,
which is symmetric by definition.

Hoping this helps!
Ted.

> In my lit search I am not finding an explanation for why this works, so
> I am seeking R-help since there may be a computational rationale that
> can be explained by users (or authors) of the function. In my
> experimentation with some computations it seems that the latter
> approach is more general in that it yields back the matrix I began
> with, but deviates from the formula I commonly see in texts.
> 
> Thanks,
> Harold

--------------------------------------------------------------------
E-Mail: (Ted Harding) <ted.harding at nessie.mcc.ac.uk>
Fax-to-email: +44 (0)870 094 0861
Date: 29-Jun-07                                       Time: 14:23:03
------------------------------ XFMail ------------------------------