[R] SVD/Eigenvector confusion

[Prof Brian Ripley]

On Sun, 29 Feb 2004, Philip Warner wrote:

> My understanding of SVD is that, for A an mxn matrix, m > n:
> 
>      A = UWV*
> 
> where W is square root diagonal eigenvalues of A*A extended with zero 
> valued rows, and U and V are the left & right eigen vectors of A. But this 
> does not seem to be strictly true and seems to require specific 
> eigenvectors, and I am not at all sure how these are computed.

(A %*% t(A) is required, BTW.)  That is not the definition of the SVD.  
It is true that U are eigenvectors of A %*% t(A) and V of t(A) %*% A, but
that does not make them left/right eigenvectors of A (unless that is your
private definition).  Since eigenvectors are not unique, it does mean that
you cannot reverse the process, as you seem to be trying to do.

Eigenvectors are only defined up to a sign (and more if there are 
duplicate eigenvalues) and singular vectors are only defined up to a sign 
(changing both U and V).  You will find both vary by sign depending on the 
exact version of R used (including which BLAS and which compiler 
optimization level).  Singular vectors have unit length, but eigenvectors 
do not have to (although they do in the code you have used).

> Since W should have a zero row at the bottom, which when multiplied by U 
> will just remove the last column of U, I have just omitted the last row of 
> u from the outset:
> 
> eg, in R:
> 
>      a <- matrix(c(c(1,2,3),c(5,14,11)),3,2)
>      u <- eigen(a %*% t(a))$vectors[,1:2]
>      v <- eigen(t(a) %*% a)$vectors
>      w <- sqrt(diag(eigen(t(a) %*% a)$values))
>      u %*% w %*% t(v)
> 
> gives:
>             [,1]       [,2]
> [1,] -0.9390078  -5.011812
> [2,] -3.3713773 -13.734403
> [3,] -1.3236615 -11.324660
> 
> which seems a little off the mark. 

It is not expected to work.

> The value for v is:
> 
>            [,1]       [,2]
> [1,] 0.1901389  0.9817572
> [2,] 0.9817572 -0.1901389
> 
> Where as svd(a)$v is:
> 
>             [,1]       [,2]
> [1,] -0.1901389  0.9817572
> [2,] -0.9817572 -0.1901389

> If I substitute this in the above, I get:
> 
>      u %*% w %*% t(svd(a)$v)
> 
> which returns:
> 
>       [,1] [,2]
> [1,]    1    5
> [2,]    2   14
> [3,]    3   11
> 
> which is what the SVD should do. I assume there is some rule about setting 
> the signs on eigenvectors for SVD, and would appreciate any help.

There is no rule: the SVD is computed by a different algorithm.

-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595