[R] How Can SVD Reconstruct a Matrix

[Peter Langfelder]

On Wed, Aug 13, 2014 at 11:57 PM, Peter Brady
<subscriptions at simonplace.net> wrote:
> Hi All,
>
> I've inherited some R code that I can't work out what they've done.  It
> appears to work and give sort of reasonable answers, I'm just trying to
> work out why they've done what they have.  I suspect that this is a
> simple vector identity that I've just been staring at too long and have
> forgotten...
>
> The code:
>
> GGt <- M0 - M1 %*% M0inv %*% t(M1)
> svdGG <- svd(GGt)
> Gmat <- svdGG$u %*% diag(sqrt(svdGG$d))
>
> It is supposed to solve:
>
> G*G^T = M0 - M1*M0^-1*M1^T
>
> for G, where G^T is the transpose of G.  It is designed to reproduce a
> numerical method described in two papers:
>
> Srikanthan and Pegram, Journal of Hydrology, 371 (2009) 142-153,
> Equation A13, who suggest the SVD method but don't describe the
> specifics, eg: "...G is found by singular value decomposition..."
>
> Alternatively, Matalas (1967) Water Resources Research 3 (4) 937-945,
> Equation 17, say that the above can be solved using Principle Component
> Analysis (PCA).
>
> I use PCA (specifically POD) and SVD to look at the components after
> decomposition, so I'm a bit lost as to how the original matrix G can be
> constructed in this case from only the singular values and the left
> singular vectors.

GG' is a symmetric matrix, so left- and right-singular vectors are the
same. If I recall right, in general it is impossible to find G from
GG' (I denote the transpose by ') since, given an orthogonal
transformation U (that is, UU'=1), GUU'G' = GG', so you can only find
G up to multiplication with an orthogonal transformation matrix.

Since SVD decomposes a matrix X = UDV', the decomposition for GG' is

GG' = UDU'; setting S = sqrt(D) (i.e., diagonal matrix with elements
that are sqrt of those in D), GG' = USSU' = USS'U', so one solution is
G = US which is the solution used.

You could use PCA on G, which is roughly equivalent to doing SVD on
GG' (up to centering and scaling of the columns of G). I am not very
familiar with PCA in R since I always use SVD, but here's what the
help file for prcomp (PCA in R) says:

   The calculation is done by a singular value decomposition of the
     (centered and possibly scaled) data matrix, not by using ‘eigen’
     on the covariance matrix.  This is generally the preferred method
     for numerical accuracy.

HTH,

Peter


> Like I said earlier, I suspect that this is a simple
> array identity that I've forgotten.  My Google Fu is letting me down at
> this point.
>
> My questions:
> 1) What is the proof, or where can I better find it to satisfy myself,
> that the above works?
>
> 2) Alternatively, can anyone suggest how I could apply PCA in R to
> compute the same?
>
> Thanks in advance,
> -pete
>
> --
> Peter Brady
> Email: pdbrady at ans.com.au
> Skype: pbrady77
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.