[R] FastICA and matching variance of inputs

[Andrew Moroz]

I have been playing around with the FastICA algorithm in R, and I have
come across the following issue. I'm new to this topic, so I'm hoping
someone will be able to shed some insight.

We know that the variance of the Independent Components is constrained
to be 1 and they are uncorrelated, so the sum of squares of the
coefficients of the mixing matrix (a$A) will give the variance of the
reconstructed inputs (a$X). It seems to me that this should match the
variance of the actual inputs (X). But the former seems to be smaller
than the latter, by varying amounts. Does anyone know why that might
be?


# results are close

S <- matrix(runif(10000), 500, 20)
A <- matrix(runif(4000), 20, 200)
X <- S%*%A
a <- fastICA(X, 20)

as.vector(apply(X,2,sd))[1:10]
apply(a$A,2,function(x) sqrt(sum(x^2)))[1:10]

--

> as.vector(apply(X,2,sd))[1:10]
[1] 0.6815156 0.7160899 0.8237067 0.6777511 0.8045670 0.7584224 0.7818801
[8] 0.8015249 0.7252270 0.6337887
> apply(a$A,2,function(x) sqrt(sum(x^2)))[1:10]
[1] 0.6808337 0.7153735 0.8228825 0.6770730 0.8037620 0.7576636 0.7810978
[8] 0.8007229 0.7245014 0.6331546

--


# results are not close

S <- matrix(runif(10000), 500, 20)
A <- matrix(runif(4000), 20, 200)
X <- S%*%A
a <- fastICA(X, 2)

as.vector(apply(X,2,sd))[1:10]
apply(a$A,2,function(x) sqrt(sum(x^2)))[1:10]

--

> as.vector(apply(X,2,sd))[1:10]
[1] 0.7400437 0.7775435 0.6862873 0.7890104 0.7516134 0.8099342 0.8095056
[8] 0.7653593 0.6743729 0.7875858
> apply(a$A,2,function(x) sqrt(sum(x^2)))[1:10]
[1] 0.6463631 0.7131771 0.5987285 0.6979883 0.6664734 0.7300787 0.7179024
[8] 0.6571067 0.5549075 0.6684818


--