I'm currently studying PCA (in particular: Härdle, W. and Simar, L. (2003), "Applied Multivariate Statistical Analysis").

 

According to the authors, the "main direction of the data" is the eigenvector corresponding to the largest eigenvalue. 

The PC are then computed by (t)Gamma %*% (X-mu) 

where (t)Gamma is the transpose of the matrix consisting the eigenvectors and (X-mu) is the demeaned data matrix.

 

For the two dimensional case I tried out the following example:

 

par(mfrow = c(2,2))

set.seed(222)

for (i in c(-2, -0.1, 0.8, 2)) {    #choose values for the slope parameters of y

x <- seq(1,100,1)

y <- 20 + i*x + rnorm(100,0,10)

#x<- x -mean(x)     # Härdle suggests to demean the data but this results in a graphical shift of the PC

#y <- y - mean(y)

S <- cov(cbind(x,y))

 

 

PC1 <- eigen(S)$vectors[1,1]*x + eigen(S)$vectors[2,1]*y      #eigenvector corresponding to the first (largest) eigenvalue

PC2 <- eigen(S)$vectors[1,2]*x + eigen(S)$vectors[2,2]*y

plot(x,y, xlim = c(-200, 200), ylim = c(-200, 200))

lines(PC1, type = "l", col = "red", lwd = 2)

lines(PC2, type = "l", col = "blue", lwd = 2)

 

my questions:

 

1.	Why do the PC fit the data better when the slope is steep (e.g. 2 or -2) but fit poorly when the slope is flat (in particular when it's below one).
2.	The PC are not orthogonal to each other although they should be

 

Thanks

 

Zeno



EBS European Business School gemeinnuetzige GmbH
Amtsgericht Wiesbaden HRB 19951
Umsatzsteuer-ID DE 113891213
Geschaeftsfuehrung: Prof. Dr. Christopher Jahns, Rektor; Dr. Reimar Palte, Kanzler;  Sabine Fuchs, Prokuristin
Verwaltungsrat: Dr. Hellmut K. Albrecht, Vorsitzender

	[[alternative HTML version deleted]]

