I am trying to understand what is the difference between linear and 
polynomial kernel:

          linear: u'*v

          polynomial: (gamma*u'*v + coef0)^degree

It would seem that polynomial kernel with gamma = 1; coef0 = 0 and degree 
= 1
should be identical to linear kernel, however it gives me significantly 
different results  for very simple
data set, with linear kernel significantly outperforming polynomial 
kernel.

*** mse, r2 = 0.5, 0.9 for linear
*** mse, r2 = 1.8, 0.1 for polynomial

What am I missing ?

Ryszard

P.S.

Here are my results:

# simple cross validation function
cv.svm <- function(formula, data, ntry = 3, kernel = "linear", scale = 
FALSE, cross = 3,
                   gamma = 1/(dim(data)-1), degree = 3) {
   mse <- 0; r2 <- 0
   for (n in 1:ntry) {
      svm.model <- svm(formula , data = data, scale = scale, kernel = 
kernel,
                       cross = cross)
      mse <- mse + svm.model$tot.MSE
      r2  <- r2 + svm.model$scorrcoeff
   }
   mse <- mse/ntry; r2 <- r2/ntry; result <- c(mse, r2)
   cat(sprintf("cv.svm> mse, r2 = %5.3f %5.3f\n", mse, r2))
   return (result)
}

# define data set

x1 <- rnorm(9); x2 <- rnorm(9)
df <- data.frame(y = 2*x1 + x2, x1, x2)

#  invoke cv.svm() for linear and polynomial kernels few times

> r <- cv.svm( y ~ ., df, kernel = "polynomial", gamma = 1, degree = 1, 
ntry = 32)
cv.svm> mse, r2 = 1.888 0.162
> r <- cv.svm( y ~ ., df, kernel = "polynomial", gamma = 1, degree = 1, 
ntry = 32)
cv.svm> mse, r2 = 1.867 0.146
> r <- cv.svm( y ~ ., df, kernel = "polynomial", gamma = 1, degree = 1, 
ntry = 32)
cv.svm> mse, r2 = 1.818 0.105
> r <- cv.svm( y ~ ., df, kernel = "linear", gamma = 1, degree = 1, ntry = 
32)
cv.svm> mse, r2 = 0.525 0.912
> r <- cv.svm( y ~ ., df, kernel = "linear", gamma = 1, degree = 1, ntry = 
32)
cv.svm> mse, r2 = 0.537 0.878
> r <- cv.svm( y ~ ., df, kernel = "linear", gamma = 1, degree = 1, ntry = 
32)
cv.svm> mse, r2 = 0.528 0.913


	[[alternative HTML version deleted]]


