[R] svm in e1071 package: polynomial vs linear kernel

[David Meyer]

On Mon, 3 Nov 2003 ryszard.czerminski at pharma.novartis.com wrote:

> I am trying to understand what is the difference between linear and 
> polynomial kernel:
> 
>           linear: u'*v
> 
>           polynomial: (gamma*u'*v + coef0)^degree
> 
> It would seem that polynomial kernel with gamma = 1; coef0 = 0 and degree 
> = 1
> should be identical to linear kernel, however it gives me significantly 
> different results  for very simple
> data set, with linear kernel significantly outperforming polynomial 
> kernel.
> 
> *** mse, r2 = 0.5, 0.9 for linear
> *** mse, r2 = 1.8, 0.1 for polynomial
> 
> What am I missing ?

Well: perhaps, that you should pass *all* parameters from your cv.svm
function to the call of svm()?

g.,
-d

> 
> Ryszard
> 
> P.S.
> 
> Here are my results:
> 
> # simple cross validation function
> cv.svm <- function(formula, data, ntry = 3, kernel = "linear", scale = 
> FALSE, cross = 3,
>                    gamma = 1/(dim(data)-1), degree = 3) {
>    mse <- 0; r2 <- 0
>    for (n in 1:ntry) {
>       svm.model <- svm(formula , data = data, scale = scale, kernel = 
> kernel,
>                        cross = cross)
>       mse <- mse + svm.model$tot.MSE
>       r2  <- r2 + svm.model$scorrcoeff
>    }
>    mse <- mse/ntry; r2 <- r2/ntry; result <- c(mse, r2)
>    cat(sprintf("cv.svm> mse, r2 = %5.3f %5.3f\n", mse, r2))
>    return (result)
> }
> 
> # define data set
> 
> x1 <- rnorm(9); x2 <- rnorm(9)
> df <- data.frame(y = 2*x1 + x2, x1, x2)
> 
> #  invoke cv.svm() for linear and polynomial kernels few times
> 
> > r <- cv.svm( y ~ ., df, kernel = "polynomial", gamma = 1, degree = 1, 
> ntry = 32)
> cv.svm> mse, r2 = 1.888 0.162
> > r <- cv.svm( y ~ ., df, kernel = "polynomial", gamma = 1, degree = 1, 
> ntry = 32)
> cv.svm> mse, r2 = 1.867 0.146
> > r <- cv.svm( y ~ ., df, kernel = "polynomial", gamma = 1, degree = 1, 
> ntry = 32)
> cv.svm> mse, r2 = 1.818 0.105
> > r <- cv.svm( y ~ ., df, kernel = "linear", gamma = 1, degree = 1, ntry = 
> 32)
> cv.svm> mse, r2 = 0.525 0.912
> > r <- cv.svm( y ~ ., df, kernel = "linear", gamma = 1, degree = 1, ntry = 
> 32)
> cv.svm> mse, r2 = 0.537 0.878
> > r <- cv.svm( y ~ ., df, kernel = "linear", gamma = 1, degree = 1, ntry = 
> 32)
> cv.svm> mse, r2 = 0.528 0.913
> 
> 
> 	[[alternative HTML version deleted]]
> 
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://www.stat.math.ethz.ch/mailman/listinfo/r-help
>