Hi
I am doing this sort of thing:
POLY:
> > obj = best.tune(svm, similarity ~., data = training, kernel =
"polynomial")
> summary(obj)
Call:
best.tune(svm, similarity ~ ., data = training, kernel = "polynomial")
Parameters:
SVM-Type: eps-regression
SVM-Kernel: polynomial
cost: 1
degree: 3
gamma: 0.04545455
coef.0: 0
epsilon: 0.1
Number of Support Vectors: 754
> svm.model <- svm(similarity ~., data = training, kernel =
"polynomial", cost = 1, degree = 3, gamma = 0.04545455, coef.0 = 0,
epsilon = 0.1)
> pred=predict(svm.model, testing)
> pred[pred > .5] = 1
> pred[pred <= .5] = 0
> table(testing$similarity, pred)
pred
0 1
0 30 8
1 70 63
> obj = best.tune(svm, similarity ~., data = training, kernel =
"linear")
> summary(obj)
LINEAR:
Call:
best.tune(svm, similarity ~ ., data = training, kernel = "linear")
Parameters:
SVM-Type: eps-regression
SVM-Kernel: linear
cost: 1
gamma: 0.04545455
epsilon: 0.1
Number of Support Vectors: 697
> svm.model <- svm(similarity ~., data = training, kernel = "linear",
cost = 1, gamma = 0.04545455, epsilon = 0.1)
> pred=predict(svm.model, testing)
> pred[pred > .5] = 1
> pred[pred <= .5] = 0
> table(testing$similarity, pred)
pred
0 1
0 6 32
1 4 129
RADIAL:
> obj = best.tune(svm, similarity ~., data = training, kernel =
"radial")
> summary(obj)
Call:
best.tune(svm, similarity ~ ., data = training, kernel = "linear")
Parameters:
SVM-Type: eps-regression
SVM-Kernel: linear
cost: 1
gamma: 0.04545455
epsilon: 0.1
Number of Support Vectors: 697
> svm.model <- svm(similarity ~., data = training, kernel = "radial",
cost = 1, gamma = 0.04545455, epsilon = 0.1)
> pred=predict(svm.model, testing)
> pred[pred > .5] = 1
> pred[pred <= .5] = 0
> table(testing$similarity, pred)
pred
0 1
0 27 11
1 64 69
SIGMOID:
> obj = best.tune(svm, similarity ~., data = training, kernel =
"sigmoid")
> summary(obj)
Call:
best.tune(svm, similarity ~ ., data = training, kernel = "sigmoid")
Parameters:
SVM-Type: eps-regression
SVM-Kernel: sigmoid
cost: 1
gamma: 0.04545455
coef.0: 0
epsilon: 0.1
Number of Support Vectors: 986
> svm.model <- svm(similarity ~., data = training, kernel = "sigmoid",
cost = 1, gamma = 0.04545455, coef.0 = 0, epsilon = 0.1)
> pred=predict(svm.model, testing)
> pred[pred > .5] = 1
> pred[pred <= .5] = 0
> table(testing$similarity, pred)
pred
0 1
0 8 30
1 26 107
>
and then taking out the kappa statistic to see if I am getting anything
significant.
I get kappas of 15 - 17% - I don't think that is very good. I know
kappa is really for comparing the outcomes of two taggers but it seems a
good way to measure if your results might be by chance.
Two questions:
Any comments on Kappa and what it might be telling me?
What can I do to tune my kernels further?
Stephen
[[alternative HTML version deleted]]