[R] tune an support vector machine
Uwe Bohne
balu555 at gmx.de
Mon Dec 9 10:57:56 CET 2013
This is the solution fpr the problem.
Its not the cleanest one i guess but it works just the way i wanted.
If anybody has better performing or cleaner code, please send it to me.
best regards
Uwe
type<-sample(c(-1,1) , 20, replace = TRUE )
weight<-sample(c(20:50),20, replace=TRUE)
height<-sample(c(100:200),20, replace=TRUE)
width<-sample(c(30:50),20,replace=TRUE)
volume<-sample(c(1000:5000),20,replace=TRUE)
data<-cbind(type,weight,height,width,volume)
train<-as.data.frame(data)
train$type<-as.factor(train$type)
library("e1071")
features <- c("weight","height","width","volume")
(formula<-as.formula(paste("type ~ ", paste(features, collapse= "+"))))
svmtune=tune.svm(formula, data=train, kernel="radial",
type="C-classification",cost=2^(-2:5), gamma=2^(-2:1),cross=10)
summary(svmtune)
namen<-expand.grid(c("weight",NA),
c("height",NA),c("width",NA),c("volume",NA), stringsAsFactors=FALSE)
l <- apply(head(namen, -1), 1, function(x)
reformulate(paste(na.omit(x), collapse = "+"), response =
"type"))
for (i in (1:(nrow(namen)-1)))
{assign(paste("svmtune",i,sep=""),tune.svm(l[[i]], data=train,
kernel="radial",type="C-classification", cost=2^(-2:1),
gamma=2^(-2:1),cross=10))}
for (i in (1:(nrow(namen)-1)))
{Modell[i]<-eval(parse(text=paste("svmtune",i,"$best.performance",sep="")))}
Modell2<-as.data.frame(Modell)
bestesModell_Index<-which(Modell2==max(Modell2))[1]
bestesModell_Gamma<-eval(parse(text=paste("svmtune",bestesModell_Index,"$bes
t.parameters$gamma",sep="")))
bestesModell_Cost<-eval(parse(text=paste("svmtune",bestesModell_Index,"$best
.parameters$cost",sep="")))
bestesModell_Formel<-l[[bestesModell_Index]]
#bestesModell_Parameter
bestesModell<-c(bestesModell_Index,bestesModell_Gamma,bestesModell_Cost,
bestesModell_Formel)
bestesModell_SVM<-svm(bestesModell_Formel, data=train, kernel="radial",
gamma=bestesModell_Gamma, cost=bestesModell_Cost, type="C-classification")
test<-data[1:10 ,]
pred<-predict(bestesModell_SVM, test)
test[, 1]
pred
table(pred, test[, 1])
Gesendet: Samstag, 07. Dezember 2013 um 09:15 Uhr
Von: "Uwe Bohne" <balu555 at gmx.de>
An: "Wuming Gong" <gongx030 at umn.edu>
Cc: "r-help mailinglist" <r-help at r-project.org>
Betreff: Re: [R] tune an support vector machine
Thank you very much,
your proposal is one practical way to check for significant features.
I tried to check for all combination in a loop, but unfortunately there is a
problem with NA values.
Maybe anybody has an idea.
This is my expansion of the former code:
namen<-expand.grid(c("weight",NA),
c("height",NA),c("width",NA),c("volume",NA), stringsAsFactors=FALSE)
namen2<-as.data.frame(namen)
for(i in 1:nrow(namen2)){
assign(paste("a", i, sep = ""), namen2[i,])
}
This generates vectors containing the features.
If i pick one of them i can produce a formula that i can use for svm tuning.
For example
a7
a7q<-t(as.data.frame(a7[!is.na(a7)]))
a7q
a7f<-as.formula(paste("type~",paste(a7q,collapse="+")))
a7f
and
svmtune_a7=tune.svm(a7f, data=train, kernel="radial", cost=2^(-2:5),
gamma=2^(-2:1),cross=10)
works as desired.
So my key idea was to tune SVM with every possibel "a...f" formula and
choose the best one according to the best performance measure in the
summary.
Unfortunately I just have problems to make it in a loop.
I tried
for(iin1:nrow(namen2)){paste("a",i,"q",sep="")<-t(as.data.frame(paste("a",
i,"[!is.na(a",i,")]", sep="")))}
and produced error. Probably i didnt paste correctly.
Any ideas?
Thanks a lot!
Uwe
Gesendet: Samstag, 07. Dezember 2013 um 08:26 Uhr
Von: "Wuming Gong" <gongx030 at umn.edu>
An: "Uwe Bohne" <balu555 at gmx.de>
Cc: "r-help mailinglist" <r-help at r-project.org>
Betreff: Re: [R] tune an support vector machine
Hi Uwe,
It looks SVM in e1071 and Kernlab does not support feature selection, but
you can take a look at package penalizedSVM
([1][1]http://cran.r-project.org/web/packages/penalizedSVM/penalizedSVM.pdf)
.
Or you can implement a SVM-RFE
([2][2]http://axon.cs.byu.edu/Dan/778/papers/Feature%20Selection/guyon*.pdf)
by
the alpha values returned by svm() in e1071 or ksvm() in Kernlab.
Wuming
On Fri, Dec 6, 2013 at 7:06 AM, Uwe Bohne <[3]balu555 at gmx.de> wrote:
Hej all,
actually i try to tune a SVM in R and use the package "e1071" wich
works
pretty well.
I do some gridsearch in the parameters and get the best possible
parameters
for classification.
Here is my sample code
type<-sample(c(-1,1) , 20, replace = TRUE )
weight<-sample(c(20:50),20, replace=TRUE)
height<-sample(c(100:200),20, replace=TRUE)
width<-sample(c(30:50),20,replace=TRUE)
volume<-sample(c(1000:5000),20,replace=TRUE)
data<-cbind(type,weight,height,width,volume)
train<-as.data.frame(data)
library("e1071")
features <- c("weight","height","width","volume")
(formula<-as.formula(paste("type ~ ", paste(features, collapse= "+"))))
svmtune=tune.svm(formula, data=train, kernel="radial", cost=2^(-2:5),
gamma=2^(-2:1),cross=10)
summary(svmtune)
My question is if there is a way to tune the features.
So in other words - what i wanna do is to try all possible combinations
of
features : for example use only (volume) or use (weight, height) or use
(height,volume,width) and so on for the SVM and to get the best
combination
back.
Best wishes
Uwe
______________________________________________
[4]R-help at r-project.org mailing list
[5][3]https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
[6][4]http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
References
1. [5]http://cran.r-project.org/web/packages/penalizedSVM/penalizedSVM.pdf
2. [6]http://axon.cs.byu.edu/Dan/778/papers/Feature%20Selection/guyon*.pdf
3. file://localhost/tmp/balu555@gmx.de
4. file://localhost/tmp/R-help@r-project.org
5. [7]https://stat.ethz.ch/mailman/listinfo/r-help
6. [8]http://www.R-project.org/posting-guide.html
______________________________________________
R-help at r-project.org mailing list
[9]https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
[10]http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
References
1. http://cran.r-project.org/web/packages/penalizedSVM/penalizedSVM.pdf
2. http://axon.cs.byu.edu/Dan/778/papers/Feature%20Selection/guyon*.pdf)by
3. https://stat.ethz.ch/mailman/listinfo/r-help
4. http://www.R-project.org/posting-guide.html
5. http://cran.r-project.org/web/packages/penalizedSVM/penalizedSVM.pdf
6. http://axon.cs.byu.edu/Dan/778/papers/Feature%20Selection/guyon*.pdf
7. https://stat.ethz.ch/mailman/listinfo/r-help
8. http://www.R-project.org/posting-guide.html
9. https://stat.ethz.ch/mailman/listinfo/r-help
10. http://www.R-project.org/posting-guide.html
More information about the R-help
mailing list