[R] Recursive Feature Elimination with SVM

[David Winsemius]

On 1/1/19 5:31 PM, Priyanka Purkayastha wrote:
> Thankyou David.. I tried the same, I gave x as the data matrix and y 
> as the class label. But it returned an empty "featureRankedList". I 
> get no output when I try the code.


If you want people to spend time on this you should post a reproducible 
example. See the Posting Guide ... and learn to post in plain text.


--

David

>
> On Tue, 1 Jan 2019 at 11:42 PM, David Winsemius 
> <dwinsemius using comcast.net <mailto:dwinsemius using comcast.net>> wrote:
>
>
>     On 1/1/19 4:40 AM, Priyanka Purkayastha wrote:
>     > I have a dataset (data) with 700 rows and 7000 columns. I am
>     trying to do
>     > recursive feature selection with the SVM model. A quick google
>     search
>     > helped me get a code for a recursive search with SVM. However, I
>     am unable
>     > to understand the first part of the code, How do I introduce my
>     dataset in
>     > the code?
>
>
>     Generally the "labels" is given to such a machine learning device
>     as the
>     y argument, while the "features" are passed as a matrix to the x
>     argument.
>
>
>     -- 
>
>     David.
>
>     >
>     > If the dataset is a matrix, named data. Please give me an
>     example for
>     > recursive feature selection with SVM. Bellow is the code I got for
>     > recursive feature search.
>     >
>     >      svmrfeFeatureRanking = function(x,y){
>     >
>     >      #Checking for the variables
>     >      stopifnot(!is.null(x) == TRUE, !is.null(y) == TRUE)
>     >
>     >      n = ncol(x)
>     >      survivingFeaturesIndexes = seq_len(n)
>     >      featureRankedList = vector(length=n)
>     >      rankedFeatureIndex = n
>     >
>     >      while(length(survivingFeaturesIndexes)>0){
>     >      #train the support vector machine
>     >      svmModel = svm(x[, survivingFeaturesIndexes], y, cost = 10,
>     > cachesize=500,
>     >                  scale=FALSE, type="C-classification",
>     kernel="linear" )
>     >
>     >      #compute the weight vector
>     >      w = t(svmModel$coefs)%*%svmModel$SV
>     >
>     >      #compute ranking criteria
>     >      rankingCriteria = w * w
>     >
>     >      #rank the features
>     >      ranking = sort(rankingCriteria, index.return = TRUE)$ix
>     >
>     >      #update feature ranked list
>     >      featureRankedList[rankedFeatureIndex] =
>     > survivingFeaturesIndexes[ranking[1]]
>     >      rankedFeatureIndex = rankedFeatureIndex - 1
>     >
>     >      #eliminate the feature with smallest ranking criterion
>     >      (survivingFeaturesIndexes =
>     survivingFeaturesIndexes[-ranking[1]])}
>     >      return (featureRankedList)}
>     >
>     >
>     >
>     > I tried taking an idea from the above code and incorporate the
>     idea in my
>     > code as shown below
>     >
>     >      library(e1071)
>     >      library(caret)
>     >
>     >      data<- read.csv("matrix.csv", header = TRUE)
>     >
>     >      x <- data
>     >      y <- as.factor(data$Class)
>     >
>     >      svmrfeFeatureRanking = function(x,y){
>     >
>     >        #Checking for the variables
>     >        stopifnot(!is.null(x) == TRUE, !is.null(y) == TRUE)
>     >
>     >        n = ncol(x)
>     >        survivingFeaturesIndexes = seq_len(n)
>     >        featureRankedList = vector(length=n)
>     >        rankedFeatureIndex = n
>     >
>     >        while(length(survivingFeaturesIndexes)>0){
>     >          #train the support vector machine
>     >          svmModel = svm(x[, survivingFeaturesIndexes], y,
>     cross=10,cost =
>     > 10, type="C-classification", kernel="linear" )
>     >
>     >          #compute the weight vector
>     >          w = t(svmModel$coefs)%*%svmModel$SV
>     >
>     >          #compute ranking criteria
>     >          rankingCriteria = w * w
>     >
>     >          #rank the features
>     >          ranking = sort(rankingCriteria, index.return = TRUE)$ix
>     >
>     >          #update feature ranked list
>     >          featureRankedList[rankedFeatureIndex] =
>     > survivingFeaturesIndexes[ranking[1]]
>     >          rankedFeatureIndex = rankedFeatureIndex - 1
>     >
>     >          #eliminate the feature with smallest ranking criterion
>     >          (survivingFeaturesIndexes =
>     survivingFeaturesIndexes[-ranking[1]])}
>     >
>     >        return (featureRankedList)}
>     >
>     > But couldn't do anything at the stage "update feature ranked list"
>     > Please guide
>     >
>     >       [[alternative HTML version deleted]]
>     >
>     > ______________________________________________
>     > R-help using r-project.org <mailto:R-help using r-project.org> mailing list
>     -- To UNSUBSCRIBE and more, see
>     > https://stat.ethz.ch/mailman/listinfo/r-help
>     > PLEASE do read the posting guide
>     http://www.R-project.org/posting-guide.html
>     > and provide commented, minimal, self-contained, reproducible code.
>
> -- 
> Regards,
>
> Priyanka Purkayastha, M.Tech, Ph.D.,
> SERB National Postdoctoral Researcher
> Genomics and Systems Biology Lab,
> Department of Chemical Engineering,
> Indian Institute of Technology Bombay (IITB),
> Powai, Mumbai- 400076
>
>
>