[R] e1071/svm?

[David Meyer]

OK, after having inspected the code of libsvm, I agree it is not
completely obvious to use the results produced by svm() for prediction.

For class prediction in the binary case, the class of a new data vector
``n'' is usually given by *the sign* of

Sum(a_i * y_i * K(x_i, n)) + rho
 i

where x_i is the i-th support vector, y_i the corresponding label, a_i
the corresponding coefficiant, and K is the kernel (in your case, the
linear one, so 
K(u,v) = u'v).


Now, ``libsvm'' actually returns a_i * y_i as i-th coefficiant and the
*negative* rho, so in fact uses the formula:

Sum(coef_i * K(x_i, n)) - rho
 i

where the training examples (=training data) are labeled {1,-1}.


A simplified R function for prediction with linear kernel would be:

svmpred <- function (m, newdata, K=crossprod) {
  ## this guy does the computation:
  pred.one <- function (x)
    sign(sum(sapply(1:m$tot.nSV, function (j)
                    K(m$SV[j,], x) * m$coefs[j]
                    )
             ) - m$rho
         )

  ## this is just for convenience:
  if (is.vector(newdata))
    newdata <- t(as.matrix(x))
  sapply (1:nrow(newdata),
          function (i) pred.one(newdata[i,]))
}


where ``pred.one'' does the actual prediction for one new data vector,
the rest is just a convenience for prediction of multiple new examples.

It's easy to extend this to other kernels, just replace ``K'' with the
appropriate function [see the help page for the formulas used] and
supply the additional constants.

Note, however, that multi-class prediction is more complicated, because
the coefficiants of the diverse binary svm's are stored in a compressed
format.

And finally, you could directly use the C source code of libsvm, if your
target language supports foreign function calls.

regards,

-d


Alexander Skomorokhov wrote:
> 
> Thank you fot your reply. Sorry, but I still haven't got the problem.
> Here is a trivial example (cpy from R session):
> ----------------------------------------------------------------------------
> ------------------
> > x
>      [,1] [,2]
> [1,]    0    0
> [2,]    0    1
> [3,]    1    0
> [4,]    1    1
> [5,]    2    2
> [6,]    2    3
> [7,]    3    2
> [8,]    3    3
> > y
> [1] 1 1 1 1 2 2 2 2
> Levels:  1 2
> > is.factor(y)
> [1] TRUE
> > library(e1071)
> > m<-svm(x,y,kernel='linear')
> *
> optimization finished, #iter = 3
> nu = 0.250000
> obj = -1.000000, rho = -3.000000
> nSV = 2, nBSV = 2
> Total nSV = 2
> 
> > summary(m)
> Call:
>  svm.default(x = x, y = y, kernel = "linear")
> Parameters:
>    SVM-Type:  C-classification
>  SVM-Kernel:  linear
>        cost:  1
>      degree:  3
>       gamma:  0.5
>      coef.0:  0
>          nu:  0.5
>     epsilon:  0.5
>        cost:  1
> Number of Support Vectors:  2  ( 1 1 )
> Number of Classes:  2
> Levels:
>  1 2
> Rho:
>  -3
> Support Vectors:
>      [,1] [,2]
> [1,]    1    1
> [2,]    2    2
> Coefficiants:
>      [,1]
> [1,]    1
> [2,]   -1
> ----------------------------------------------------------------------------
> ----------
> I see Coefficiants, but can't guess how to use them (only them?) for
> prediction???
> I may guess that the question sounds like a stupid one, but it is that:-(
> 
> Thanks,
> Alexander.
> 
> > -----Original Message-----
> > From: meyer at ci.tuwien.ac.at [mailto:meyer at ci.tuwien.ac.at]On
> > Behalf Of David Meyer
> > Sent: Thursday, December 20, 2001 3:03 PM
> > To: askom at obninsk.com
> > Cc: r-help
> > Subject: Re: [R] e1071/svm?
> >
> >
> > Alexander Skomorokhov wrote:
> > >
> > > Hello,
> > >
> > > I use function "svm" (interface to libsvm) from package e1071.
> > It works just
> > > fine.
> > > And I may predict with function "predict" and svm model trained
> > by function
> > > "svm".
> > > What I need is moving results of svm training to another
> > application (non-R)
> > > and
> > > perform prediction there. But function "svm" returns list of
> > support vectors
> > > only
> > > and doesn't return coefficients of separating hyperplane (w).
> >
> > It does (element ``coefs'' of the returned object).
> >
> > g.
> > -d
> >
> > >
> > > So, the question is how to use results of svm training to write
> > (in other
> > > language)
> > > prediction function for linear and nonlinear cases?
> > >
> > > Thanks,
> > > Alexander.
> > >
> > >
> > -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.
> > -.-.-.-.-.-.-
> > > r-help mailing list -- Read
> http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
> > Send "info", "help", or "[un]subscribe"
> > (in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
> >
> _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._.
> _._
> 
> --
>         Mag. David Meyer                Wiedner Hauptstrasse 8-10
> Vienna University of Technology         A-1040 Vienna/AUSTRIA
>        Department for                   Tel.: (+431) 58801/10772
> Statistics and Probability Theory       mail: david.meyer at ci.tuwien.ac.at
> 
> -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
> r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
> Send "info", "help", or "[un]subscribe"
> (in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
> _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._

-- 
	Mag. David Meyer		Wiedner Hauptstrasse 8-10
Vienna University of Technology		A-1040 Vienna/AUSTRIA
       Department for			Tel.: (+431) 58801/10772
Statistics and Probability Theory	mail: david.meyer at ci.tuwien.ac.at
-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._