[R] kernlab's ksvm method freeze

Heiko Strathmann heiko.strathmann at googlemail.com
Wed Dec 21 14:05:10 CET 2011


Hi Wuming,

it has been a while. I was not able to fix this problem these day.
Also I could not get in contact with the developer, so I switched to
SHOGUN, which has an interface for R.
With the SHOGUN interface to libsvm, everything worked fine. Also, its
a cool package and worth that you check it out :)
Cheers,
Heiko

2011/12/21 Wuming Gong <gongx030 at umn.edu>:
> Hi Heiko,
>
> I run into a similar problem recently.  I have a custom kernel which have
> some small negative eigenvalues, possibly due to numerical error.  I first
> used ksvm(K, y) to train the model, but ksvm() freeze (no response to
> Ctrl+c).  I thought it was due to positive definiteness of the matrix, so I
> used nearPD() from Matrix package to correct kernel.  This sometimes solved
> the freeze problem.  But sometimes ksvm() still freeze even with the
> corrected matrix with all positive eigenvalues. Then I checked document of
> libsvm and found the following FAQ
>
>
> Q: The training time is too long. What should I do?
> For large problems, please specify enough cache size (i.e., -m). Slow
> convergence may happen for some difficult cases (e.g. -c is large). You can
> try to use a looser stopping tolerance with -e. If that still doesn't work,
> you may train only a subset of the data. You can use the program subset.py
> in the directory "tools" to obtain a random subset.
>
> If you have extremely large data and face this difficulty, please contact
> us. We will be happy to discuss possible solutions.
>
> When using large -e, you may want to check if -h 0 (no shrinking) or -h 1
> (shrinking) is faster. See a related question below.
>
>
> So I changed tor from 0.001 to 0.01 and turn off shrinking heuristic. This
> seems working well for most of the other freeze problems.
>
> Wuming
>
>
> On Tue, Dec 1, 2009 at 5:46 AM, Heiko Strathmann
> <heiko.strathmann at googlemail.com> wrote:
>> Sorry i forget to paste it.
>> But it should be there in another mail.
>>
>> Also, the suggested trick fixes the problem in with the given workspace,
>> but still fails in another one (where the kernel matrix IS positive
>> definite)
>> http://www-stud.uni-due.de/~sfhestra/freeze_on_pos_def_matrix.tar.gz
>>
>> This is not a R-workspace, but my zipped working directory.
>> The file I run is parameterSearch.R.
>> The freeze happens more or less late in the search, but not at a fixed
>> point.
>>
>>
>> Heiko
>>
>> Am Dienstag, den 01.12.2009, 10:25 +0100 schrieb Uwe Ligges:
>>>
>>> Heiko Strathmann wrote:
>>> > Hello again,
>>> >
>>> > i got this message from the maintainer:
>>>
>>> And which part is from the maintainer?
>>>
>>> Uwe
>>>
>>>
>>>
>>> >
>>> > Am Montag, den 30.11.2009, 11:22 +0100 schrieb Heiko Strathmann:
>>> >> Ok, i reported it.
>>> >> thanks for trying it out again.
>>> >>
>>> >> Am 30. November 2009 11:06 schrieb Uwe Ligges
>>> >> <ligges at statistik.tu-dortmund.de>:
>>> >>         My apologies, that must have been a copy&paste error and the
>>> >>         essential argument got lost. I can reproduce it now under
>>> >>         R-2.10.0 both Windows and Linux.
>>> >>
>>> >>         Please report your findings to the package maintainer who
>>> >>         might be able to debug this under Linux (probably easier than
>>> >>         under Windows).
>>> >>
>>> >>         Best,
>>> >>
>>> >>
>>> >>         Uwe Ligges
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>         Heiko Strathmann wrote:
>>> >>                 I tried out the code you wrote, it also works for me,
>>> >>                 but it lacks a
>>> >>                 parameter i use in my code.
>>> >>
>>> >>                 The problem (at this computer) seems to be this
>>> >>                 "cross"-parameter of
>>> >>                 ksvm - if I, for example, add the parameter cross=10,
>>> >>                 i get the old
>>> >>                 problem:
>>> >>
>>> >>                 library("kernlab")
>>> >>                 load("freeze_workspace.RDATA")
>>> >>                 replicate(10, ksvm(kernel="matrix", kernelMatrix,
>>> >>                 trainingDataYs, type="C-svc", C=2, cross=10))
>>> >>
>>> >>                 gets me a frozen R process, CTRL-C does not work
>>> >>                 anymore, and the only thing left is to kill it.
>>> >>
>>> >>                 (for cross < 4, the thing still works)
>>> >>
>>> >>                 (I also just reinstalled my Ubuntu and R)
>>> >>
>>> >>                 Heiko Strathmann
>>> >>
>>> >>
>>> >>                 Am Sonntag, den 29.11.2009, 19:52 +0100 schrieb Uwe
>>> >>                 Ligges:
>>> >>                         Heiko Strathmann wrote:
>>> >>                                 Hello uwe,
>>> >>                                 Thanks for trying out.
>>> >>                                 the freeze happens after about 10 to
>>> >>                                 20 iterations. Did you try as many?
>>> >>                         I just tried again:
>>> >>
>>> >>                         library("kernlab")
>>> >>                         load("freeze_workspace.RDATA")
>>> >>                         replicate(100, ksvm(kernel="matrix",
>>> >>                         kernelMatrix, trainingDataYs, type="C-svc",
>>> >>                         C=2))
>>> >>
>>> >>                         and everything is still fine (same on Linux).
>>> >>
>>> >>                         Uwe Ligges
>>> >>
>>> >>
>>> >>
>>> >>                                 Am Sonntag, den 29.11.2009, 17:22
>>> >>                                 +0100 schrieb Uwe Ligges:
>>> >>                                         I just tried
>>> >>
>>> >>                                         ksvm(kernel="matrix",
>>> >>                                         kernelMatrix, trainingDataYs,
>>> >>                                         type="C-svc", cross=10, C=2)
>>> >>
>>> >>                                         several times on both
>>> >>                                         workspaces and both returned
>>> >>                                         some results after a couple of
>>> >>                                         seconds under the same
>>> >>                                         versions (R version 2.10.0 and
>>> >>                                         kernlab 0.9-9.) under Windows
>>> >>                                         XP.
>>> >>
>>> >>                                         There mist be something else
>>> >>                                         going on...
>>> >>
>>> >>                                         Best wishes,
>>> >>                                         Uwe Ligges
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>                                         Heiko Strathmann wrote:
>>> >>                                                 Hello again,
>>> >>
>>> >>                                                 the freeze seems to
>>> >>                                                 depend on the kernel
>>> >>                                                 matrix.
>>> >>                                                 With another kernel
>>> >>                                                 matrix of similiar
>>> >>                                                 size, gernerated with
>>> >>                                                 the same
>>> >>                                                 kernel, but on another
>>> >>                                                 dataset, there is no
>>> >>                                                 freeze.
>>> >>
>>> >>                                                 I have put a workspace
>>> >>                                                 with the working
>>> >>                                                 matrix and one with
>>> >>                                                 the freezing
>>> >>                                                 matrix online for
>>> >>                                                 testing (see old
>>> >>                                                 email)
>>> >>
>>> >> http://www-stud.uni-due.de/~sfhestra/
>>> >>
>>> >>                                                 In my eyes this
>>> >>                                                 behavior is really
>>> >>                                                 strange, and i have no
>>> >>                                                 clue, what to
>>> >>                                                 do to solve this.
>>> >>
>>> >>                                                 Regards,
>>> >>                                                 Heiko Strathmann
>>> >>
>>> >>                                                 Am Sonntag, den
>>> >>                                                 29.11.2009, 14:21
>>> >>                                                 +0100 schrieb Heiko
>>> >>                                                 Strathmann:
>>> >>                                                         Hello,
>>> >>
>>> >>                                                         I am using
>>> >>                                                         kernlab to do
>>> >>                                                         some binary
>>> >>                                                         classification
>>> >>                                                         on aminoacid
>>> >>                                                         strings.
>>> >>
>>> >>                                                         I am using a
>>> >>                                                         custom kernel,
>>> >>                                                         so i use the
>>> >>



More information about the R-help mailing list