Christoph Lehman had problems with seperated data in two-class logistic regression.

One useful little trick is to penalize the logistic regression using a quadratic penalty on the coefficients.
I am sure there are functions in the R contributed libraries to do this; otherwise it is easy to achieve via IRLS
using ridge regressions. Then even though the data are separated, the penalized log-likelihood 
has a unique maximum. One intriguing feature is that as the penalty parameter goes to zero, 
the solution converges to the SVM solution - i.e. the optimal separating hyperplane
see  http://www-stat.stanford.edu/~hastie/Papers/margmax1.ps 

--------------------------------------------------------------------
  Trevor Hastie                                  hastie@stanford.edu  
  Professor, Department of Statistics, Stanford University
  Phone: (650) 725-2231 (Statistics)        Fax: (650) 725-8977  
    (650) 498-5233 (Biostatistics)   Fax: (650) 725-6951
  URL: http://www-stat.stanford.edu/~hastie  
  address: room 104, Department of Statistics, Sequoia Hall
           390 Serra Mall, Stanford University, CA 94305-4065  
--------------------------------------------------------------------

	[[alternative HTML version deleted]]


