[R] Questions on implementing logistic regression

[Bert Gunter]

Perhaps I should have added and FWIW, R, like essentially all
statistical software, has "logistic regression" already built in, if I
understand what you mean by the term (which I may not), via glm's.

-- Bert

On Tue, Mar 5, 2013 at 1:36 PM, Ivan Li <machinelearning2010 at gmail.com> wrote:
> Hi there,
>
> I am trying to write a tool which involves implementing logistic
> regression. With the batch gradient descent method, the convergence is
> guaranteed as it is a convex problem. However, I find that with the
> stochastic gradient decent method, it typically converges to some random
> points (i.e., not very close to the minimum point resulted from the batch
> method). I have tried different ways of decreasing the learning rate, and
> different starting points of weights. However, the performance (e.g.,
> accuracy, precision/recall, ...) are comparable (to the batch method).
>
> I understand that this is possible, since SGD(stochastic gradient descent)
> uses an approximation to the real cost each step. Does it matter? I guess
> it does since otherwise the interpretation of the weights would not make
> much sense even the accuracy is comparable. If it matters, I wonder if you
> have some suggestions on how to make it converge or getting close to the
> global optimal point.
>
>
>
> Thanks!
>
>         [[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.



-- 

Bert Gunter
Genentech Nonclinical Biostatistics

Internal Contact Info:
Phone: 467-7374
Website:
http://pharmadevelopment.roche.com/index/pdb/pdb-functional-groups/pdb-biostatistics/pdb-ncb-home.htm