1101011 at gmx.net
Wed Oct 19 19:12:34 CEST 2016
>From my reading of the above cited document I get the impression that the algorithm
(algorithm 3.16, p27) can easily be adapted to handle the case m<n.
In this case the Jacobian Jf(x) is mxn and the matrix A(x)=Jf(x)'Jf(x) is nxn and rank deficient.
This seems to be a problem since the search direction h at iterate x is computed from
where mu -> 0 and so the system becomes ill conditioned.
Why can we not get around this as follows: as soon as mu is below some threshold
we solve instead the seemingly worse A(x)h=-J(x)'f(x) which however is of the form
and can be solved by solving J(x)h=-f(x) and the condition for this to be solvable
is that J(x) have full rank. In fact this is the Gauss-Newton step
and to switch to that in the case of small mu is exactly in keeping with the idea of the
If h is _any_ solution we have h'F'(x)=-||J(x)h||²<=0 (document, p21, eq(3.10))
and is a descent direction if this is in fact < 0.
If it is equal to zero, then f(x)=J(x)h=0 and we have a global minimum at x.
More information about the R-help