[RsR] OGK covariance estimator

[Martin Maechler]

>>>>> "Peter" == Peter Filzmoser <P.Filzmoser using tuwien.ac.at>
>>>>>     on Mon, 05 Dec 2005 09:02:54 +0100 writes:

    Peter> Martin Maechler wrote:
    >> But indeed, that shouldn't be a problem, since I think
    >> one should program the OGK using R (S) code alone {just
    >> using eigen() and other matrix operations}, and have the
    >> univariate scale estimate be a plug-in, i.e., a function
    >> argument.  There, one could use Qn, Sn {I now have these
    >> two in the not yet-released package on "basic robust
    >> statistic"},

    Peter> Martin, we are working now on a fast implementation
    Peter> of the Qn, using C++ code in R. Have you done that
    Peter> similarly?

I have based it on C code which had been based on the original
Fortran code from Croux and Rousseeuw available from Antwerpen.
Is there a faster algorithm now?

Given the traffic on this list, 
and the not-yet-publication of the Treviso working group
"minutes", let me open the topic to which I alluded above as
well: Our working group ("regression", later joined by
"econometrics") talked about the possibility and then
desirability of an R package on "(basic) robust statitics" which

 1)  will be backed up by many `robustniks'
 2)  will remain focused, and hence (at least initially)
     primarily based on the methodology in the upcoming book 
     by Maronna, Martin and Yohai (=: MMY); and even from there,
     primarily on
      a) regression [including GLM, non-lin reg, maybe a bit more]
      b) multivariate : "location + scatter"
 3)  have quite a few authors (in the work group, we had some "commitments"),
     with me as coordinator and maintainer.
 4)  other "Robustness in R" packages would *build*
     (i.e. "Depend" in  R package lingo) on that basic package,
     and authors that contribute code from their own package
     would of course remain authors and would also keep the
     functions in their own packages (for a while at least; later their
     package will load or even attach the "basic robust"
     package, and hence have the functionality available indirectly).

-- more details are in the minutes of our workgroup(s) that I've
   sent to Matias {as Treviso co-organizer} to be put on the
   (Treviso RSR) webpage.

I think it would be great to get feedback from the R-SIG-robust
audience on the above; particularly if you think that the idea
is problematic or needs amendments/improvements before being
workable.

Last week, I had added the following to (my latex version of) the minutes:

------------------------

 Later notes -- by Martin Maechler, only
 ---------------------------------------

  o  Proposed names for the `basic robust statistics' package:
     1. robstats
     2. robbase
     3. baseRob

   I'm choosing the first one for now.

  o I've added (Rousseeuw & Croux)'s Qn and Sn estimators of
    scale, based on an R package I had started in 2002, which is
    based on the Fortran code from Antwerpen. 

  \item I plan to add ``\emph{psi - Function}'' objects, using an S4 class,
    with instances for Huber, Hampel, Biweight, etc, at least for the
    M-estimators (of location / regression).
    Such an object should contain  $\rho$, $\psi$, $w(x) := \psi(x)/x$,
    $\psi'$ (= $d/\mathrm{dx} \psi$) functions, default values for the tuning
    constants, and functionals such as $E_X[\psi(X)^2]$,  $E_X[\psi'(X)]$,
    for $X\sim N(0,1)$ (these functionals are still \emph{functions} of the
    tuning parameters).

    Very similarly, I'm interested also in the $\chi$ functions used for
    B-/V- optimal M-estimators of \emph{scale}, both the monotone and the
    redescending ones.

    In general, I think we should add ``\textbf{basic M-estimation}''
    things to the package as well, similar to, but
    more general than \code{MASS::huber}; note that I'd like to
    contribute \code{sfsmisc::huberM} (my `improved' \code{huber})
    to the \pkg{robstats} as well.

------------------------

and actually I've done the S4 class for "psi-function" objects
last Friday.

Now I'm interested to hear more..
Martin