[R] nearest correlation to polychoric

[Martin Maechler]

>>>>> "DR" == Dimitris Rizopoulos <dimitris.rizopoulos at med.kuleuven.be>
>>>>>     on Fri, 13 Jul 2007 14:43:08 +0200 writes:

    DR> you could also have a look at function posdefify() from
    DR> package `sfsmisc'.

    DR> I hope it helps.

Yes, thanks, Dimitris; note that my posdefify() function uses a
pretty arbitrary "fudge" value for posdefification, namely
      eps.ev = 1e-07

As a matter of fact, earlier this week (in "the first
international R/Rmetrics workshop"),
I've talked to people from finance who also need that (or
something better?).

Jordi Molins Coronado (Madrid) drew my attention to an idea
he found in the book (English re-edition of French as of 1996)
Jean-Philippe Bouchaud (2000)
    Theory of Financial Risk and Derivative Pricing:
    From Statistical Physics to Risk Management

which supposedly uses theory of random matrices and
the entailing distribution of random eigenvalues in order to
find a more sensible cutoff than my 'eps.ev' default of 1e-7.
Unfortunately that book is checked out from our library and I
can't have a look.   Googling and Wikipedia seem to indicate to
me that most of the random matrix theory does not directly apply
here, since I'm really interested in the spectrum of X'X where X
is a de-meaned n x p random matrix.


OTOH, help(posdefify) already mentions more sophisticated
approaches to the problem, the one I (as I vaguely remember)
should be made available being 

  Cheng, Sheung Hun and Higham, Nick (1998)
  A Modified Cholesky Algorithm Based on a Symmetric Indefinite Factorization;
  \emph{SIAM J. Matrix Anal.\ Appl.}, \bold{19}, 1097--1110.


Jens, could you make your code (mentioned below) available to
the community, or even donate to be included as a new method of
posdefify() ?

Regards,
Martin Maechler, ETH Zurich


    DR> Best, Dimitris

    DR> ---- Dimitris Rizopoulos Ph.D. Student Biostatistical
    DR> Centre School of Public Health Catholic University of
    DR> Leuven

    DR> Address: Kapucijnenvoer 35, Leuven, Belgium Tel:
    DR> +32/(0)16/336899 Fax: +32/(0)16/337015 Web:
    DR> http://med.kuleuven.be/biostat/
    DR> http://www.student.kuleuven.be/~m0390867/dimitris.htm


    DR> ----- Original Message ----- From: ""Jens Oehlschlägel""
    DR> <oehl_list at gmx.de> To: <r-help at stat.math.ethz.ch> Sent:
    DR> Friday, July 13, 2007 2:25 PM Subject: [R] nearest
    DR> correlation to polychoric


    DR> Dear all,

    DR> Has someone implemented in R (or any other language)

    DR> Knol DL, ten Berge JMF.  Least-squares approximation of
    DR> an improper correlation matrix by a proper one.
    DR> Psychometrika, 1989, 54, 53-61.

    DR> or any other similar algorithm?

    DR> Best regards


    DR> Jens Oehlschlägel


    DR> Background:

    DR> I want to factanal() matrices of polychoric correlations
    DR> which have negative eigenvalue. I coded

    DR> Highham 2002 Computing the nearest correlation matrix -
    DR> a problem from finance, IMA Journal of Numerical
    DR> Analysis (2002), 22, 329-343.

    DR> which basically works but still leaves very small
    DR> negative eigenvalues which causes factanal() to fail
    DR> with

    >> factanal(cov=ncor$cor, factors=2)
    DR> Fehler in optim(start, FAfn, FAgr, method = "L-BFGS-B",
    DR> lower = lower, : L-BFGS-B benötigt endliche Werte von
    DR> 'fn' Zusätzlich: Warning message: NaNs wurden erzeugt
    DR> in: log(x)
    >> version
    DR>                _ platform i386-pc-mingw32 arch i386 os
    DR> mingw32 system i386, mingw32 status major 2 minor 4.1
    DR> year 2006 month 12 day 18 svn rev 40228 language R
    DR> version.string R version 2.4.1 (2006-12-18)