[R] Difficulty understanding sem errors / failed confirmatory factor analysis

[John Fox]

Dear Adam,

> -----Original Message-----
> From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org]
On
> Behalf Of Adam D. I. Kramer
> Sent: September-18-08 1:02 PM
> To: r-help at r-project.org
> Subject: [R] Difficulty understanding sem errors / failed confirmatory
factor
> analysis
> 
> Hello,
> 
>  	I'm trying to fit a pretty simple confirmatory factor analysis using
> the sem package. There's a CFA example in the examples, which is helpful,
> but the output for my (failing) model is hard to understand. I'd be
> interested in any other ways to do a CFA in R, if this proves troublesome.
> 
>  	The CFA is replicating a 5 uncorrelated-factor structure (for those
> interested, it is a structure of word usage patterns in weblogs) in a
> special population. The model looks like model.txt (attached as many
people
> hate long emails); the correlation matrix cors.txt as well.

As far as I can see, the attachments aren't there. If you like, you can send
them to me privately. Without the input covariance matrix and your model,
it's very hard to tell what the source of the problem is, but one guess
(assuming that you've specified the model correctly) is that the assumption
of uncorrelated factors is too far off. Also see below.

> 
>  	I'm setting no overlap between factors, no correlation between
> factors, and estimating a separate variance for each observed variable
> (which should be everything on the right-hand side of the -> arrows), but
> setting the factor variances equal to 1...pretty standard. I've ensured
that
> everything is typed correctly to the best I am able.
> 
>  	The problem:
> 
> library(sem)
> model.kr <- specify.model(file="model.txt") # printing it checks out ok
> correl <- read.csv("cors.csv", header=TRUE) # printing it checks out ok
> kr.sem <- sem(ram=model.kr,S=correl,N=3034)
> ...about 10 seconds pass...
> Warning message:
> In sem.default(ram = ram, S = S, N = N, param.names = pars, var.names =
vars,
> :
>    Could not compute QR decomposition of Hessian.
> Optimization probably did not converge.
> 
> (running qr on correl works fine; randomly-generated correl matrices fail
in
> the same way; I do not know how to further troubleshoot this)

Doing a QR decomposition on the correlation matrix of the data is
essentially irrelevant. The issue is the Hessian. (The scaled inverse
Hessian is the covariance matrix of the parameter estimates, not of the
data.) That you observe similar problems for randomly generated covariance
matrices may or may not be troublesome, depending upon how you generated
them.

> 
> ...and then the model itself (which is produced, as the above was just a
> warning):
> 
> summary(kr.sem)
> Error in data.frame(object$coeff, se, z, 2 * (1 - pnorm(abs(z))),
par.code) :
>    arguments imply differing number of rows: 47, 0

If the Hessian isn't positive-definite, it won't be possible to get
estimated coefficient standard errors. I suspect that this is the source of
this error message. If so, it would be better for summary.sem() to provide a
more informative error message.

Regards,
 John

> 
> ...both of these error messages are beyond my ability to troubleshoot. Any
> help would be greatly appreciated. Because I am unsure what exactly the
> problem with this analysis is, I can't create a simpler example for
testing
> purposes...but I think my model and correlation matrix are fairly simple.
> 
> > unlist(R.Version())
>                        platform                           arch
>      "x86_64-unknown-linux-gnu"                       "x86_64"
>                              os                         system
>                     "linux-gnu"            "x86_64, linux-gnu"
>                          status                          major
>                              ""                            "2"
>                           minor                           year
>                           "7.2"                         "2008"
>                           month                            day
>                            "08"                           "25"
>                         svn rev                       language
>                         "46428"                            "R"
>                  version.string
> "R version 2.7.2 (2008-08-25)"
> 
> ...sem installed via install.packages("sem") which I assume is current.
> 
> Cordially,
> Adam Kramer
> 
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.