[R] Difficulty understanding sem errors / failed confirmatory factor analysis
John Fox
jfox at mcmaster.ca
Thu Sep 18 20:56:42 CEST 2008
Dear Adam,
> -----Original Message-----
> From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org]
On
> Behalf Of Adam D. I. Kramer
> Sent: September-18-08 2:33 PM
> To: John Fox
> Cc: r-help at r-project.org
> Subject: Re: [R] Difficulty understanding sem errors / failed confirmatory
> factor analysis
>
> Hi John,
>
> Thanks very much for your response. It does appear now that my
> correlation matrix and not your software is the problem. I apologize for
> wasting your time! I do think that a more informative error message may
have
> prompted me to consider this possibility more fully.
Agreed. I've put a more informative error message into summary.sem() in the
development version of the package on R-Forge. I think that I'll also add a
check for singularity of the input covariance matrix.
Regards,
John
>
> --Adam
>
> On Thu, 18 Sep 2008, John Fox wrote:
>
> > Dear Adam,
> >
> > (1) Note that your input correlation matrix appears to be numerically
> > singular:
> >
> >> solve(R)
> > Error in solve.default(R) :
> > system is computationally singular: reciprocal condition number =
> > 2.38183e-17
> >> det(R)
> > [1] -1.753523e-25
> >> qr(R)$rank
> > [1] 23
> >
> > (2) In addition, you have specified what are essentially redundant
> > constraints on the model, fixing *both* the loading for a "reference"
> > indicator for each factor to 1 *and* fixing the factor variances to 1.
One
> > would normally do one or the other.
> >
> > I think that (1) rather than (2) is the essential source of the problem,
> and
> > fixing (2) doesn't make the problem go away.
> >
> > Because sem() can't compute the covariance matrix of the estimated
> > parameters, this component is missing from the returned object, causing
the
> > cryptic error message in summary.sem(). I'll provide a more informative
> > error or warning.
> >
> > I hope this helps,
> > John
> >
> > ------------------------------
> > John Fox, Professor
> > Department of Sociology
> > McMaster University
> > Hamilton, Ontario, Canada
> > web: socserv.mcmaster.ca/jfox
> >
> >> -----Original Message-----
> >> From: r-help-bounces at r-project.org
[mailto:r-help-bounces at r-project.org]
> > On
> >> Behalf Of Adam D. I. Kramer
> >> Sent: September-18-08 1:37 PM
> >> To: r-help at r-project.org
> >> Subject: Re: [R] Difficulty understanding sem errors / failed
confirmatory
> >> factor analysis
> >>
> >> No new info, but the model and correlation table are pasted at the end
of
> >> this message.
> >>
> >> --Adam
> >>
> >> On Thu, 18 Sep 2008, Adam D. I. Kramer wrote:
> >>
> >>> Hello,
> >>>
> >>> I'm trying to fit a pretty simple confirmatory factor analysis using
> >>> the sem package. There's a CFA example in the examples, which is
> > helpful,
> >>> but the output for my (failing) model is hard to understand. I'd be
> >>> interested in any other ways to do a CFA in R, if this proves
> > troublesome.
> >>>
> >>> The CFA is replicating a 5 uncorrelated-factor structure (for those
> >>> interested, it is a structure of word usage patterns in weblogs) in a
> >>> special population. The model looks like model.txt (attached as many
> > people
> >>> hate long emails); the correlation matrix cors.txt as well.
> >>>
> >>> I'm setting no overlap between factors, no correlation between
> >>> factors, and estimating a separate variance for each observed variable
> >>> (which should be everything on the right-hand side of the -> arrows),
> > but
> >>> setting the factor variances equal to 1...pretty standard. I've
ensured
> >> that
> >>> everything is typed correctly to the best I am able.
> >>>
> >>> The problem:
> >>>
> >>> library(sem)
> >>> model.kr <- specify.model(file="model.txt") # printing it checks out
ok
> >>> correl <- read.csv("cors.csv", header=TRUE) # printing it checks out
ok
> >>> kr.sem <- sem(ram=model.kr,S=correl,N=3034)
> >>> ...about 10 seconds pass...
> >>> Warning message:
> >>> In sem.default(ram = ram, S = S, N = N, param.names = pars, var.names
=
> >> vars,
> >>> :
> >>> Could not compute QR decomposition of Hessian.
> >>> Optimization probably did not converge.
> >>>
> >>> (running qr on correl works fine; randomly-generated correl matrices
> > fail
> >> in
> >>> the same way; I do not know how to further troubleshoot this)
> >>>
> >>> ...and then the model itself (which is produced, as the above was just
a
> >>> warning):
> >>>
> >>> summary(kr.sem)
> >>> Error in data.frame(object$coeff, se, z, 2 * (1 - pnorm(abs(z))),
> > par.code)
> >> :
> >>> arguments imply differing number of rows: 47, 0
> >>>
> >>> ...both of these error messages are beyond my ability to troubleshoot.
> > Any
> >>> help would be greatly appreciated. Because I am unsure what exactly
the
> >>> problem with this analysis is, I can't create a simpler example for
> > testing
> >>> purposes...but I think my model and correlation matrix are fairly
> > simple.
> >>>
> >>>> unlist(R.Version())
> >>> platform arch
> >>> "x86_64-unknown-linux-gnu" "x86_64"
> >>> os system
> >>> "linux-gnu" "x86_64, linux-gnu"
> >>> status major
> >>> "" "2"
> >>> minor year
> >>> "7.2" "2008"
> >>> month day
> >>> "08" "25"
> >>> svn rev language
> >>> "46428" "R"
> >>> version.string "R version 2.7.2 (2008-08-25)"
> >>>
> >>> ...sem installed via install.packages("sem") which I assume is
current.
> >>>
> >>> Cordially,
> >>> Adam Kramer
> >>>
> >>
> >> model.kr <- specify.model()
> >> Melancholy -> Affect, mel.aff, NA
> >> Melancholy -> Negemo, mel.neg, NA
> >> Melancholy -> Sad, mel.sad, NA
> >> Melancholy -> Physcal, mel.phys, NA
> >> Melancholy -> Body, mel.phys, NA
> >> Melancholy -> Eating, mel.eat, NA
> >> Melancholy -> Groom, NA, 1
> >> Social -> numlines, soc.nrow, NA
> >> Social -> Leisure, soc.leis, NA
> >> Social -> Home, soc.home, NA
> >> Social -> Sports, soc.sports, NA
> >> Social -> TV, soc.tv, NA
> >> Social -> Music, soc.mus, NA
> >> Social -> Money, NA, 1
> >> Rant -> Swear, rant.swear, NA
> >> Rant -> Sexual, rant.sex, NA
> >> Rant -> Anger, rant.anger, NA
> >> Rant -> I, NA, 1
> >> Metaphysical -> Metaph, met.met, NA
> >> Metaphysical -> Relig, met.relig, NA
> >> Metaphysical -> Death, NA, 1
> >> Work -> Occup, work.occ, NA
> >> Work -> School, work.school, NA
> >> Work -> Job, NA, 1
> >> Affect <-> Affect,Affect.var,NA
> >> Negemo <-> Negemo,Negemo.var,NA
> >> Sad <-> Sad,Sad.var,NA
> >> Physcal <-> Physcal,Physcal.var,NA
> >> Body <-> Body,Body.var,NA
> >> Eating <-> Eating,Eating.var,NA
> >> Groom <-> Groom,Groom.var,NA
> >> numlines <-> numlines,numlines.var,NA
> >> Leisure <-> Leisure,Leisure.var,NA
> >> Home <-> Home,Home.var,NA
> >> Sports <-> Sports,Sports.var,NA
> >> TV <-> TV,TV.var,NA
> >> Music <-> Music,Music.var,NA
> >> Money <-> Money,Money.var,NA
> >> Swear <-> Swear,Swear.var,NA
> >> Sexual <-> Sexual,Sexual.var,NA
> >> Anger <-> Anger,Anger.var,NA
> >> I <-> I,I.var,NA
> >> Metaph <-> Metaph,Metaph.var,NA
> >> Relig <-> Relig,Relig.var,NA
> >> Death <-> Death,Death.var,NA
> >> Occup <-> Occup,Occup.var,NA
> >> School <-> School,School.var,NA
> >> Job <-> Job,Job.var,NA
> >> Melancholy <-> Melancholy, NA, 1
> >> Social <-> Social, NA, 1
> >> Rant <-> Rant, NA, 1
> >> Work <-> Work, NA, 1
> >> Metaphysical <-> Metaphysical, NA, 1
> >>
> >> correl <- matrix(0,nrow=24,ncol=24)
> >> correl[lower.tri(correl,diag=TRUE)] <- c(1, 0.940530496413442,
> >> 0.765560263936915, 0.705665939921134,
> >> 0.659038546655712, 0.282665099938120, 0.234892888297051,
> > 0.0554321360979252,
> >> 0.671137592040541, 0.54382418910777, 0.463922205203901,
0.353418190097785,
> >> 0.414864918025334, 0.436177075274485, 0.401019650838241,
> > 0.370116202378091,
> >> 0.777297151879925, 0.676782523496444, 0.384244495774914,
> > 0.233569710080454,
> >> 0.381213315694389, 0.792870007525815, 0.50437548997674,
0.700522809676332,
> >> 1, 0.789549528191674, 0.706300687178796, 0.677210560231246,
> >> 0.260692661416488, 0.229468929161093, 0.0715008916402315,
> > 0.575592411881254,
> >> 0.495811001574585, 0.387005398515213, 0.286945225104112,
> > 0.332994303548624,
> >> 0.380017238311269, 0.430036391324925, 0.331754269605511,
> > 0.841233992941869,
> >> 0.648407420058087, 0.383101710409617, 0.226088422466275,
0.38890278815819,
> >> 0.694900967907394, 0.445767105915179, 0.611121179516979, 1,
> >> 0.554261042721096, 0.527433237517158, 0.195884095023418,
> > 0.180722683807294,
> >> 0.0233909540224985, 0.516740406669213, 0.435018065577118,
> > 0.357120136043065,
> >> 0.256978759119895, 0.310082859183584, 0.346636599600033,
> > 0.286753730820772,
> >> 0.260480159024979, 0.549021337624957, 0.48888735524692,
0.310472252813801,
> >> 0.162629025914762, 0.341951482059891, 0.604151525748714,
> > 0.404455472396666,
> >> 0.513232045563814, 1, 0.926617967581845, 0.496550925591883,
> >> 0.348467198716147, 0.061642221232731, 0.520273451850585,
> > 0.454688691566968,
> >> 0.361179853804786, 0.251729488407341, 0.276070295108034,
> > 0.301418864648534,
> >> 0.416591776932984, 0.444945330539674, 0.559869838931842,
> > 0.544386905545782,
> >> 0.27126759338533, 0.133247574277541, 0.310271891666118,
0.521537483862256,
> >> 0.311771883417551, 0.471080247067523, 1, 0.296375815811106,
> >> 0.245934996387214, 0.0955552805445385, 0.452337692426552,
> > 0.363463874290988,
> >> 0.339558959480213, 0.244564602900861, 0.253767733666974,
> > 0.271411427718931,
> >> 0.391479465009918, 0.232142608383647, 0.518262010456117,
> > 0.491601506679358,
> >> 0.267644288407907, 0.125392653581526, 0.314025855170014,
> > 0.481718318983948,
> >> 0.28021913467524, 0.425176692670869, 1, 0.183300375385584,
> >> 0.033752849675936, 0.251307861560496, 0.273349357055731,
> > 0.154752778367666,
> >> 0.0726227557418435, 0.0810855443617927, 0.169413646643786,
> > 0.14876182751722,
> >> 0.090027807464796, 0.21707073520028, 0.227245091073637,
> > 0.0648985279708279,
> >> 0.0184155556622148, 0.0917329040389644, 0.224131495587368,
> >> 0.141524409848304, 0.225250463174961, 1, 0.0422205901067624,
> >> 0.335813173017425, 0.462314781501668, 0.137068724813020,
> > 0.0723839601971734,
> >> 0.0727494211937804, 0.112428911506304, 0.09425456138948,
> > 0.0599407261374292,
> >> 0.190826379314435, 0.211704707365072, 0.0599306146160989,
> >> 0.0296829331196419, 0.0682294826412284, 0.188274984783352,
> >> 0.139422045025596, 0.161353066485886, 1, 0.05257676892534,
> >> 0.0280480438748768, 0.00910257508927212, 0.0871857793285287,
> >> 0.0160729045542020, 0.0569498365312614, -0.0278792722216208,
> >> -0.064821779008319, 0.0611534438874241, -0.0649834249949666,
> >> 0.0355509895650055, 0.0135445926170511, 0.0457566918082741,
> >> 0.0604365669560489, 0.0585041719421536, 0.0453745903935696, 1,
> >> 0.761816109289408, 0.589874357357904, 0.649504824987434,
> > 0.658362125608313,
> >> 0.342115375404423, 0.236861363583780, 0.248095510733153,
> > 0.443279264097818,
> >> 0.492471838105872, 0.248701650770021, 0.128552959879838,
> > 0.276153894635528,
> >> 0.607556471316203, 0.449181155519957, 0.510375756616414, 1,
> >> 0.247939328658872, 0.239982254184636, 0.241566313660839,
> > 0.287799155446014,
> >> 0.209294226000604, 0.159562148116037, 0.379361578341484,
> > 0.497577130930334,
> >> 0.218564559998989, 0.09910467906514, 0.260723180575602,
0.467924257159533,
> >> 0.326606718662009, 0.418920180615036, 1, 0.279500783890266,
> >> 0.315054486693380, 0.222378322027149, 0.175009967321649,
> > 0.171392982263514,
> >> 0.303685681481775, 0.277189930927861, 0.112262899972587,
> > 0.0486869353393796,
> >> 0.136799339182926, 0.439080141905746, 0.335750129078362,
> > 0.357071201278175,
> >> 1, 0.354042555500904, 0.186671191628747, 0.0988194936238798,
> >> 0.161389983251376, 0.225124570709853, 0.166070213591245,
> > 0.121050680740651,
> >> 0.0555233084604175, 0.143574719627254, 0.314332705080321,
> > 0.233696321192899,
> >> 0.250453706966593, 1, 0.194173439107269, 0.128637091334456,
> >> 0.188288180271912, 0.249261729284129, 0.294002924006152,
> > 0.180881281305379,
> >> 0.130825345306321, 0.152316871694807, 0.400492580214257,
> > 0.313557675560858,
> >> 0.314387144865789, 1, 0.184856884327958, 0.147800700051061,
> >> 0.314348653487276, 0.223052727856440, 0.160836428547767,
> > 0.0788465159084819,
> >> 0.184166318589758, 0.464779622013576, 0.296714228371417,
> > 0.509475424226691,
> >> 1, 0.243783177939616, 0.433294149172899,
> >> 0.385435708448960,0.165204287365281, 0.122878967887846,
0.134705298271121,
> >> 0.292692353692189,
> >> 0.175825474932413, 0.302111138660379, 1, 0.325702026715680,
> >> 0.265660847381981, 0.144919463251553, 0.105698215649586,
> > 0.120886293524908,
> >> 0.266994210674339, 0.158210355462592, 0.256685716586081, 1,
> >> 0.50128293661388, 0.376612773737787, 0.200216055489841,
0.410973414330935,
> >> 0.544902004754666, 0.340576879662518, 0.495895604908182, 1,
> >> 0.210485715826384, 0.162841078754691, 0.163460255178148,
> > 0.530490890488822,
> >> 0.327408998238198, 0.501112169084101, 1, 0.834421294834807,
> >> 0.697570273879324, 0.284190669063073, 0.165736914780320,
> > 0.243184491397893,
> >> 1, 0.187175885786238, 0.178409162228837, 0.107264487137595,
> >> 0.157214632976541, 1, 0.274591668601347, 0.155954924109372,
> >> 0.229057302047291, 1, 0.824656000910955, 0.82923430112462, 1,
> >> 0.476782357161847, 1)
> >>
> >> ______________________________________________
> >> R-help at r-project.org mailing list
> >> https://stat.ethz.ch/mailman/listinfo/r-help
> >> PLEASE do read the posting guide
> > http://www.R-project.org/posting-guide.html
> >> and provide commented, minimal, self-contained, reproducible code.
> >
> >
> >
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
More information about the R-help
mailing list