[R-sig-ME] lme capable of running with missing data?
Kenneth Frost
kfrost at wisc.edu
Sat Feb 4 02:45:14 CET 2012
On 02/03/12, Charles Determan Jr wrote:
> Kevin,
>
> I understand that but then how is SAS accomplishing the interactions?
I have been following this conversation a little bit and this seems to be the right question to ask. I would also like to know the answer. However, this could be the wrong venue to get an answer to this question.
> On Fri, Feb 3, 2012 at 10:58 AM, Kevin Wright <kw.stat at gmail.com> wrote:
>
> > Charles,
> >
> > Here's a simple thought example. Use a piece of graph paper (or just a
> > simple sketch). Write the following letters at the coordinates specified:
> >
> > A1 (1,1)
> > A2 (3,2)
> > B1 (1,2)
> > B2 (missing)
> >
> > Draw a line from A1 to A2. Imagine a line from B1 to the missing value of
> > B2.
> >
> > By looking at this, you could calculate an overall mean for the A factor.
> > You could also estimate an overall mean for the B factor, if you assume the
> > lines are parallel. This is what happens with fixed effects, as in lme (
> > ... A + B, ...).
> > But, when you specify lme(... A*B, ...) which is the same as lme(... A + B
> > + A:B, ...), you are essentially saying to the computer, "The A1-A2 and
> > B1-B2 lines are not parallel, but please give me an estimate of the slope
> > of the B1-B2 line."
> >
> > Could _you_ draw the B1-B2 line? No. Neither can lme.
> >
> > It's okay that B2 is missing if you don't want to fit an interaction, but
> > when B2 is missing, there is no way to estimate an interaction
> > (non-parallel slope).
> >
> > Kevin
> >
> >
> >
> >
> >
> >
> > On Fri, Feb 3, 2012 at 10:18 AM, Charles Determan Jr <deter088 at umn.edu>wrote:
> >
> >> After the data is input, and factors are assigned,
> >>
> >> model=lme(arginine~group*time*survival, random=~1|subj, method="REML",
> >> data=x)
> >>
> >> Error in MEEM(object, conLin, control$niterEM) : Singularity in backsolve
> >> at level 0, block 1
> >>
> >>
> >>
> >> On Fri, Feb 3, 2012 at 10:01 AM, Kevin Wright <kw.stat at gmail.com> wrote:
> >>
> >>> Providing the data is not a "reproducible example". Complete data and
> >>> R-code are helpful.
> >>>
> >>> Kevin
> >>>
> >>>
> >>>
> >>> On Fri, Feb 3, 2012 at 9:31 AM, Charles Determan Jr <deter088 at umn.edu>wrote:
> >>>
> >>>> Here is the dataset, everything should be run as a factor except 'met'
> >>>> which is numeric. Thanks for the assistance,
> >>>>
> >>>> time group survival subj met
> >>>> 1 1 2 1 2 1.3954
> >>>> 2 2 2 1 2 1.8063
> >>>> 3 3 2 1 2 1.3684
> >>>> 4 4 2 1 2 2.0046
> >>>> 5 5 2 1 2 1.0334
> >>>> 6 6 2 1 2 0.3644
> >>>> 7 7 2 1 2 0.4819
> >>>> 8 8 2 1 2 1.4558
> >>>> 9 9 2 1 2 0.9718
> >>>> 10 1 1 2 5 0.7771
> >>>> 11 2 1 2 5 1.2439
> >>>> 12 1 2 2 8 1.0980
> >>>> 13 2 2 2 8 0.9511
> >>>> 14 1 2 1 9 1.0534
> >>>> 15 2 2 1 9 1.7279
> >>>> 16 3 2 1 9 1.4904
> >>>> 17 4 2 1 9 1.2737
> >>>> 18 5 2 1 9 0.8929
> >>>> 19 6 2 1 9 0.5828
> >>>> 20 7 2 1 9 0.3260
> >>>> 21 8 2 1 9 1.0373
> >>>> 22 9 2 1 9 0.9624
> >>>> 23 1 2 2 10 1.1391
> >>>> 24 2 2 2 10 1.3945
> >>>> 25 3 2 2 10 0.9414
> >>>> 26 4 2 2 10 1.1152
> >>>> 27 5 2 2 10 0.8222
> >>>> 28 6 2 2 10 0.4417
> >>>> 29 7 2 2 10 0.4126
> >>>> 30 1 1 1 12 1.3024
> >>>> 31 2 1 1 12 1.1811
> >>>> 32 3 1 1 12 0.9379
> >>>> 33 4 1 1 12 1.3000
> >>>> 34 5 1 1 12 1.2977
> >>>> 35 6 1 1 12 0.4949
> >>>> 36 7 1 1 12 0.5238
> >>>> 37 8 1 1 12 1.3862
> >>>> 38 1 1 1 16 1.2259
> >>>> 39 2 1 1 16 0.8681
> >>>> 40 3 1 1 16 1.2645
> >>>> 41 4 1 1 16 0.7316
> >>>> 42 5 1 1 16 0.6648
> >>>> 43 6 1 1 16 0.9671
> >>>> 44 7 1 1 16 1.0131
> >>>> 45 8 1 1 16 1.1762
> >>>> 46 9 1 1 16 0.8776
> >>>> 47 1 2 2 18 1.1231
> >>>> 48 2 2 2 18 1.2133
> >>>> 49 3 2 2 18 1.2005
> >>>> 50 4 2 2 18 0.7198
> >>>> 51 5 2 2 18 0.6620
> >>>> 52 6 2 2 18 0.5908
> >>>> 53 7 2 2 18 0.3945
> >>>> 54 1 2 2 19 0.7852
> >>>> 55 2 2 2 19 0.6758
> >>>> 56 3 2 2 19 0.5246
> >>>> 57 4 2 2 19 0.5263
> >>>> 58 1 2 2 20 1.2284
> >>>> 59 2 2 2 20 0.7017
> >>>> 60 1 2 1 23 0.9604
> >>>> 61 2 2 1 23 0.7977
> >>>> 62 3 2 1 23 1.2267
> >>>> 63 4 2 1 23 1.3857
> >>>> 64 5 2 1 23 0.9486
> >>>> 65 6 2 1 23 0.3571
> >>>> 66 7 2 1 23 0.3134
> >>>> 67 8 2 1 23 1.9984
> >>>> 68 9 2 1 23 0.4837
> >>>> 69 1 1 1 24 1.1793
> >>>> 70 2 1 1 24 1.3883
> >>>> 71 3 1 1 24 2.1080
> >>>> 72 4 1 1 24 0.8810
> >>>> 73 5 1 1 24 0.8825
> >>>> 74 6 1 1 24 0.4124
> >>>> 75 7 1 1 24 0.5270
> >>>> 76 8 1 1 24 1.9003
> >>>> 77 9 1 1 24 1.4344
> >>>> 78 1 1 1 27 1.1905
> >>>> 79 2 1 1 27 1.1033
> >>>> 80 3 1 1 27 1.4976
> >>>> 81 4 1 1 27 1.9018
> >>>> 82 5 1 1 27 0.5815
> >>>> 83 6 1 1 27 0.4428
> >>>> 84 7 1 1 27 0.4728
> >>>> 85 8 1 1 27 1.6309
> >>>> 86 9 1 1 27 0.4054
> >>>> 87 1 1 1 28 0.9538
> >>>> 88 2 1 1 28 0.7796
> >>>> 89 3 1 1 28 1.7906
> >>>> 90 5 1 1 28 0.4715
> >>>> 91 6 1 1 28 0.4214
> >>>> 92 7 1 1 28 0.4120
> >>>> 93 8 1 1 28 1.3111
> >>>> 94 9 1 1 28 0.3677
> >>>> 95 1 1 2 1 1.3853
> >>>> 96 2 1 2 1 1.5966
> >>>> 97 3 1 2 1 1.4542
> >>>> 98 4 1 2 1 1.3084
> >>>> 99 5 1 2 1 1.2826
> >>>> 100 6 1 2 1 0.6835
> >>>> 101 7 1 2 1 0.9709
> >>>> 102 1 1 1 3 1.3175
> >>>> 103 2 1 1 3 0.7792
> >>>> 104 3 1 1 3 1.8763
> >>>> 105 5 1 1 3 1.4633
> >>>> 106 6 1 1 3 0.0735
> >>>> 107 7 1 1 3 0.5612
> >>>> 108 8 1 1 3 1.3777
> >>>> 109 9 1 1 3 0.3810
> >>>> 110 1 1 2 4 1.3486
> >>>> 111 1 1 1 6 1.2635
> >>>> 112 2 1 1 6 0.7572
> >>>> 113 3 1 1 6 1.5011
> >>>> 114 5 1 1 6 0.6873
> >>>> 115 6 1 1 6 0.3778
> >>>> 116 7 1 1 6 0.4231
> >>>> 117 8 1 1 6 1.3817
> >>>> 118 9 1 1 6 0.5850
> >>>> 119 1 2 2 7 0.7362
> >>>> 120 2 2 2 7 0.5495
> >>>> 121 3 2 2 7 0.7621
> >>>> 122 4 2 2 7 0.8421
> >>>> 123 5 2 2 7 1.0438
> >>>> 124 6 2 2 7 0.9802
> >>>> 125 7 2 2 7 0.5627
> >>>> 126 1 1 1 11 1.5575
> >>>> 127 2 1 1 11 2.1356
> >>>> 128 3 1 1 11 1.3575
> >>>> 129 4 1 1 11 1.3056
> >>>> 130 5 1 1 11 0.8144
> >>>> 131 6 1 1 11 0.5876
> >>>> 132 7 1 1 11 0.4104
> >>>> 133 9 1 1 11 0.4942
> >>>> 134 1 2 1 13 1.0046
> >>>> 135 2 2 1 13 0.8805
> >>>> 136 3 2 1 13 0.7685
> >>>> 137 4 2 1 13 0.8786
> >>>> 138 5 2 1 13 1.4249
> >>>> 139 6 2 1 13 0.5339
> >>>> 140 7 2 1 13 0.5480
> >>>> 141 8 2 1 13 2.6369
> >>>> 142 9 2 1 13 1.7159
> >>>> 143 1 2 1 14 0.7161
> >>>> 144 2 2 1 14 0.3968
> >>>> 145 3 2 1 14 0.8142
> >>>> 146 4 2 1 14 0.6140
> >>>> 147 5 2 1 14 0.6585
> >>>> 148 6 2 1 14 0.7176
> >>>> 149 7 2 1 14 0.6613
> >>>> 150 8 2 1 14 1.6494
> >>>> 151 9 2 1 14 0.3903
> >>>> 152 1 1 1 15 1.4357
> >>>> 153 2 1 1 15 1.4772
> >>>> 154 3 1 1 15 1.3156
> >>>> 155 4 1 1 15 0.9654
> >>>> 156 5 1 1 15 1.2709
> >>>> 157 6 1 1 15 0.9330
> >>>> 158 7 1 1 15 0.3515
> >>>> 159 8 1 1 15 1.6801
> >>>> 160 9 1 1 15 0.3584
> >>>> 161 1 2 2 17 0.8077
> >>>> 162 2 2 2 17 0.7560
> >>>> 163 1 1 1 21 1.1890
> >>>> 164 2 1 1 21 0.9631
> >>>> 165 3 1 1 21 0.9753
> >>>> 166 4 1 1 21 0.9519
> >>>> 167 5 1 1 21 0.6348
> >>>> 168 6 1 1 21 0.8516
> >>>> 169 7 1 1 21 0.2366
> >>>> 170 8 1 1 21 1.0440
> >>>> 171 9 1 1 21 0.5360
> >>>> 172 1 2 1 22 1.0747
> >>>> 173 2 2 1 22 0.6451
> >>>> 174 3 2 1 22 0.8408
> >>>> 175 5 2 1 22 0.8730
> >>>> 176 6 2 1 22 0.3594
> >>>> 177 7 2 1 22 0.3019
> >>>> 178 9 2 1 22 1.2053
> >>>> 179 1 2 2 25 0.4654
> >>>> 180 2 2 2 25 0.3024
> >>>> 181 3 2 2 25 0.7525
> >>>> 182 4 2 2 25 0.7808
> >>>> 183 5 2 2 25 0.6294
> >>>> 184 6 2 2 25 0.3016
> >>>> 185 7 2 2 25 0.3223
> >>>> 186 1 2 1 26 0.5363
> >>>> 187 2 2 1 26 0.2279
> >>>> 188 3 2 1 26 0.4756
> >>>> 189 4 2 1 26 0.6644
> >>>> 190 5 2 1 26 0.6631
> >>>> 191 6 2 1 26 0.3419
> >>>> 192 7 2 1 26 0.4188
> >>>> 193 8 2 1 26 0.3199
> >>>> 194 9 2 1 26 0.2889
> >>>> 195 1 1 2 29 1.2765
> >>>> 196 2 1 2 29 1.0653
> >>>> 197 3 1 2 29 1.5607
> >>>> 198 1 1 1 30 0.8641
> >>>> 199 2 1 1 30 0.9250
> >>>> 200 3 1 1 30 1.0887
> >>>> 201 4 1 1 30 0.5537
> >>>> 202 5 1 1 30 0.7930
> >>>> 203 6 1 1 30 0.3960
> >>>> 204 7 1 1 30 0.3917
> >>>> 205 8 1 1 30 1.2687
> >>>> 206 9 1 1 30 0.5328
> >>>> 207 1 2 1 31 1.0765
> >>>> 208 2 2 1 31 0.8778
> >>>> 209 3 2 1 31 0.8228
> >>>> 210 4 2 1 31 1.2017
> >>>> 211 5 2 1 31 1.1787
> >>>> 212 6 2 1 31 0.4037
> >>>> 213 7 2 1 31 0.2625
> >>>> 214 8 2 1 31 2.2690
> >>>> 215 9 2 1 31 0.4423
> >>>> 216 1 1 2 32 1.2880
> >>>> 217 2 1 2 32 0.8537
> >>>>
> >>>> On Fri, Feb 3, 2012 at 9:25 AM, Baldwin, Jim -FS <jbaldwin at fs.fed.us>
> >>>> wrote:
> >>>>
> >>>> > I think the only way to resolve this is to provide a specific example.
> >>>> >
> >>>> > Jim Baldwin
> >>>> > Station Statistician
> >>>> > USDA Forest Service
> >>>> > Albany, California
> >>>> >
> >>>> > -----Original Message-----
> >>>> > From: r-sig-mixed-models-bounces at r-project.org [mailto:
> >>>> > r-sig-mixed-models-bounces at r-project.org] On Behalf Of Charles
> >>>> Determan Jr
> >>>> > Sent: Friday, February 03, 2012 7:18 AM
> >>>> > To: Thompson,Paul; r-sig-mixed-models at r-project.org
> >>>> > Subject: Re: [R-sig-ME] lme capable of running with missing data?
> >>>> >
> >>>> > So, is there a way in which I can alter the design matrix so the mixed
> >>>> > model will work or is this something that can only be done in SAS
> >>>> > currently? The output from the SAS run did provide Type III fixed
> >>>> effect
> >>>> > test values.
> >>>> >
> >>>> > On Fri, Feb 3, 2012 at 9:14 AM, Thompson,Paul <
> >>>> > Paul.Thompson at sanfordhealth.org> wrote:
> >>>> >
> >>>> > > That's interesting. SAS uses the sweep approach (it was in fact
> >>>> > > devised by Goodnight). The method used in construction of various
> >>>> > > types of SS does allow you to estimate when cells are missing. I
> >>>> would
> >>>> > > wonder if Type II SS can be done. Type III (despite the incorrect
> >>>> > > statement that they are
> >>>> > > illegitimate) and Type IV would work fine. ****
> >>>> > >
> >>>> > > ** **
> >>>> > >
> >>>> > > It's really an issue of the manner in which the design matrix is
> >>>> > > contructed.****
> >>>> > >
> >>>> > > ** **
> >>>> > >
> >>>> > > *From:* Charles Determan Jr [mailto:deter088 at umn.edu](javascript:main.compose()
> >>>> > > *Sent:* Friday, February 03, 2012 8:36 AM
> >>>> > > *To:* Thompson,Paul; r-sig-mixed-models at r-project.org
> >>>> > > *Subject:* Re: [R-sig-ME] lme capable of running with missing
> >>>> > > data?****
> >>>> > >
> >>>> > > ** **
> >>>> > >
> >>>> > > Thank you Paul, I do appreciate your response and especially your
> >>>> time.
> >>>> > > The reason I am so persistent is that I know the prior data I posted
> >>>> > > was run in SAS (however I don't have the exact coding although I do
> >>>> > > know it was done with PROC MIXED with an unstructured covariance
> >>>> > > structure and REML estimation method) and it provided all the
> >>>> > > interactions. As such, I have scoured the web and literature as to
> >>>> > > how this could be done with the missing data (timepoints as a result
> >>>> > > of survival). Perhaps this simply has not yet been done in R and I
> >>>> am
> >>>> > > stuck for the time being. None-the-less, I want to be certain
> >>>> before I
> >>>> > give up on running this type of analysis in R.
> >>>> > >
> >>>> > > Thanks again,****
> >>>> > >
> >>>> > > On Fri, Feb 3, 2012 at 8:26 AM, Thompson,Paul <
> >>>> > > Paul.Thompson at sanfordhealth.org> wrote:****
> >>>> > >
> >>>> > > Charles:
> >>>> > >
> >>>> > > I did suggest the use of specific contrasts to do the analysis with
> >>>> > > missing cells. I played around, and just have to admit that this is
> >>>> > > not possible. I tried to use standard construction techniques to
> >>>> > > produce main effects using contrast coding, and then multiply those
> >>>> to
> >>>> > > produce interactions. This does not work. It may be possible to use
> >>>> > > orthonormalization and the sweep operator to produce a consistent
> >>>> > > estimator, but I ran out of time to work on this.
> >>>> > >
> >>>> > > What you can do is convert the design to a single factor, and do the
> >>>> > > analysis with specific contrasts, recognizing that this will not
> >>>> > > enable you to get to specific things like interaction effects. To
> >>>> > > understand why, consider the situation with a 2 x 2, where one cell
> >>>> is
> >>>> > entirely missing.
> >>>> > > You have lost 1 df for the design, and the interaction is entirely
> >>>> > missing.
> >>>> > > You can estimate and test specific contrasts, but you can't even
> >>>> > > really test the A factor or the B factor. If Cell(2,2) is missing,
> >>>> you
> >>>> > > can test Cell (1,1) v Cell(1,2) and you can test Cell (2,1) v Cell
> >>>> > > (1,1), but neither of these is the test of the "main effect" of A or
> >>>> > > B. When you have larger designs with 2 or 3 factors, the comparisons
> >>>> > > again have fewer df than should be encountered. This means that the
> >>>> > > interactions are not defined properly.
> >>>> > >
> >>>> > > Do you NEED the interactions for theoretical purposes, or are they
> >>>> > > there simply for completedness? Are the cells missing due to your
> >>>> > > design or due to happenstance?
> >>>> > >
> >>>> > > It is the case that fractional factorial designs eliminate cells
> >>>> from
> >>>> > > the design to estimate main effects and losing the ability to
> >>>> estimate
> >>>> > > interactions. So, missing cells, when planned for appropriately, can
> >>>> > > result in appropriate analysis. I am not sure how to run mixed
> >>>> models
> >>>> > > with fractional factorials, however.****
> >>>> > >
> >>>> > >
> >>>> > > -----Original Message-----
> >>>> > > From: r-sig-mixed-models-bounces at r-project.org [mailto:
> >>>> > > r-sig-mixed-models-bounces at r-project.org] On Behalf Of Charles
> >>>> > > Determan Jr
> >>>> > > Sent: Friday, February 03, 2012 7:06 AM
> >>>> > > To: r-sig-mixed-models at r-project.org
> >>>> > > Subject: [R-sig-ME] lme capable of running with missing data?
> >>>> > >
> >>>> > > Greetings,
> >>>> > >
> >>>> > > Some of you may recognize my name from a few related posts but I
> >>>> just
> >>>> > > have general question that perhaps can be clarified. I have read
> >>>> > > several times that 'lme' and 'lmer' are techniques capable of
> >>>> running
> >>>> > > data sets with missing values. Is this true? I have put up similar
> >>>> > > posts where when I try to run a two or three way interaction mixed
> >>>> > > model I get an error of singularities or X'X not positive. Does the
> >>>> > > data set need to be formatted in some way where the mixed model can
> >>>> be
> >>>> > run with all interactions?
> >>>> > > Furthermore, if the missing values are 'not missing at random' is
> >>>> > > there another method to follow for generating the mixed model? I am
> >>>> > > just confused why I see posts that lme can be run when data is
> >>>> missing.
> >>>> > >
> >>>> > > Regards,****
> >>>> > >
> >>>> > > [[alternative HTML version deleted]]
> >>>> > >
> >>>> > > _______________________________________________
> >>>> > > R-sig-mixed-models at r-project.org mailing list
> >>>> > > https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
> >>>> > >
> >>>> > >
> >>>> ----------------------------------------------------------------------
> >>>> > > - Confidentiality Notice: This e-mail message, including any
> >>>> > > attachments, is for the sole use of the intended recipient(s) and
> >>>> may
> >>>> > > contain privileged and confidential information. Any unauthorized
> >>>> > > review, use, disclosure or distribution is prohibited. If you are
> >>>> not
> >>>> > > the intended recipient, please contact the sender by reply e-mail
> >>>> and
> >>>> > > destroy all copies of the original message.****
> >>>> > >
> >>>> > > ** **
> >>>> > >
> >>>> > >
> >>>> > >
> >>>> ----------------------------------------------------------------------
> >>>> > > - Confidentiality Notice: This e-mail message, including any
> >>>> > > attachments, is for the sole use of the intended recipient(s) and
> >>>> may
> >>>> > > contain privileged and confidential information. Any unauthorized
> >>>> > > review, use, disclosure or distribution is prohibited. If you are
> >>>> not
> >>>> > > the intended recipient, please contact the sender by reply e-mail
> >>>> and
> >>>> > > destroy all copies of the original message.
> >>>> > >
> >>>> >
> >>>> > [[alternative HTML version deleted]]
> >>>> >
> >>>> >
> >>>> >
> >>>>
> >>>> [[alternative HTML version deleted]]
> >>>>
> >>>> _______________________________________________
> >>>> R-sig-mixed-models at r-project.org mailing list
> >>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> Kevin Wright
> >>>
> >>>
> >>
> >
> >
> > --
> > Kevin Wright
> >
> >
>
> [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-mixed-models at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
More information about the R-sig-mixed-models
mailing list