[R-sig-ME] lmer on dataset with missing values 'not at random'
Thompson,Paul
Paul.Thompson at SanfordHealth.org
Mon Jan 30 20:38:59 CET 2012
Another approach to the problem of interactions with missing cells would be to design specific contrasts which assess interactions in those cases, but which exclude the missing cells. To do that, you would need to define the entire experiment as a single factor situation, and write the appropriate contrasts.
-----Original Message-----
From: r-sig-mixed-models-bounces at r-project.org [mailto:r-sig-mixed-models-bounces at r-project.org] On Behalf Of Kevin Wright
Sent: Monday, January 30, 2012 1:31 PM
To: Charles Determan Jr
Cc: r-sig-mixed-models at r-project.org
Subject: Re: [R-sig-ME] lmer on dataset with missing values 'not at random'
Charles,
You are trying to fit fixed effects for two-way and three-way interactions
when, as you have shown, there are no data for certain combinations of
those factors. Generally, to estimate fixed-effect interactions you need
data in every cell of the interaction xtabs table.
You can start by removing the interactions from your model. If the
interactions are of interest, you could try fitting the interactions as
random effects, but you need to understand this leads to a different model
with different interpretations and inferences...
Kevin
On Mon, Jan 30, 2012 at 12:44 PM, Charles Determan Jr <deter088 at umn.edu>wrote:
> Greetings R users,
>
> I have been trying to fit a mixed model on the following dataset. All
> columns are factors except 'met'. When I try to run a full model with
> lmer, I get the error 'Error in mer_finalize(ans) : Downdated X'X is not
> positive definite, 27.' I took a quick look at the dataset and I can see
> there are 4 '0's in the survival 2 group. This makes sense as not all
> experiments made it to the end, therefore if they didn't finish then they
> didn't have any further timepoints. I have read about missing data at
> random but I can't find a way to run lmer on a dataset that has values
> missing 'not at random'. Is there a way to modify the lmer statement
> without cutting out data points?
>
> My sincere thanks,
>
> x
> time group survival subj met
> 1 1 2 1 2 1.3954
> 2 2 2 1 2 1.8063
> 3 3 2 1 2 1.3684
> 4 4 2 1 2 2.0046
> 5 5 2 1 2 1.0334
> 6 6 2 1 2 0.3644
> 7 7 2 1 2 0.4819
> 8 8 2 1 2 1.4558
> 9 9 2 1 2 0.9718
> 10 1 1 2 5 0.7771
> 11 2 1 2 5 1.2439
> 12 1 2 2 8 1.0980
> 13 2 2 2 8 0.9511
> 14 1 2 1 9 1.0534
> 15 2 2 1 9 1.7279
> 16 3 2 1 9 1.4904
> 17 4 2 1 9 1.2737
> 18 5 2 1 9 0.8929
> 19 6 2 1 9 0.5828
> 20 7 2 1 9 0.3260
> 21 8 2 1 9 1.0373
> 22 9 2 1 9 0.9624
> 23 1 2 2 10 1.1391
> 24 2 2 2 10 1.3945
> 25 3 2 2 10 0.9414
> 26 4 2 2 10 1.1152
> 27 5 2 2 10 0.8222
> 28 6 2 2 10 0.4417
> 29 7 2 2 10 0.4126
> 30 1 1 1 12 1.3024
> 31 2 1 1 12 1.1811
> 32 3 1 1 12 0.9379
> 33 4 1 1 12 1.3000
> 34 5 1 1 12 1.2977
> 35 6 1 1 12 0.4949
> 36 7 1 1 12 0.5238
> 37 8 1 1 12 1.3862
> 38 1 1 1 16 1.2259
> 39 2 1 1 16 0.8681
> 40 3 1 1 16 1.2645
> 41 4 1 1 16 0.7316
> 42 5 1 1 16 0.6648
> 43 6 1 1 16 0.9671
> 44 7 1 1 16 1.0131
> 45 8 1 1 16 1.1762
> 46 9 1 1 16 0.8776
> 47 1 2 2 18 1.1231
> 48 2 2 2 18 1.2133
> 49 3 2 2 18 1.2005
> 50 4 2 2 18 0.7198
> 51 5 2 2 18 0.6620
> 52 6 2 2 18 0.5908
> 53 7 2 2 18 0.3945
> 54 1 2 2 19 0.7852
> 55 2 2 2 19 0.6758
> 56 3 2 2 19 0.5246
> 57 4 2 2 19 0.5263
> 58 1 2 2 20 1.2284
> 59 2 2 2 20 0.7017
> 60 1 2 1 23 0.9604
> 61 2 2 1 23 0.7977
> 62 3 2 1 23 1.2267
> 63 4 2 1 23 1.3857
> 64 5 2 1 23 0.9486
> 65 6 2 1 23 0.3571
> 66 7 2 1 23 0.3134
> 67 8 2 1 23 1.9984
> 68 9 2 1 23 0.4837
> 69 1 1 1 24 1.1793
> 70 2 1 1 24 1.3883
> 71 3 1 1 24 2.1080
> 72 4 1 1 24 0.8810
> 73 5 1 1 24 0.8825
> 74 6 1 1 24 0.4124
> 75 7 1 1 24 0.5270
> 76 8 1 1 24 1.9003
> 77 9 1 1 24 1.4344
> 78 1 1 1 27 1.1905
> 79 2 1 1 27 1.1033
> 80 3 1 1 27 1.4976
> 81 4 1 1 27 1.9018
> 82 5 1 1 27 0.5815
> 83 6 1 1 27 0.4428
> 84 7 1 1 27 0.4728
> 85 8 1 1 27 1.6309
> 86 9 1 1 27 0.4054
> 87 1 1 1 28 0.9538
> 88 2 1 1 28 0.7796
> 89 3 1 1 28 1.7906
> 90 5 1 1 28 0.4715
> 91 6 1 1 28 0.4214
> 92 7 1 1 28 0.4120
> 93 8 1 1 28 1.3111
> 94 9 1 1 28 0.3677
> 95 1 1 2 1 1.3853
> 96 2 1 2 1 1.5966
> 97 3 1 2 1 1.4542
> 98 4 1 2 1 1.3084
> 99 5 1 2 1 1.2826
> 100 6 1 2 1 0.6835
> 101 7 1 2 1 0.9709
> 102 1 1 1 3 1.3175
> 103 2 1 1 3 0.7792
> 104 3 1 1 3 1.8763
> 105 5 1 1 3 1.4633
> 106 6 1 1 3 0.0735
> 107 7 1 1 3 0.5612
> 108 8 1 1 3 1.3777
> 109 9 1 1 3 0.3810
> 110 1 1 2 4 1.3486
> 111 1 1 1 6 1.2635
> 112 2 1 1 6 0.7572
> 113 3 1 1 6 1.5011
> 114 5 1 1 6 0.6873
> 115 6 1 1 6 0.3778
> 116 7 1 1 6 0.4231
> 117 8 1 1 6 1.3817
> 118 9 1 1 6 0.5850
> 119 1 2 2 7 0.7362
> 120 2 2 2 7 0.5495
> 121 3 2 2 7 0.7621
> 122 4 2 2 7 0.8421
> 123 5 2 2 7 1.0438
> 124 6 2 2 7 0.9802
> 125 7 2 2 7 0.5627
> 126 1 1 1 11 1.5575
> 127 2 1 1 11 2.1356
> 128 3 1 1 11 1.3575
> 129 4 1 1 11 1.3056
> 130 5 1 1 11 0.8144
> 131 6 1 1 11 0.5876
> 132 7 1 1 11 0.4104
> 133 9 1 1 11 0.4942
> 134 1 2 1 13 1.0046
> 135 2 2 1 13 0.8805
> 136 3 2 1 13 0.7685
> 137 4 2 1 13 0.8786
> 138 5 2 1 13 1.4249
> 139 6 2 1 13 0.5339
> 140 7 2 1 13 0.5480
> 141 8 2 1 13 2.6369
> 142 9 2 1 13 1.7159
> 143 1 2 1 14 0.7161
> 144 2 2 1 14 0.3968
> 145 3 2 1 14 0.8142
> 146 4 2 1 14 0.6140
> 147 5 2 1 14 0.6585
> 148 6 2 1 14 0.7176
> 149 7 2 1 14 0.6613
> 150 8 2 1 14 1.6494
> 151 9 2 1 14 0.3903
> 152 1 1 1 15 1.4357
> 153 2 1 1 15 1.4772
> 154 3 1 1 15 1.3156
> 155 4 1 1 15 0.9654
> 156 5 1 1 15 1.2709
> 157 6 1 1 15 0.9330
> 158 7 1 1 15 0.3515
> 159 8 1 1 15 1.6801
> 160 9 1 1 15 0.3584
> 161 1 2 2 17 0.8077
> 162 2 2 2 17 0.7560
> 163 1 1 1 21 1.1890
> 164 2 1 1 21 0.9631
> 165 3 1 1 21 0.9753
> 166 4 1 1 21 0.9519
> 167 5 1 1 21 0.6348
> 168 6 1 1 21 0.8516
> 169 7 1 1 21 0.2366
> 170 8 1 1 21 1.0440
> 171 9 1 1 21 0.5360
> 172 1 2 1 22 1.0747
> 173 2 2 1 22 0.6451
> 174 3 2 1 22 0.8408
> 175 5 2 1 22 0.8730
> 176 6 2 1 22 0.3594
> 177 7 2 1 22 0.3019
> 178 9 2 1 22 1.2053
> 179 1 2 2 25 0.4654
> 180 2 2 2 25 0.3024
> 181 3 2 2 25 0.7525
> 182 4 2 2 25 0.7808
> 183 5 2 2 25 0.6294
> 184 6 2 2 25 0.3016
> 185 7 2 2 25 0.3223
> 186 1 2 1 26 0.5363
> 187 2 2 1 26 0.2279
> 188 3 2 1 26 0.4756
> 189 4 2 1 26 0.6644
> 190 5 2 1 26 0.6631
> 191 6 2 1 26 0.3419
> 192 7 2 1 26 0.4188
> 193 8 2 1 26 0.3199
> 194 9 2 1 26 0.2889
> 195 1 1 2 29 1.2765
> 196 2 1 2 29 1.0653
> 197 3 1 2 29 1.5607
> 198 1 1 1 30 0.8641
> 199 2 1 1 30 0.9250
> 200 3 1 1 30 1.0887
> 201 4 1 1 30 0.5537
> 202 5 1 1 30 0.7930
> 203 6 1 1 30 0.3960
> 204 7 1 1 30 0.3917
> 205 8 1 1 30 1.2687
> 206 9 1 1 30 0.5328
> 207 1 2 1 31 1.0765
> 208 2 2 1 31 0.8778
> 209 3 2 1 31 0.8228
> 210 4 2 1 31 1.2017
> 211 5 2 1 31 1.1787
> 212 6 2 1 31 0.4037
> 213 7 2 1 31 0.2625
> 214 8 2 1 31 2.2690
> 215 9 2 1 31 0.4423
> 216 1 1 2 32 1.2880
> 217 2 1 2 32 0.8537
>
> ds=lmer(met~group*time*survival + (1|subj), data=x)
> Error in mer_finalize(ans) : Downdated X'X is not positive definite, 27.
>
> xtabs(~group+time+survival, x)
> , , survival = 1
>
> time
> group 1 2 3 4 5 6 7 8 9
> 1 11 11 11 8 11 11 11 10 10
> 2 8 8 8 7 8 8 8 7 8
>
> , , survival = 2
>
> time
> group 1 2 3 4 5 6 7 8 9
> 1 5 4 2 1 1 1 1 0 0
> 2 8 8 5 5 4 4 4 0 0
>
> [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-mixed-models at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>
--
Kevin Wright
[[alternative HTML version deleted]]
_______________________________________________
R-sig-mixed-models at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
-----------------------------------------------------------------------
Confidentiality Notice: This e-mail message, including any attachments,
is for the sole use of the intended recipient(s) and may contain
privileged and confidential information. Any unauthorized review, use,
disclosure or distribution is prohibited. If you are not the intended
recipient, please contact the sender by reply e-mail and destroy
all copies of the original message.
More information about the R-sig-mixed-models
mailing list