Dear John,
Your explanation makes sense to me in that between-batch variability is too small compared to within batch variabiltity. Thanks a lot for your explanation!
John
On Friday, March 4, 2022, 06:00:29 PM PST, John Fox <jfox using mcmaster.ca> wrote:
Dear John,
On 2022-03-04 8:08 p.m., array chip via R-sig-mixed-models wrote:
> Dear all, I have this simple dataset to measure the yeild of a crop collected in 2 batches (https://drive.google.com/file/d/1lgVZVLHeecp9a_sFxEPeg6353O-qXZhM/view). when I ran a simple inear mixed model using lmer to estimate within-batch and between-batch variability, the between-batch variability is 0. The run showed that data is singular. Does anyone know why the data is singular and what's the reason for 0 variability? is it because the dataset only has 2 batches?
I'd say yes and no.
Yes, in that you have effectively approximately only 1 degree of freedom
for between-group variability, very little data in each group, and small
variability between groups relative to within-group
variability--apparently sufficiently small that the REML criterion is
maximized at the boundary of the parameter space:
> # not quite right because group n's differ:
> sd(with(daty, tapply(yield, batch, mean)))
[1] 0.3032702
> with(daty, tapply(yield, batch, sd))
D K
2.952165 2.779894
Perhaps someone else can provide a more precise explanation.
No, in that it's not hard to construct data for which a proper solution
can be obtained; e.g.,
> set.seed(123)
> Data <- data.frame(yield=c(rnorm(100, -5, 1), rnorm(100, 5, 1)),
+ batch=rep(c("D", "K"), each=100))
> lmer(yield ~ 1 + (1|batch), Data)
Linear mixed model fit by REML ['lmerMod']
Formula: yield ~ 1 + (1 | batch)
Data: Data
REML criterion at convergence: 554.1332
Random effects:
Groups Name Std.Dev.
batch (Intercept) 6.9305
Residual 0.9403
Number of obs: 200, groups: batch, 2
Fixed Effects:
(Intercept)
-0.00857
> sd(with(Data, tapply(yield, batch, mean))) # estimates 5
[1] 6.931094
> with(Data, tapply(yield, batch, sd)) # estimate 1
D K
0.9128159 0.9669866
BTW, your data *were* included with the original message that you sent
to r-help, because .txt-file attachments are permitted, but, as you can
see, your input and output are garbled here because your email wasn't
plain-text:
>> daty<-read.table("datx.txt",sep='\t',header=T,row.names=NULL)
>> library(lme4)> lmer(yield~1+(1|batch),daty)boundary (singular) fit: see ?isSingularLinear mixed model fit by REML ['lmerMod']Formula: yield ~ 1 + (1 | batch) Data: datyREML criterion at convergence: 115.6358Random effects: Groups Name Std.Dev. batch (Intercept) 0.000 Residual 2.789 Number of obs: 24, groups: batch, 2Fixed Effects:(Intercept) 5.788
> Thanks!John
I hope this helps,
John
>
> [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-mixed-models using r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
--
John Fox, Professor Emeritus
McMaster University
Hamilton, Ontario, Canada
web: https://socialsciences.mcmaster.ca/jfox/
[[alternative HTML version deleted]]