[R-sig-ME] fixed effect testing again (but different)

[Daniel Ezra Johnson]

Thanks to all of your help, I have a simulation method running to
assess fixed effect significance. I have a more conceptual question
today.

Again, the fixed effects I'm usually interested in testing are
"between-subject" or "outer" to a subject random effect. I'll continue
using the example of gender since that's a simple and clear one.

Up until now my procedure is to evaluate two models, viz.

m0 <- lmer(response~(1|subject),data)
m1 <- lmer(response~gender+(1|subject),data)

The likelihood-ratio test approach directly compares these two.
The simulation approach generates data from the parameters of m0, and
compares the resulting test statistic to its value from m1.

However, the more I think about it, I think this might not be the best
way to go.
Basically the way I see the question is, "is the difference between
men and women in the data greater than what might occur by chance?"

In order to answer that, we need some estimate of how much subjects
vary within each gender group, or perhaps we could say how much they
vary, apart from gender variation. We have two obvious estimates of
this quantity: the subject random effect variance parameter of m0 and
of m1.

So far all my simulation attempts have used the parameter from m0 as
the basis for the simulation.
I'm starting to think that doesn't make sense.

In models of the form of m0, the subject random effect is large, and
it "substitutes" for the fixed effect. I mean that by accurately (and
fully, given large and balanced datasets) estimating the behavior of
individual subjects, it necessarily accounts for the between-subject
effect at the same time. (In particular, the random effects are not
constrained to a normal distribution in practice, even though they are
supposed to be normally distributed in theory.)

I'm beginning to wonder if it wouldn't make sense to do simulations
(and maybe LRT's) using, as the null model, a model with no fixed
effects but with the subject random effect parameter taken from m1,
not m0. The alternative model would still be the full m1.

Otherwise it seems as though the fixed effect (if there is one)
basically makes its way into the subject variance parameter of the
null model, increasing it, and leading eventually to an overly
conservative test.

If it seems wrong to take the parameter from m1, is there some way to
change the specification of m0 so as to obtain a separate random
effect for males and females? I've seen that done but I've forgotten
how to do it.

If I can run something like

m00 <- lmer(response~(1|subject-females)+(1|subject-males),data)   #
how is this specified?

then simulated data could be generated from the parameters of that
model (not the random-effect estimates, only the overall variance
parameters), and then that data could be fit to

m11 <- lmer(response~gender+(1|subject-females)+(1|subject-males),data)

to evaluate the fixed effect... A nice side effect is if there is
heteroscedasticity between males and females, that's built right in.

Daniel