[R-sig-ME] lme: nested and not-nested random factors

[maya mayrose]

Dear R users,

I have a question regarding the lme function. I am not sure how to model 2 random factors, one random factor is nested with the fixed factor but the second one is not.

Specifically, I have collected RNA expression data for nine populations that are nested within three plant groups (that is, each population belongs to only one group; the populations were sampled randomly from a large pool of populations in each group). Each population was sampled once.

I then replicated the experiment 3 times, so there may be some differences between the replicates that are shared across all populations.

My aim is to look for differences in expression between the three groups, accounting for population as a random factor and for the variability between the three replicates.

 
The formula I thought of is:

z <- lme(y ~ group, random = list(population=~ 1,replicate=~1))

but this would treat both replicate as nested within population.

 
Another option would be to treat replicate as an additional fixed factor, as there were three independent replicates (but on the other hand, the plants for each replicate were randomly chosen).

I would greatly appreciate your help!

Maya