[R-sig-ME] Selection of the appropriate random structure in a partially non counterbalanced experimental design

[Paolo Canal]

We designed a study manipulating two factors: one factor (FACTOR1) has
2 levels (A,B), the second factor (FACTOR2) has three levels (C,D,E) .
Because in psychology we often have to carry out unreasonably
time-consuming experiments we planned to split the experiment (3by2)
in two experiments (2by2) in which participants were exposed to all
levels of FACTOR1 and only two levels of FACTOR2. One level (C - the
logical reference level) of FACTOR2 was presented to all participants
while the half of participants exposed to D were not exposed to E (and
viceversa). We had 41 participants and 60 different items. I thought
we could use mixed models to exploit the better precision in
estimating C and by-item-fully crossed design, still keeping decent
power to capture differences between D and E, with no need to add a
third (by-subjects) factor referring to which Experiment participants
were assigned to.

A dummy dataset would look like the following:

sj  item  FACTOR1  FACTOR2  y
1   1     A        C        123
1   2     B        C        145
1   3     A        D        110
1   4     B        D        NA
2   1     A        C        159
2   2     B        C        189
2   3     A        E        165
2   4     B        E        123

Since the specification of the correct random structure is crucial to
gain conservative results I fitted the maximal structure and tested
the interaction:

y~FACTOR1*FACTOR2+(1+FACTOR1*FACTOR2|sj)+(1+FACTOR1*FACTOR2|it)

the model does not converge and possibly because it is trying to
compute the adjustments for those levels of FACTOR2 the participants
were not exposed to (participants that saw only C and E were adjusted
for D as well).

What is our best choice?

use the full specification in the by-item random component only and
"relax" the by-subject component?

y~FACTOR1*FACTOR2+(1+FACTOR1|sj)+(1+FACTOR1*FACTOR2|it)