Hi Aline –

In the code you have below, the problem arises because there are 125 rows in the new combination grid but it’s hard-coded later as if there are 27 rows.  Just replace “27” with “nrow(P)”.  (avoid hard-coding!)

Now, as for 2-way balance … it’s going to be difficult to get good 2-way balance *within* each respondent unless they have a lot of trials. I’d look instead for 2-way balance *across* respondents. Using sampling as above, that should happen automatically if you have more than a few respondents.

Here’s some code that: (a) fixes the “nrow” issue, (b) adds proportional sampling (in case you want fewer than 125 combinations per respondent), and (c) checks the 2-way frequencies across respondents.

# generate all combinations of the 3 factors
f1<-1:5; f2<-1:5; f3<-1:5
P<-expand.grid(f1,f2,f3)

# built respondents
N <- 200
resp<-NULL; for(i in 1:N) { resp<-rbind(resp,P) }

# randomize theur orders :
prop <- 0.1
samp<-NULL; for(i in 1:N) { samp<-c(samp,sample(nrow(P),nrow(P)*prop)) }
#
offset<-rep(0:(N-1),each=nrow(P)*prop)
samp.ind<-offset*nrow(P)+samp

final<-resp[samp.ind,]
PP<-cbind(offset+1,final)
colnames(PP)<-c("ID","X1","X2","X3")

table(PP$X1,PP$X2)
table(PP$X1,PP$X3)
table(PP$X2,PP$X3)

# by(PP$X1,PP$ID,table)
# by(PP$X2,PP$ID,table)
# by(PP$X3,PP$ID,table)

If you want to optimize the 2-way frequencies beyond what random sampling gives, then the way to do that would be to wrap the generation above in a loop and check some kind of exit criterion. For instance, you might check a chisq.test() for the 2-way (just wrap “chisq.test()” around the call to table()), keep the best one found thus far, and iterate a certain number of times.  With N=10, getting good balance will require a lot of trials and possibly design iteration; but with N=100 (say), it shouldn’t be so hard.

I hope that helps,

-- Chris

From: Aline Deschamps [mailto:deschamps.aline@yahoo.fr]
Sent: Thursday, September 01, 2011 2:08 AM
To: Chris Chapman
Cc: r-sig-dcm@r-project.org
Subject: Re : Re : [R-sig-DCM] Re : Balanced design of experiments for CBC

Hi Chris,

I'm really confused, I've totally mixed up things yesterday and the code I gave you was wrong.
This is what I tried to do :

# generate all combinations of the 3 factors
f1<-1:5; f2<-1:5; f3<-1:5
P<-expand.grid(f1,f2,f3)

# built respondents (N=10)
resp<-NULL;  for(i in 1:10) { resp<-rbind(resp,P) }

# randomize theur orders :
samp<-NULL; for(i in 1:10) { samp<-c(samp,sample(1:27)) }
#
offset<-rep(0:9,each=27)
samp.ind<-offset*27+samp

final<-resp[samp.ind,]


PP<-cbind(offset+1,final)

colnames(PP)<-c("ID","X1","X2","X3")


by(PP$X1,PP$ID,table)
by(PP$X2,PP$ID,table)
by(PP$X3,PP$ID,table)

And this is an example of what I obtain :

by(PP$X1,PP$ID,table)

PP$ID: 1

1 2 3 4 5
6 6 5 5 5
------------------------------------------------------------
PP$ID: 2

1 2 3 4 5
5 5 6 6 5
------------------------------------------------------------
# it's what we want

by(PP$X3,PP$ID,table)

PP$ID: 1

 1  2
25  2
------------------------------------------------------------
PP$ID: 2

 2  3
23  4
------------------------------------------------------------
PP$ID: 3

 3  4
21  6

# it's not what I want


So, you could really see in this example that the design matrix is not well balanced because, for instance, the first respondent sees only the levels 1 and 2 for the third factor.

What I want for my design matrix is first, that each respondent sees all the levels of the factors a same number of times (approximately), BUT I also want that they see all of the possible pairs of levels of 2 different factors the same number of times, and I don't know how to program it.

I hope you will understand what I'm trying to do and again, I'm really sorry for my yesterday's mess.


Best regards,

Aline





________________________________
De : Chris Chapman <Chris.Chapman@microsoft.com>
À : Aline Deschamps <deschamps.aline@yahoo.fr>
Cc : "r-sig-dcm@r-project.org" <r-sig-dcm@r-project.org>
Envoyé le : Mer 31 août 2011, 17h 17min 04s
Objet : RE: Re : [R-sig-DCM] Re : Balanced design of experiments for CBC
Hi Aline –

You’re welcome.  I’m not sure what’s happening; your code seems to be correct.  However, I notice that it’s not completely self-contained (e.g., the column names change along the way to things such as “ID”) so I suspect there may be an issue elsewhere.

When I run the following, I get occurrence of 9 (i.e., 1/3 of 27 trials) for every level:

f1<-1:3; f2<-1:3; f3<-1:3
P<-expand.grid(f1,f2,f3)

# built respondents (N=10)
resp<-NULL; for(i in 1:10) { resp<-rbind(resp,P) }

# randomize theur orders :
samp<-NULL; for(i in 1:10) { samp<-c(samp,sample(1:27)) }
#
offset<-rep(0:9,each=27)
samp.ind<-offset*27+samp
final<-resp[samp.ind,]

# compile final data matrix
PP<-cbind(offset+1,final)
names(PP) <- c("ID","X1","X2","X3")   # I added this – otherwise it’s the same as your code

# check factorial occurrence
by(PP$X1,PP$ID,table)
by(PP$X2,PP$ID,table)
by(PP$X3,PP$ID,table)

Output excerpt:

> by(PP$X3,PP$ID,table)
PP$ID: 1

1 2 3
9 9 9
-------------------------------------------------------------------------------------------
PP$ID: 2

1 2 3
9 9 9

Is that what you expected?  Or are you looking for a way to sample and have fewer than 27 trials but preserve overall balance?  (in which case, you could sample fewer than 27 rows in the “sample(1:27)” call).

I hope that helps – happy to help more if there’s something I don’t understand …  best,

-- Chris

From: Aline Deschamps [mailto:deschamps.aline@yahoo.fr]
Sent: Wednesday, August 31, 2011 7:06 AM
To: Chris Chapman
Cc: r-sig-dcm@r-project.org
Subject: Re : [R-sig-DCM] Re : Balanced design of experiments for CBC

Hi Chris,

Thank you for your reply and forgive me for having been slow to respond, I work on several project and I haven't had the time to test your code before this last days.

I was glad to discover the "expand.grid" function (I used to use the "gen.factorial" function in the "AlgDesign" package), but I -- unfortunately -- think that your code doesn't fit exactly to what I'm trying to do.
In fact, if i try your code for, for instance, 3 factors with 5 levels each one, and 27 combinations for each respondent (the maximal number of combinations is 5*5*5=125), I don't obtain a balanced design :

# generate all combinations of the 3 factors
f1<-1:3; f2<-1:3; f3<-1:3
P<-expand.grid(f1,f2,f3)

# built respondents (N=10)
resp<-NULL;  for(i in 1:10) { resp<-rbind(resp,P) }

# randomize theur orders :
samp<-NULL; for(i in 1:10) { samp<-c(samp,sample(1:27)) }
#
offset<-rep(0:9,each=27)
samp.ind<-offset*27+samp

final<-resp[samp.ind,]

PP<-cbind(offset+1,final)


Then, if I try :

by(PP$X3,PP$ID,table)

I obtain for instance :

PP$ID: 9

 1  2  3  4  5
 0  0  0  9 18
------------------------------------------------------------
PP$ID: 10

 1  2  3  4  5
20  0  0  0  7


May be I haven't well understood what you were trying to explain to me, or may be I wasn't clear when I exposed my problem...
What I want is a design matrix, and for each respondant, I want that all the combinations of the levels of the factors appear a same (approximately) number of time. For instance, if I have 3 factors (X, Y and Z) with 2 levels for the first, 3 for the second, and 4 for the last, I want that each respondent sees the same number of times the combination (X=1 ; Y=1) (for instance in the line : 1  1  3 of the design matrix), the same number of times the combination (X=1 ; Y=2), ... , the same number of time the combination (X=1 ; Z=4), etc... and all of this for each pairs of factors. I'm not sure it's really clear, I think it's already hard to explain it in french so I hope you will understand me a little bit...

Thank you for all your help !


Best regards,

Aline Deschamps
Nantes, France


________________________________
De : Chris Chapman <Chris.Chapman@microsoft.com>
À : Aline Deschamps <deschamps.aline@yahoo.fr>; Bryan Orme <bryan@sawtoothsoftware.com>
Cc : "r-sig-dcm@r-project.org" <r-sig-dcm@r-project.org>
Envoyé le : Lun 25 juillet 2011, 16h 49min 31s
Objet : RE: [R-sig-DCM] Re : Balanced design of experiments for CBC

Hi Aline --

If I understand the question properly (you want to ensure that every respondent sees all the combinations), then I'd suggest to do this:

1. generate all the combinations ("expand.grid")
2. create a design matrix of those repeated N times ("for ... rbind")
3. randomize the order of each block within respondent ("for ... sample")
4. then just add those columns to whatever design you have or need for the other factors

Here's some R code to do that part (warning, not debugged :-)

# generate all pairs of two factors
f1 <- 1:3; f2 <- 1:5
f1f2 <- expand.grid(f1,f2)

# build respondents
resp <- NULL; for (i in 1:10) { resp <- rbind(resp,f1f2) }    # now have N=10 respondent designs, but all in the same order

# randomize their orders: first pull 10 samples of 1:15 to get individual ordering
samp <- NULL; for (i in 1:10) { samp <- c(samp,sample(1:15)) }
# and then offset those into the rows of the full matrix
offset <- rep(0:9, each=15)
samp.ind <- offset*15 + samp;
final <- resp[samp,]    # now randomized within respondent

As Bryan notes, if you have too many combinations per person and just need 9 or 12 or whatever, then you could put that number into the "sample()"  function inline above to either over- or under-sample accordingly (but in a balanced way).  Thanks to how expand.grid and sample work, there will be no repetitions (of the 2 factors above) unless you oversample them.

[And yes, there are ways to do this besides using "for" loops, but they have the virtue of being easy to read :-)]

Is that what you had in mind?  Best,

-- Chris


-----Original Message-----
From: r-sig-dcm-bounces@r-project.org<mailto:r-sig-dcm-bounces@r-project.org> [mailto:r-sig-dcm-bounces@r-project.org<mailto:r-sig-dcm-bounces@r-project.org>] On Behalf Of Aline Deschamps
Sent: Monday, July 25, 2011 7:33 AM
To: Bryan Orme
Cc: r-sig-dcm@r-project.org<mailto:r-sig-dcm@r-project.org>
Subject: [R-sig-DCM] Re : Balanced design of experiments for CBC

Thank you for your answer.
I realise that my example was poorly chosen, what I'm looking for is a way to program a balanced design of experiments where each respondent sees approximately the same number of times each combination of levels of two variables. I don't know if I'm really clear, but someone showed me a software ("Design Expert") which seems to do it well, unfortunately I haven't any clue of how, except that it's based on the Fedorov algorithm apparently.

So, if anyone has a clue, I'm still looking for :)

Aline Deschamps
Nantes, France





________________________________
De : Bryan Orme <bryan@sawtoothsoftware.com<mailto:bryan@sawtoothsoftware.com>>

Envoyé le : Lun 25 juillet 2011, 16h 27min 46s Objet : Re: [R-sig-DCM] Balanced design of experiments for CBC

Mathematically not possible.  For each respondent, you have 3 tasks x 4 products = 12 products.  Then, your two 3-level factors crossed together make 9 possible combinations of those two factors.  So, it's impossible to evenly fill that 2-way counts matrix of 9 cells using 12 product concepts. So, perfect 2-way level balance within each respondent questionnaire is impossible.

--Bryan Orme, Sawtooth Software

--------------------------------------------------

Sent: Monday, July 25, 2011 3:12 AM
To: <r-sig-dcm@r-project.org<mailto:r-sig-dcm@r-project.org>>
Subject: [R-sig-DCM] Balanced design of experiments for CBC

> Hi all,
>
> I'm working on CBC (Choice-Based Conjoint) studies and my problem
> today (in fact, it's a problem I have since so many days ;) is how to
> generate a
balanced
> design of experiments for that kind of study.
>
> Here is the background of my study (based on an example) :
>
> I have 3 variables X, Y and Z. Each could take respectively 2, 3 and 3 levels.
>I
> have 500 respondents.
> All of them see 3 tasks of 4 products (I remind you that a product is
>a  combination of levels of each variable).
> Then, I want to program the generation of the design of experiments
>for such a  case.
>
>
> The design of experiments should verify two conditions :
>
> 1/ Each task should not have two identical products, and each
> respondent
should
> not see two identical tasks (and all respondents don't see the same tasks).
>
> 2/ Each respondent should see the same number of times each possible
>combination  of levels of two variables.
>
> My major issue is how to have this second condition.
>
> In fact, I've seen that the Fedorov algorithm is often used in order
>to generate  design of experiments for CBC but with the "optFederov"
>function (of the  "AlgDesign" package), I haven't be able to obtain
>such a balanced design of  experiments...
> So, I thought to program it myself (in R or may be in an other
>language, I don't  know yet) but I'm afraid it would be complicated and
>mostly not very effective  to check all the possible combinations of
>levels of two variables.
>
>
> I hope my message is not too confused, and if you have any clue of how
> to program the generation of design of experiments in such a case, I
> would be
very
> grateful.
>
> Thanks :)
>
>
> Aline Deschamps
> Nantes, France
>
> [[alternative HTML version deleted]]
>
> _______________________________________________
> R-SIG-DCM mailing list
> R-SIG-DCM@r-project.org<mailto:R-SIG-DCM@r-project.org>
> https://stat.ethz.ch/mailman/listinfo/r-sig-dcm
    [[alternative HTML version deleted]]

	[[alternative HTML version deleted]]

