[R-sig-ME] How to deal with outcomes assessed by raters?

Chris Howden chris at trickysolutions.com.au
Tue Apr 16 08:17:08 CEST 2013


I'm no expert, but I believe that with only 4 judges that's not enough to
get an accurate estimate of the variability associated with judges.

So U may be better to include them as fixed effects with 4 levels.

That said if U just want a random intercept for each judge and don't want
an accurate measure of their variance it may still be OK to include them
as a random effect? But I'm a little unclear on this point myself.

Chris Howden B.Sc. (Hons) GStat.
Founding Partner
Evidence Based Strategic Development, IP Commercialisation and Innovation,
Data Analysis, Modelling and Training
(mobile) 0410 689 945
(fax) +612 4782 9023
chris at trickysolutions.com.au




Disclaimer: The information in this email and any attachments to it are
confidential and may contain legally privileged information. If you are
not the named or intended recipient, please delete this communication and
contact us immediately. Please note you are not authorised to copy, use or
disclose this communication or any attachments without our consent.
Although this email has been checked by anti-virus software, there is a
risk that email messages may be corrupted or infected by viruses or other
interferences. No responsibility is accepted for such interference. Unless
expressly stated, the views of the writer are not those of the company.
Tricky Solutions always does our best to provide accurate forecasts and
analyses based on the data supplied, however it is possible that some
important predictors were not included in the data sent to us. Information
provided by us should not be solely relied upon when making decisions and
clients should use their own judgement.


-----Original Message-----
From: r-sig-mixed-models-bounces at r-project.org
[mailto:r-sig-mixed-models-bounces at r-project.org] On Behalf Of Joseph
Bulbulia
Sent: Tuesday, 16 April 2013 3:07 PM
To: r-sig-mixed-models at r-project.org
Subject: [R-sig-ME] How to deal with outcomes assessed by raters?

Hi all,

Id like to model emotional dynamics in a highly arousing firewalk ritual.


Four judges rated images from 42 participants for arousal and valence. The
predictor variables are ritual phase and role.

Question 1
Any thoughts about how best to handle the rater assessments?

Specifically, is it nuts to explicitly include a component for raters in
the random component of the model?

E.g.

library(MCMCglmm)
prior.fw.0 = list(
  B = list(mu=rep(0,4),V = diag(4)*1e+10),
  R = list(V =diag(2), fix = 1),
  G = list(G1 = list(V = diag(2), n = 2, alpha.mu = c(0,0), alpha.V =
diag(2)*1000),
           G2 = list (V = diag(2),n = 2, alpha.mu = c(0,0), alpha.V =
diag(2)*1000),
           G3 = list (V = diag(2),  fix=1)))


firemodel.test <-MCMCglmm(cbind(arousal, valence) ~ trait:role:trait:phase
-1,
                          random = ~us(trait):phase:id
                          + idh(trait):event:id
                          + idh(trait):rater,
                          rcov= ~ idh(trait):units,
                          family = rep("ordinal",2),
                          data=Firewalkdata, burnin=5000,
                          thin = 10,
                          nitt=20000,
                          prior=prior.fw.0)



Thanks everyone. Very grateful for and advice.

Joseph


Disclaimer
I'm new to GLMMs,so apologies if this doesn't make sense.

Data sample below
(Only 20 data points, just to get a sense of the structure)

Firewalkdata <- structure(list(obs = c("1", "2", "3", "4", "5", "6", "7",
"8", "9", "10", "11", "12", "13", "14", "15", "16", "17", "18", "19",
"20"), id = structure(c(24L, 4L, 26L, 37L, 32L, 3L, 20L, 9L, 3L, 2L, 5L,
19L, 23L, 28L, 29L, 8L, 3L, 18L, 40L, 26L), .Label = c("a", "b", "c", "d",
"e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "pa", "pb",
"pc", "pd", "pe", "pf", "pg", "ph", "pi", "pj", "pk", "pl", "pm", "pn",
"po", "pp", "q", "r", "s", "t", "u", "v", "w", "x", "y"), class =
"factor"), phase = c(1, 2, 5, 5, 2, 3, 1, 4, 5, 2, 5, 1, 5, 4, 3, 4, 5, 3,
2, 4), event = structure(c(11L, 4L, 13L, 21L, 22L, 3L, 4L, 9L, 3L, 2L, 5L,
3L, 9L, 16L, 17L, 8L, 3L, 2L, 24L, 13L), .Label = c("a", "b", "c", "d",
"e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", "s",
"t", "u", "v", "w", "x", "y"), class = "factor"), role = structure(c(2L,
1L, 2L, 1L, 2L, 1L, 2L, 1L, 1L, 1L, 1L, 2L, 2L, 2L, 2L, 1L, 1L, 2L, 1L,
2L), .Label = c("FW", "PS"), class = "factor"), dyad = structure(c(2L, 2L,
2L, 1L, 2L, 2L, 2L, 2L, 2L, 2L, 1L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 1L, 2L),
.Label = c("n", "y"), class = "factor"), gender = structure(c(1L, 2L, 2L,
2L, 2L, 2L, 1L, 2L, 2L, 2L, 2L, 1L, 1L, 1L, 1L, 2L, 2L, 1L, 2L, 2L),
.Label = c("f", "m"), class = "factor"), rater = structure(c(4L, 1L, 3L,
1L, 3L, 4L, 1L, 2L, 4L, 3L, 4L, 3L, 4L, 1L, 1L, 4L, 4L, 1L, 4L, 4L),
.Label = c("rat1", "rat2", "rat3", "rat4"), class = "factor"),
    arousal = c(4, 5, NA, 6, 6, 6, 4, 4, 7, 4, 3, 7, 5, 5, 5,
    6, 5, NA, 4, 6), valence = c(4, 3, NA, 2, 6, 1, 2, 6, 3,
    5, 5, 7, 5, 2, 2, 1, 4, NA, 4, 6)), .Names = c("obs", "id", "phase",
"event", "role", "dyad", "gender", "rater", "arousal", "valence"),
row.names = c(3977L, 83L, 2996L, 525L, 3134L, 3213L, 726L, 1267L, 3221L,
2134L, 3273L, 2801L, 3975L, 944L, 964L, 3344L, 3223L, 688L, 3758L, 4041L),
class = "data.frame")


IMAGE SAMPLE (for the curious)
https://www.dropbox.com/s/xmbci5814h73i0l/4_MF_e5.png

GRAPH bootstrapped means
https://www.dropbox.com/s/509sgh5zqxq18tn/Figure_FireWalk.pdf

Crude overview of the design
https://www.dropbox.com/s/0o0a9kkrh5ttsd2/plot.plan.emotions_firewalk.pdf




Joseph Bulbulia
Senior Lecturer, Religious Studies
Faculty of Humanities and Social Sciences Victoria University, New Zealand
+64 21 95 94 23
http://www.metaphysicalclub.com




	[[alternative HTML version deleted]]



More information about the R-sig-mixed-models mailing list