[R-sig-ME] How to deal with outcomes assessed by raters?

[David Duffy]

On Tue, 16 Apr 2013, Joseph Bulbulia wrote:

> Hi all,
>
> I?d like to model emotional dynamics in a highly arousing firewalk ritual.
>
> Four judges rated images from 42 participants for arousal and valence. The predictor variables are ritual ?phase? and ?role.?
>
> Question 1
> Any thoughts about how best to handle the rater assessments?
>
> Specifically, is it nuts to explicitly include a component for raters in the random component of the model?

So what do the inter-rater agreements look like?  I presume rater is 
actually a nuisance variable?  The path model I would usually use would 
have phase and role acting on the averaged-over-raters a and v 
scores (measurement model bit).

Just 2c, David Duffy.