[R-sig-ME] Analysis of signal detection data

[Ken Knoblauch]

David Duffy <davidD at ...> writes:
> On Sun, 14 Nov 2010, Thompson,Paul wrote:
> >> This situation is well described and most appropriately analyzed by the 
> >> use of GEE methods.
> > Mike Lawrence wrote:
> > Yet another query on whether traditional stats employed in psychology
> > might be improved by mixed effects modelling...
> > [diagnostic accuracy v. gold standard]
> As I understand it, GEE can be seen as an approximate method related to 
> approaches such as quasi-likelihood, though there may be cases where its 
> underlying model is closer to the true state of what you are actually 
> looking at.  So, a GLMM may better -- I'm more familiar with SEM/ 
> graphical models for this type of setup, in that they are more flexible
> eg diagnosis where there is no gold standard diagnosis to compare to, 
> mixture models for task difficulty... 
> These will usually have a "full likelihood" GLMM underlying them, or some 
> type of approximation eg Browne's WLS seen in Lisrel/MX.
> 
> Just 2c, David Duffy.
The classic signal detection theory (SDT) model, 
in which an observer's choice behavior is modeled by an 
unobserved decision variable contaminated by noise, is
essentially a probit model (see Green & Swets and 
Macmillan &  Creelman references below).  A logistic 
version called choice theory was developed by Luce.  
A clear formalization in terms of GLMs is given in the 
DeCarlo reference below (but see also the Klein reference 
for a general extension to psychometric function fitting
and the 2 papers by Wichmann & Hill in the same special
issue) with extensions to rating scales using ordinal
regression.

In R, these models can be simply fit with a glm, taking
the observer's response as a 2-level variable (factor or
logical) and the stimulus (signal present/absent) as a 
2-level factor.  The model would be 

g(E(Pr(Resp = 1))) = beta_0 + beta_1 Stim,

where g is the link function, taken as an inverse
Gaussian for the probit case.  When, the stimulus is
absent, the linear predictor gives beta_0 which
equals Phi^-1(Pr(False Alarm)) = z(FA) (to use the
more conventional SDT terminology).  Treatment contrasts
then directly give you z(Hit) - z(FA) = d'.
For example,

Stim <- rep(factor(c(0, 1), labels = c("A", "P")), 
	each = 500)
##unbiased observer w/ true d' = 1.0	
Resp <- ((Stim == "P") + rnorm(length(Stim))) > 0

glm(Resp ~ Stim, binomial(probit))
Call:  glm(formula = Resp ~ Stim, family = binomial(probit))

Coefficients:
(Intercept)        StimP  
   -0.05015      1.03642  

Degrees of Freedom: 999 Total (i.e. Null);  998 Residual
Null Deviance:	    1283 
Residual Deviance: 1135 	AIC: 1139

Compare this with the manual method of calculating d' 
given in SDT texts.

( dp.tab <- prop.table(table(Stim, Resp), margin = 1) )

    Resp
Stim FALSE  TRUE
   A 0.520 0.480
   P 0.162 0.838

diff(qnorm(dp.tab[, 2]))

       P 
1.036425 


The extension to more complex GLM's with more complex
explanatory variables as well as to miwed-effects models
with glmer then follows quite naturally.
(You can't extend this to m-alternative forced-choice
in a simple way, i.e., you cannot simply substitute
one of the links from the psyphy package in the glm.
The situation is more complicated than that.)
With glmer you can add observer and item random effects.
If you had several observers in the above toy example,
you could include terms (1 | Obs) for random criterion
models and (Stim | Obs) to model both random criterion 
and random sensitivity (using SDT jargon, here).

The sdtalt package provides a simplified interface
(wrapper function) for specifying response, stimulus, 
observer and item variables although the last time that I
looked at the code, the Resp and Stim (or equivalent) 
variables were reversed in the call to glmer.

See also, the MLDS and MLCM packages for extensions of
an SDT model to choices involving suprathreshold stimuli


@Book{GreenSwets66,
  author = 	 {D. M. Green  and J. A. Swets},
  title = 	 {Signal Detection Theory and Psychophysics},
  publisher = 	 {Robert E. Krieger Publishing Company},
  year = 	 {1966},
  address = 	 {Huntington},
}

@Book{MacCreel,
  author = 	 {N.~A.~Macmillan and C.~D.~Creelman},
  title = 	 {Detection Theory:  A User's Guide},
  publisher = 	 {Lawrence Erlbaum Associates},
  year = 	 {2005},
  address = 	 {New York},
  edition = 	 {Second},
}

@Article{DeCarlo98,
  author = 	 {L. T. De{C}arlo},
  title = 	 {Signal detection theory and generalized linear models},
  journal = 	 {Psych Meth},
  year = 	 {1998},
  volume = 	 {3},
  pages = 	 {186--205},
}

@Article{Klein01,
   Author="Klein, S. A. ",
   Title="{{M}easuring, estimating, and understanding 
the psychometric function: a commentary}",
   Journal="Perception \& Psychophysics",
   Year="2001",
   Volume="63",
   Pages="1421--1455",
   Month="Nov"
}