[R-sig-ME] Analysis of signal detection data

[John Sorkin]

Would logistic regression with a dichotomous variable indicating pre- or post-training provide an adequate model?
John 
John Sorkin
Chief Biostatistics and Informatics
Univ. of Maryland School of Medicine
Division of Gerontology and Geriatric Medicine
JSorkin at grecc.umaryland.edu 
-----Original Message-----
From: Mike Lawrence <Mike.Lawrence at dal.ca>
To:  <r-sig-mixed-models at r-project.org>

Sent: 11/14/2010 10:58:03 PM
Subject: [R-sig-ME] Analysis of signal detection data

Hi folks,

Yet another query on whether traditional stats employed in psychology
might be improved by mixed effects modelling...

Consider a radiologist looking at a CT scan and attempting to make the
binary diagnosis of cancer/no cancer. Signal detection theory suggests
that the normalized difference between the radiologist's hit rate and
false alarm rate provides a metric of the radiologist's discrimination
skill (d'). That is:

d' = qnorm(hit_rate) - qnorm(FA_rate)

Now, if we wanted to see if discrimination skill was improved by some
intervention, we might recruit a bunch of radiologists and measure
their d' both before and after the intervention. That is, both before
the intervention, each radiologist would be presented with a number of
"trials" where they review CT scans, mark them as cancer/no cancer,
and we experimentalists score each diagnosis as a hit, miss, false
alarm, or correct rejection.

Presented with data like this, most psychologists would compute a d'
score for each radiologist both before and after the intervention,
then submit the d' scores to a repeated-measures ANOVA, which assumes
gaussian error. However, hit and false alarm rates should yield
binomially distributed error distributions, and monte carlo
experimentation in R leads me to believe that in cases where only a
moderate number of CT scans are reviewed per session (say, 10-20), d'
may be expected to be considerably non-gaussian.

I know mixed effects modelling can handle binomially distributed
error, but is there any way to handle this sort of signal detection
data? My first thought is that glmmer with 4 categories corresponding
to the hit, miss, false alarm, and correction categorization of
responses, but I don't immediately see how this would properly connect
the hit-vs-miss data to reflect a hit rate and the
false-alarm-vs-correct-rejection data to reflect a FA rate.

Thoughts?

Mike

--
Mike Lawrence
Graduate Student
Department of Psychology
Dalhousie University

Looking to arrange a meeting? Check my public calendar:
http://tr.im/mikes_public_calendar

~ Certainty is folly... I think. ~

_______________________________________________
R-sig-mixed-models at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models

Confidentiality Statement:
This email message, including any attachments, is for th...{{dropped:6}}