[R-sig-ME] Multilevel logistic regression guessing parameter

[Doran, Harold]

First, think of IRT as logistic regression. That is the point (well, one point) that we make in this article connecting lmer with the Rasch model

https://www.jstatsoft.org/article/view/v020i02

It sounds as though you are modeling the probability of a binary response where you want the lower asymptote to be non-zero. So, we refer to this as a 3 parameter logistic model and for each item, you estimate a location parameter, a slope, and the lower asymptote which represents the probability of guessing at low values of the latent trait.

From: Dominik Ćepulić <dcepulic at gmail.com<mailto:dcepulic at gmail.com>>
Date: Friday, May 12, 2017 at 1:32 PM
To: AIR <hdoran at air.org<mailto:hdoran at air.org>>
Cc: Paul Buerkner <paul.buerkner at gmail.com<mailto:paul.buerkner at gmail.com>>, "r-sig-mixed-models at r-project.org<mailto:r-sig-mixed-models at r-project.org>" <r-sig-mixed-models at r-project.org<mailto:r-sig-mixed-models at r-project.org>>
Subject: Re: [R-sig-ME] Multilevel logistic regression guessing parameter

Ok, I was also thinking about an IRT package before, but had problems understanding how I could conceptualize my problem insde IRT framework.
Regarding that issue, this are really basic questions, but I really haven´t found any concrete answer to them:
1) I always got the impression that IRT is used when observed variables are used to predict latent trait measured by them. In my experiment that is not the case - observed reaction times predict observed accuracies. How can I set predictors and criterion variables in IRT framework?

2) As I understood IRT, you always get several parameters for a certain task (depending on which IRT model you are using). That means that for a certain stimuli I would get e.g. 3 parameters. My issue is that I am not interested in single stimuli, because I have over 500 of them - I am interested in a domain certain group of stimuli describe. I.e To me it is not important to know the parameters for a single exemplar but for a whole category which is measured by 64 exemplars. How can I calculate category parameters with an IRT package?

I know this might not be the questions best suited for this forum, but if anyone has some guidelines, I would be very grateful.

Besides that, do you have any suggestions for an IRT package that´s well suited for my problem?

Best,
Dominik

On Fri, May 12, 2017 at 4:57 PM, Doran, Harold <HDoran at air.org<mailto:HDoran at air.org>> wrote:
Dominik

It in fact is well known issue that the lower asymptote of the 3 parameter model you are essentially working with can be different than .5. The empirical results are suggesting that there is some information in the item itself, perhaps in one if the distractors, or in the item stem itself that leads responders to have a probability of response that differs from .5.

You might consider going straight to some IRT packages in R or IRT-specific software instead

-----Original Message-----
From: R-sig-mixed-models [mailto:r-sig-mixed-models-bounces at r-project.org<mailto:r-sig-mixed-models-bounces at r-project.org>] On Behalf Of Dominik Cepulic
Sent: Friday, May 12, 2017 4:55 AM
To: Paul Buerkner <paul.buerkner at gmail.com<mailto:paul.buerkner at gmail.com>>
Cc: r-sig-mixed-models at r-project.org<mailto:r-sig-mixed-models at r-project.org>
Subject: Re: [R-sig-ME] Multilevel logistic regression guessing parameter

Dear everybody, thank you for your ideas and messages!

First, Philipp, yes, you are right. We have a simple two-choice recognition task. Participants were learning some stimuli,  and after some the recognition phase started. Always one stimuli per screen, and they have to say whether it is one of the learnt ones or not. B is therefore coded as
response1 and response2 and afterwards coded in correct/incorrect.
The problem that might have appeared is that some distractors may have been very similar to some well learnt items, and were simultaneously paired with a poorly learnt target. That might produce the effect of correctness below
0.5 We searched for such tasks and deleted them from further analysis.

My problem is that when I try to plot probability functions (x - predictor variable, y - Accuracy from 0 to 1) for domains, they go below 0.5 which doesn´t make sense, as this was a two-choice task. Their lower asymptote should be on 0.5 not on 0. That´s why I am asking.

@Paul: Thanks for recommendation, but what do you mean by "Stan under the hood"? I basically need a typical multilevel logistic regression (with random effects for 2 crossed levels) but with lower asymptote being 0.5 and not 0.

I will take a look at the functions!

Best,
Dominik

On Fri, May 12, 2017 at 9:36 AM, Paul Buerkner <paul.buerkner at gmail.com<mailto:paul.buerkner at gmail.com>>
wrote:

> Hi Dominik,
>
> in addition to what Jake said, you can do this with the brms package
> (using Stan under the hood). After installing brms, you can learn how
> to fit such models in the "brms_nonlinear" vignette: Type
> vignette("brms_nonlinear") in R.
>
> Best,
> Paul
>
> 2017-05-11 13:00 GMT+02:00 Dominik Ćepulić <dcepulic at gmail.com<mailto:dcepulic at gmail.com>>:
>
>> I  have a following situation:
>>
>> I want to predict variable B (which is dichotomous) from variable A
>> (continous) controlling for random effects on the level of a)
>> Subjects; b) Tasks.
>>
>> A -> B (1)
>>
>> The problem is that when I use model to predict the values of B from
>> A, values below probability of 0.5 get predicted, and in my case that
>> doesn´t make sense, because, if you guess at random, the probability
>> of correct answer on B would be 0.5.
>>
>> I want to know how I can constrain the model (1) in lme4 so that it
>> doesn´t predict values lower than 0.5 in variable B.
>>
>> Thank you,
>>
>> Dominik!
>>
>>         [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> R-sig-mixed-models at r-project.org<mailto:R-sig-mixed-models at r-project.org> mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>
>
>

        [[alternative HTML version deleted]]

_______________________________________________
R-sig-mixed-models at r-project.org<mailto:R-sig-mixed-models at r-project.org> mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models


	[[alternative HTML version deleted]]