[R-sig-ME] contradictory odds ratios--a problem with the equation or the interpretation?

[John Maindonald]

It looks to me that there is a strong interaction between language and Association,
or there may be effects that are more complicated than that.  If so, that would seem
to me an interesting outcome to probe further.  Have you tried fitting a language by
Association interaction?

On what you call the “uneven levels” issue, I do not know what the basis may be
for thinking that "mixed models were flexible and robust enough to deal with
this kind of thing.”  Much depends, of course, on how the “unevenness” feeds into
the weights given to factor or factor combinations.  But if there is an effect
consistently over the different levels of the relevant random effect or effects,
it will feature in the mixed models results.  Note Simpson’s paradox type effects,
or the same kind of effect for regression coefficients, sometimes called Laird’s
paradox.  Hopefully, the experiment has been executed in a way that these are
not issues.


John Maindonald             email: john.maindonald using anu.edu.au<mailto:john.maindonald using anu.edu.au>


On 11/05/2021, at 13:41, Johnathan Jones <johnathan.jones using gmail.com<mailto:johnathan.jones using gmail.com>> wrote:

Good suggestions and notes here, thanks all for taking the time. It amounted to a very well-rounded accounting.
Mitchell, yes, this makes sense.
John, thanks for the questions. Yes, these are from a designed experiment. Results were all as expected (from descriptives to inferentials and the Rasch analysis) until this mixed methods output was produced. Of note, I’m surprised that uneven levels matter for mixed models. It was my understanding mixed models were flexible and robust enough to deal with this kind of thing. I have omitted other explanatory factors which “broke” R and wouldn’t converge (vowel duration was address in item random effects; word familiarity was explained by association), but I think the most important predictors remain—at least the ones which address my research question. Thanks for the suggestion for checking the matrix correlations.
Ben, yes, very small differences in outputs for the languages. I’m not sure I quite follow you here:
“If you look at the participant-by-participant predictions (i.e. including both the language group and the participant-level random effect in the prediction) do the results make more sense?”
Do you mean something like this: Y ~ iso1 + iso2 + language + association + (language|participant) + (1|item)?
Removing language as a predictor is a possibility. As you mentioned, a lot of that variation is covered by participant and language isn’t key to my research question. This results in a drastically reduced intercept (3.48, CI 2.08-5.84), but a higher AIC (2266 vs 2299). How much relative weight does each carry? Is there literature on this? I may simply use the Mandarin group to keep things clean (results are solid in doing this).
To the “tangential worry”, yes, the predictors iso1 and iso2 are continuous, but the confidence interval doesn’t make sense. (It does seem reasonable when language is removed as a predictor, however).
For association, participants’ associations aren’t correct or incorrect. It simply indicates whether the participant associated the target word (a word of interest to me, but unknown to them at the time) with the sentence. E.g. “really” vs “rarely”; I ___ enjoy family gatherings. Which is correct? Neither, but many people will associate one over the other. Results are showing that participants are “hearing” the word they associate with the context rather than the word that is said. (This correspondingly explains why odds ratios are so high for when the target word is the same as the one participants associate with the context.).

All the best,
John Jones
E: johnathan.jones using gmail.com<mailto:johnathan.jones using gmail.com>
SM: linkedin.com/in/johnathanjones<http://linkedin.com/in/johnathanjones>



	[[alternative HTML version deleted]]