[R-meta] Can random-effects answer research questions in meta-analysis?

Luke Martinez m@rt|nez|ukerm @end|ng |rom gm@||@com
Fri Aug 6 02:58:17 CEST 2021

Dear Jack,

This is super helpful (and thank you for the super clear example:)! I will
look forward to the answers to my other questions.

Thank you all,

On Thu, Aug 5, 2021 at 6:20 PM Jack Solomon <kj.jsolomon using gmail.com> wrote:

> Dear Luke,
> I will attempt to answer your main question and leave the other ones
> to my senior colleagues. IMHO yes, you can answer RQs with random effects.
> A bit of background may clarify how this can happen. Inclusion of
> random-effects in general, and correlated random-effects in particular can
> be thought of as initiating a "secondary regression" (some may call it
> "latent regression") that is performed on the random effects. Specifically
> using correlated random-effects, you can potentially get the true effects
> specific to each level of whatever variable (e.g., outcomes relating to K =
> 3 aspects of writing competence, i.e., accuracy, fluency, and complexity)
> to vary & *covary* across the levels of your targeted grouping variable
> (e.g., study).
> Depending on how many unique correlations you can allow your model to
> estimate (e.g., let's say theoretically all of them; a K x K correlation
> matrix whose 3 unique entries are to be all estimated), the estimates of
> the correlation among the random-effects can tell you how the true effects
> specific to each level of your outcome variable are associated with true
> effects specific to "any other" level of your outcome variable. Here is an
> example from my substantive background.
> For example, if you were meta-analyzing studies to find out how teachers'
> feedback on students' writing impacts those 3 aspects of their writing
> competence that I mentioned above, then an estimate of +0.7 correlation
> between the true effects of accuracy and those of fluency would indicate
> that if such feedback has a small impact on students' writing accuracy
> (e.g., their grammar), then it is likely going to have a small impact on
> their fluency (e.g., how fast they write) as well and vice versa. So,
> potentially this can be a research question that is answered by
> "correlated" random effects.
> I'm sure senior list members can provide a much more comprehensive answer
> but, this was just my two cents,
> Jack
> # If I may, I do have two follow-ups, myself, for the senior list members:
> 1- Is there a way to get any kind of statistical significance for
> correlations among random-effects? (and is it easily implementable in R)
> 2- I know `rma.mv()` has an undocumented struct="GEN", I wonder how my
> answer to Luke would change if we used struct="GEN"?
> Thanks again,
> Jack
> On Thu, Aug 5, 2021 at 11:57 AM Luke Martinez <martinezlukerm using gmail.com>
> wrote:
>> Hello Friends,
>> We often only use estimates of fixed-effects to answer one or more
>> research
>> questions in a meta-analysis.
>> But is it also possible to specify one or more random effects to actually
>> answer one or more research questions in a meta-analysis?
>> If yes, then, which of the following should be a *priority* when fitting a
>> model:
>> (A) Specifying one or more random effects based on the research questions?
>> OR
>> (B) Specifying one or more random effects based on the fit of the model to
>> the data?
>> By *priority*, I mean deciding what to do when there is a conflict between
>> the above choices (e.g., A is desired, but B gives the superior fit).
>> Many thanks for sharing your insights,
>> Luke
>>         [[alternative HTML version deleted]]
>> _______________________________________________
>> R-sig-meta-analysis mailing list
>> R-sig-meta-analysis using r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis

	[[alternative HTML version deleted]]

More information about the R-sig-meta-analysis mailing list