[R-meta] Can random-effects answer research questions in meta-analysis?

[Luke Martinez]

Dear Jack,

This is super helpful (and thank you for the super clear example:)! I will
look forward to the answers to my other questions.

Thank you all,
Luke

On Thu, Aug 5, 2021 at 6:20 PM Jack Solomon <kj.jsolomon using gmail.com> wrote:

> Dear Luke,
>
> I will attempt to answer your main question and leave the other ones
> to my senior colleagues. IMHO yes, you can answer RQs with random effects.
> A bit of background may clarify how this can happen. Inclusion of
> random-effects in general, and correlated random-effects in particular can
> be thought of as initiating a "secondary regression" (some may call it
> "latent regression") that is performed on the random effects. Specifically
> using correlated random-effects, you can potentially get the true effects
> specific to each level of whatever variable (e.g., outcomes relating to K =
> 3 aspects of writing competence, i.e., accuracy, fluency, and complexity)
> to vary & *covary* across the levels of your targeted grouping variable
> (e.g., study).
>
> Depending on how many unique correlations you can allow your model to
> estimate (e.g., let's say theoretically all of them; a K x K correlation
> matrix whose 3 unique entries are to be all estimated), the estimates of
> the correlation among the random-effects can tell you how the true effects
> specific to each level of your outcome variable are associated with true
> effects specific to "any other" level of your outcome variable. Here is an
> example from my substantive background.
>
> For example, if you were meta-analyzing studies to find out how teachers'
> feedback on students' writing impacts those 3 aspects of their writing
> competence that I mentioned above, then an estimate of +0.7 correlation
> between the true effects of accuracy and those of fluency would indicate
> that if such feedback has a small impact on students' writing accuracy
> (e.g., their grammar), then it is likely going to have a small impact on
> their fluency (e.g., how fast they write) as well and vice versa. So,
> potentially this can be a research question that is answered by
> "correlated" random effects.
>
> I'm sure senior list members can provide a much more comprehensive answer
> but, this was just my two cents,
> Jack
>
> # If I may, I do have two follow-ups, myself, for the senior list members:
> 1- Is there a way to get any kind of statistical significance for
> correlations among random-effects? (and is it easily implementable in R)
> 2- I know `rma.mv()` has an undocumented struct="GEN", I wonder how my
> answer to Luke would change if we used struct="GEN"?
>
> Thanks again,
> Jack
>
> On Thu, Aug 5, 2021 at 11:57 AM Luke Martinez <martinezlukerm using gmail.com>
> wrote:
>
>> Hello Friends,
>>
>> We often only use estimates of fixed-effects to answer one or more
>> research
>> questions in a meta-analysis.
>>
>> But is it also possible to specify one or more random effects to actually
>> answer one or more research questions in a meta-analysis?
>>
>> If yes, then, which of the following should be a *priority* when fitting a
>> model:
>>
>> (A) Specifying one or more random effects based on the research questions?
>> OR
>> (B) Specifying one or more random effects based on the fit of the model to
>> the data?
>>
>> By *priority*, I mean deciding what to do when there is a conflict between
>> the above choices (e.g., A is desired, but B gives the superior fit).
>>
>> Many thanks for sharing your insights,
>> Luke
>>
>>         [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> R-sig-meta-analysis mailing list
>> R-sig-meta-analysis using r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>>
>

	[[alternative HTML version deleted]]