[R-meta] Can random-effects answer research questions in meta-analysis?

[Jack Solomon]

Dear Luke,

I will attempt to answer your main question and leave the other ones
to my senior colleagues. IMHO yes, you can answer RQs with random effects.
A bit of background may clarify how this can happen. Inclusion of
random-effects in general, and correlated random-effects in particular can
be thought of as initiating a "secondary regression" (some may call it
"latent regression") that is performed on the random effects. Specifically
using correlated random-effects, you can potentially get the true effects
specific to each level of whatever variable (e.g., outcomes relating to K =
3 aspects of writing competence, i.e., accuracy, fluency, and complexity)
to vary & *covary* across the levels of your targeted grouping variable
(e.g., study).

Depending on how many unique correlations you can allow your model to
estimate (e.g., let's say theoretically all of them; a K x K correlation
matrix whose 3 unique entries are to be all estimated), the estimates of
the correlation among the random-effects can tell you how the true effects
specific to each level of your outcome variable are associated with true
effects specific to "any other" level of your outcome variable. Here is an
example from my substantive background.

For example, if you were meta-analyzing studies to find out how teachers'
feedback on students' writing impacts those 3 aspects of their writing
competence that I mentioned above, then an estimate of +0.7 correlation
between the true effects of accuracy and those of fluency would indicate
that if such feedback has a small impact on students' writing accuracy
(e.g., their grammar), then it is likely going to have a small impact on
their fluency (e.g., how fast they write) as well and vice versa. So,
potentially this can be a research question that is answered by
"correlated" random effects.

I'm sure senior list members can provide a much more comprehensive answer
but, this was just my two cents,
Jack

# If I may, I do have two follow-ups, myself, for the senior list members:
1- Is there a way to get any kind of statistical significance for
correlations among random-effects? (and is it easily implementable in R)
2- I know `rma.mv()` has an undocumented struct="GEN", I wonder how my
answer to Luke would change if we used struct="GEN"?

Thanks again,
Jack

On Thu, Aug 5, 2021 at 11:57 AM Luke Martinez <martinezlukerm using gmail.com>
wrote:

> Hello Friends,
>
> We often only use estimates of fixed-effects to answer one or more research
> questions in a meta-analysis.
>
> But is it also possible to specify one or more random effects to actually
> answer one or more research questions in a meta-analysis?
>
> If yes, then, which of the following should be a *priority* when fitting a
> model:
>
> (A) Specifying one or more random effects based on the research questions?
> OR
> (B) Specifying one or more random effects based on the fit of the model to
> the data?
>
> By *priority*, I mean deciding what to do when there is a conflict between
> the above choices (e.g., A is desired, but B gives the superior fit).
>
> Many thanks for sharing your insights,
> Luke
>
>         [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-meta-analysis mailing list
> R-sig-meta-analysis using r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>

	[[alternative HTML version deleted]]