[R-meta] multiple models in one study

[James Pustejovsky]

Hi Valeria,

It sounds like you're interested in synthesizing sets of regression
coefficients and the issue is that some papers report multiple regression
specifications that fit your criteria. For instance, a paper might report
three models:
Model 1: Y = b0 + b1 A + b2 B + b3 C
Model 2: Y = b0 + b1 A + b2 B + b3 C + b4 D + b5 E + b6 F
Model 3: Y = b0 + b1 A + b2 B + b3 C + b4 D + b5 E + b6 F + <a bunch of
other stuff>
And perhaps you're just interested in analyzing the coefficients (b1, b2,
b3). Does this description track with what you're wondering about?

If so, then the challenge is that the definition of regression coefficients
depends on ALL of the variables in the model, so the coefficients (b1, b2,
b3) from Model 1 aren't really estimating the same parameters as the (b1,
b2, b3) from Model 3. From your research aims and inclusion criteria, is it
possible to define an "ideal analysis" that most closely matches the
questions you're trying to investigate? If so, then perhaps you can select
results from each study that come closest to matching the ideal analysis.
This would be pretty similar to the "best-set" strategy.

Another thing to consider is how similar the regression specifications are
across studies. For example, say that you've got 10 studies meeting
inclusion criteria. For the first 8 studies, the specification that most
closely matches your ideal analysis is Model 2. But then for study 9, the
only thing that's reported is Model 1 and for study 10, Models 1, 2, and 3
are all reported and Model 3 is the one that most closely matches your
ideal analysis. For study 10, should you take the coefficients from Model 2
or Model 3? The tension is between choosing the results that are closest to
the ideal or choosing the results that are most comparable across all
included studies. And for study 1, should you take the results from Model 1
or just exclude it entirely because it doesn't report a specification that
controls for factors D, E, and F?

James


On Thu, Mar 23, 2023 at 3:17 AM Viechtbauer, Wolfgang (NP) via
R-sig-meta-analysis <r-sig-meta-analysis using r-project.org> wrote:

> Dear Valeria,
>
> I would say this depends on the aims. If there is one key predictor of
> interest, then I would focus on that. If that's not the case, then I would
> extract all the ones that are of interest. Taking an average of all
> coefficients (if this is what the "average-set" approach entails) doesn't
> make much sense to me unless they are all measuring the same construct in
> the same direction and in the same units (all unlikely).
>
> If you extract multiple coefficients, you of course have to account for
> the fact that they are not independent.
>
> Best,
> Wolfgang
>
> >-----Original Message-----
> >From: R-sig-meta-analysis [mailto:
> r-sig-meta-analysis-bounces using r-project.org] On
> >Behalf Of Valeria Ivaniushina via R-sig-meta-analysis
> >Sent: Wednesday, 22 March, 2023 17:52
> >To: R meta
> >Cc: Valeria Ivaniushina
> >Subject: [R-meta] multiple models in one study
> >
> >Hi
> >
> >I want to perform a meta-analysis of the relation between the outcome and
> >key explanatory variables expressed as regression coefficients.
> >
> >As a rule, authors report several models with different specifications. I
> >wonder which regression coefficients should I collect?
> >
> >In the book Meta-regression analysis in economics and business (Stanley &
> >Doucouliagos, 2012)
> >several approaches are described:
> >- The best-set = ONE estimate from each study, using the KEY regression
> >from each paper
> >- The average-set = an average of all coefficients reported in the study
> >- The all-set = all relevant estimates reported in the study
> >
> >Which approach is preferable? Are there additional considerations that I
> >have to take into account?
> >
> >Regards,
> >Valeria
>
> _______________________________________________
> R-sig-meta-analysis mailing list @ R-sig-meta-analysis using r-project.org
> To manage your subscription to this mailing list, go to:
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>

	[[alternative HTML version deleted]]