[R-meta] Meta-analysis with observational and experimental studies

[Viechtbauer, Wolfgang (NP)]

Dear Karen,

I suspect this thread isn't going to lead to a whole lot of useful responses (but would love to be proven wrong). The reason why I say this is that you seem to be at the stage of a meta-analysis where the goal is to figure out how to structure the data, what kind of effect sizes can be computed, and what kind of analyses can be conducted, given the goals and purposes of the meta-analysis itself, under the constraints of the types of studies that have been conducted and the information reported therein. In my experience, this is the most difficult task of a meta-analysis with complex data structures. This step is best carried out in person where we can look together at the data, some example studies, and some possible ways of structuring the data, using a white board or a whole lot of scratch paper.

But briefly: There is no absolute rule on whether one can meta-analyze observational and experimental research together or not. Whether this makes sense or not is highly context dependent. Beyond this, I cannot comment on any approaches without a *lot* more details. But again, I suspect a mailing list like this is not a replacement for sitting down in person when it comes to addressing such general questions.

Best,
Wolfgang

> -----Original Message-----
> From: R-sig-meta-analysis <r-sig-meta-analysis-bounces using r-project.org> On Behalf
> Of PARRA DE LA ROSA, KAREN via R-sig-meta-analysis
> Sent: Sunday, November 5, 2023 13:48
> To: Michael Dewey <lists using dewey.myzen.co.uk>
> Cc: PARRA DE LA ROSA, KAREN <karen.parra01 using estudiant.upf.edu>; R Special
> Interest Group for Meta-Analysis <r-sig-meta-analysis using r-project.org>
> Subject: Re: [R-meta] Meta-analysis with observational and experimental studies
>
> Dear Michael,
>
> Many thanks for your reply. I am interested in multiple groups in each
> trial. What you are suggesting is to use the raw mean per group and
> then transform it into r (which is the metric I am using) instead of
> calculating the contrast between groups?
>
> Best,
>
> Karen
>
> El dom, 5 nov 2023 a las 12:49, Michael Dewey
> (<lists using dewey.myzen.co.uk>) escribió:
> >
> > Dear Karen
> >
> > I will leave it to others to coment on model choice but one thing which
> > occurs to me is that if you just use one arm in a trial does that not
> > convert it into an observational study?
> >
> > Michael
> >
> > On 05/11/2023 10:15, PARRA DE LA ROSA, KAREN via R-sig-meta-analysis wrote:
> > > Hi everyone,
> > >
> > > I am conducting a systematic review and meta-analysis on the impact of
> > > different types of appearance-focused social media activities on body
> > > image. I wanted to test a novel categorization of these types of
> > > social media activities so we can better understand if the shared
> > > characteristics of different activities could explain their
> > > effect/relationship with body image. On the other hand, I am also
> > > interested in having an estimate per activity. The dataset has 130
> > > studies, half observational (cross-sectional, longitudinal, experience
> > > sampling), and half experimental studies. The vast majority of them
> > > contribute more than one effect size because of different activities
> > > and multiple outcomes. This is my first meta-analysis and also my
> > > first time using R so I have many questions regarding how to specify
> > > the model.
> > >
> > > I have read that observational and experimental research cannot be
> > > synthesized in the same meta-analysis as they are answering different
> > > questions. So I was running two different analyses for each type of
> > > study. As for observational studies I fitted with metafor and
> > > clubsandwich a correlated and hierarchical model with robust variance
> > > estimation to account for the multiple dependencies among effect
> > > sizes. So effect sizes are nested within studies. However, the problem
> > > I am facing is with experimental studies. I have organized them in a
> > > different file with the mean, sd, and n per treatment condition. I am
> > > not interested in the comparison with control groups, rather I am
> > > interested in understanding if, for instance, exposure to body ideals
> > > is better/worse than posting this type of content. I have several
> > > comparisons like the one in the example. I thought that a network
> > > meta-analysis in netmeta could be the best solution but as far as I
> > > know, this package does not allow for moderator analysis, so could not
> > > be able to run a subgroup analysis with the categories of activities.
> > > I do not know what the best approach to analyze this data. Also, I
> > > have never seen in my field a paper with two different types of
> > > meta-analysis: a three-level for observational and an NMA for
> > > experimental.
> > >
> > > Hope someone on the mailing list could help me with some of this doubt.
> > >
> > > Thanks in advance for your help and guidance.
> > >
> > > Best,
> > >
> > > Karen