[R-meta] IPD meta analysis / complex survey design

[GOSLING Corentin]

Dear Prof Viechtbauer,

Thank you so much for your very clear answer.

We really like this solution. As soon as we have completed the
meta-analysis, I will keep you updated on the results.

 Again, thank you so much for your insights

Corentin Gosling


Le ven. 5 mars 2021 à 12:07, Viechtbauer, Wolfgang (SP) <
wolfgang.viechtbauer using maastrichtuniversity.nl> a écrit :

> Hi Corentin,
>
> I did not mean to suggest that one should run several different analyses
> on a single dataset. That would indeed place too much of a burden on the
> authors of the individual studies.
>
> My suggestion is really about this part:
>
> >Our question was whether - within the same meta-analysis - we could
> >"safely" include effect sizes estimated by a standard logistic regression
> (when
> >data have a regular structure) +  effect sizes estimated by the svyglm
> function
> >(when the data have a complex structure).
>
> I cannot tell you if is safe or not. But what you can always do is combine
> these different types in a single analysis and then check if there are
> systematic differences between these two types of effect sizes. If there
> are no systematic differences, then this is (empirical) evidence that
> combining them is in some sense an acceptable thing to do.
>
> This approach is similar to checking if effect sizes extracted from
> published articles are systematically different from those extracted from
> unpublished sources in a meta-analysis. If there are systematic
> differences, we need to think about what the reason for the difference may
> be. If not, then this is one less thing to worry about.
>
> Best,
> Wolfgang
>
> >-----Original Message-----
> >From: GOSLING Corentin [mailto:corentin.gosling using gmail.com]
> >Sent: Friday, 05 March, 2021 10:33
> >To: Viechtbauer, Wolfgang (SP)
> >Cc: r-sig-meta-analysis using r-project.org
> >Subject: Re: [R-meta] IPD meta analysis / complex survey design
> >
> >Dear Prof Viechtbauer,
> >
> >Thank you very much for your reply!
> >
> >Sorry, my question was a bit misleading. In line with your suggestion,
> our aim is
> >to avoid merging ‘marginal ‘coefficients and ‘conditional’ coefficients
> by using
> >only the svyglm function as soon as the data has a complex structure
> (clustering
> >and/or weighting, etc...).
> >
> >You are entirely right, in situations with clustering only, we could
> compare 3
> >approaches : (i) select only 1 individual per cluster and use glm
> function, or
> >keep clustering and use (ii) glmer function or (iii) svyglm function.
> However,
> >we are a bit reluctant to make these comparisons for two reasons.
> First, as soon
> >as data have a more complex structure (e.g. sampling weights), the only
> approach
> >allowing to take this into account is the svyglm function. This makes
> comparisons
> >a bit strange, as in our examples, since one analysis is taking account
> of some
> >specificity of the design while the others are not. Second, from a
> practical point
> >of view, the burden on authors will become even more complicated as the
> time
> >required for analysis is already sometimes quite long (in particular
> because of
> >several multiple imputation models). We are concerned that the
> multiplication of
> >tests may sometimes make the analysis time so long that it may discourage
> some
> >authors from participating.
> >
> >Our question was whether - within the same meta-analysis - we could
> >"safely" include effect sizes estimated by a standard logistic regression
> (when
> >data have a regular structure) +  effect sizes estimated by the svyglm
> function
> >(when the data have a complex structure). By safely, I mean without
> having to
> >compare the results of the svyglm function to other functions (such as
> glm or
> >glmer) when data have a complex structure.
> >
> >If this is not possible, a more anecdotal question was whether it is
> possible to
> >"safely" include  effect sizes estimated by a  standard logistic
> regression (when
> >data have a regular structure) + effect sizes estimated by the glmer
> function
> >(when data have clustering).
> >
> >Thank you so much for your help!
> >
> >Best wishes
> >Corentin Gosling
> >
> >Le ven. 5 mars 2021 à 09:32, Viechtbauer, Wolfgang (SP)
> ><wolfgang.viechtbauer using maastrichtuniversity.nl> a écrit :
> >Dear Corentin,
> >
> >I cannot answer your question directly, that is, to what extent those
> results are
> >comparable to each other, although if svyglm() gives 'marginal'
> (population
> >averaged) coefficients in the sense of what a GEE model would do, then
> one could
> >argue that those should not be combined with 'conditional' coefficients
> that
> >glmer() provides (searching for combinations of terms like "GEE, marginal,
> >population averaged, logistic mixed-effects, conditional,
> subject-specific" should
> >turn up relevant discussions / papers).
> >
> >But leaving this aside, one could also just approach this issue entirely
> >empirically, that is, simply code the type of analysis / type of
> coefficient for
> >each study and examine in a moderator analysis whether there are
> systematic
> >differences between the different types.
> >
> >Best,
> >Wolfgang
> >
> >>-----Original Message-----
> >>From: R-sig-meta-analysis [mailto:
> r-sig-meta-analysis-bounces using r-project.org] On
> >>Behalf Of GOSLING Corentin
> >>Sent: Thursday, 04 March, 2021 11:29
> >>To: r-sig-meta-analysis using r-project.org
> >>Subject: [R-meta] IPD meta analysis / complex survey design
> >>
> >>Dear all
> >>
> >>I come back to you about the IPD meta-analysis we are conducting to
> explore
> >>the effect of month of birth on the persistence of ADHD. I had already
> >>asked for your help a few months ago when I was writing the protocol. We
> >>have since completed our systematic review and started to include data
> from
> >>different cohorts. As the month of birth is sensitive data, we do not ask
> >>the authors to send us the raw data: we have constructed an R-script that
> >>we send to the authors and which performs the analyses automatically and
> >>shares the anonymised results. We then carry out a classic two-stage
> >>meta-analysis based on summary results.
> >>
> >>We are facing a new challenge that we did not anticipate. Several studies
> >>involve complex survey design. Some studies have clusters (e.g., twin
> >>cohorts or assessments of several regular siblings per family), while
> >>others have even more complex sampling (and include for example sampling
> >>weights, stratum or finite population correction (fpc)). Some studies
> >>include both (clusters + stratum/weights/fpc).
> >>
> >>To analyse the data with clustering, naturally we thought of using mixed
> >>models via the glmer function of lme4 (our VD is binary: ADHD persistence
> >>yes/no). However, lme4 does not allow to handle - for the moment -
> sampling
> >>weights or stratifications. Therefore, for all data with clustering
> and/or
> >>weights and/or stratum and/or fpc, our idea was to use only the svyglm
> >>function of the survey package in order to have a coherent group of
> >>analyses (we know that the glmer and svyglm functions do not use the same
> >>coefficients (marginals vs. conditionals)).
> >>
> >>Our question is the following: can we group within the same meta-analysis
> >>coefficients that come from standard logistic regressions and
> coefficients
> >>that come from generalised mixed models fitted using glmer or generalised
> >>linear models adapted to complex designs fitted using svyglm?
> >>
> >>To support our question, we performed some tests on a dataset including
> >>clusters and sampling weights. Here are the results :
> >>
> >>[...]
> >>
> >>As you can see, the results are almost the same from the models, except
> >>when we take into account sampling weights. I hope that our problem is
> >>clearly exposed
> >>
> >>Thank you very much in advance for your help!
> >>
> >>Corentin J Gosling
>

	[[alternative HTML version deleted]]