[R-meta] Violation in non-independece of errors (head to head studies and mutlilevel meta-analysis)?

[Emily Finne]

Dear Wolfgang,


may I just chimp into your conversation, since after reading it, I am 
getting quite uncertain about our own analysis...

We have a combination of studies with multiple treatments compared to 
the same control group (in some studies) and of 2 different outcome 
measures (but also only in a subset of the studies, i.e. one outcome was 
present in all studies, the second was additionally present in a subset 
of studies). We first looked at the overall effect and in a next step 
tested different moderators.

We followed a multivariate approach with rma.mv and used the mutivariate 
parametrization as described in the konstantopoulos2011 example on the 
metafor website. So we have:

random = ~ Outcome | study

However,  we also have studies with multiple treatment groups. After 
reading your example code (from you reply below), I am not sure if it 
would be correct to add another random effect for each effect size, i.e.

random = ~ Outcome | study/id following your example below.

We did not do that, because we thought that with Outcome as inner factor 
we have added random variation between the different effect sizes within 
each study (for those cases where more than one effect size is included).

After trying out  random = ~ Outcome | trial/id, however, we get a 
different (higher) overall effect.

And after reading the website example again, I also compared the results 
for a three level (random = ~ 1 | study/Outcome) versus a multilevel 
parametrization (random = ~ Outcome | study).

In fact, these results also differ, and the overall estimated effect 
size for the three-level model is (in terms of robust CIs very nearly) 
the same as for the model with random = ~ Outcome | study/id.

Are we making a mistake if we ignore the additional "id" level random 
effect? Or do we add this random effect mistakenly twice, since we 
already have incorporated random varation within the studies by using 
Outcome as inner factor?

There are, in fact, 2 studies which had both: 2 outcomes but also 2 or 3 
treatment groups. So this may be the part we missed so far by ignoring 
"id" as additional level?

We have 6 studies which had both outcomes but only one treatment group. 
I am therefore not sure if we would overparameterize if we include "id", 
because these trials have two lines in the dataset (2 ids) that also 
stand for the 2 Outcomes.

The variance-covariance-matrix includes covariances for different 
outcomes within the same study, for different treatment groups within 
one study or for both, as appropriate. The profile likelihood plots for 
our orignal multivariate (~ Outcome | study) model looked fine.

Or would it be better to stick to the three level model?  - We describe, 
but not further analyze or discuss, differences between both outcomes, 
because both, though one gives a bit higher estimates, intend to measure 
the same outcome by different instruments. For the analysis of 
moderators which is our main question it makes more sense to look at 
only one moderator effect instead of one for each outcome measure, since 
only some studies used both outcome measures. But of course, we would 
like to take account of the fact that different outcome measures were used.

Would a change in the strategy likely result in changes of the fixed 
effects of moderators?

I hope it is to some extent clear what I mean.  Any help would be very 
much apreciated!

Thanks in advance!


Best,

Emily



Am 05.03.2018 um 10:12 schrieb Viechtbauer Wolfgang (SP):
> Dear Natan,
>
> If you reuse the information from a placebo group to compute multiple effects (i.e., treatment 1 vs placebo, treatment 2 vs placebo, etc.), then this automatically induces dependency in the sampling errors of the estimates. Code to compute the covariance for various effect size measures can be found here:
>
> http://www.metafor-project.org/doku.php/analyses:gleser2009
>
> So, you need to construct the full V matrix, use rma.mv(), and also include appropriate random effects (at least for studies and for each row of the dataset) in the model. So, something like this:
>
> dat$id <- 1:nrow(dat)
> res <- rma.mv(yi, V, mods = ~ <whatever fixed effects you think are needed>,
>                random = ~ 1 | study/id, data=dat)
>
> I am a bit confused about:
>
>> We are trying to avoid network meta-analysis, given we want our results
>> to be adjusted by several moderators that affect antidepressant response.
> Why do you think that network meta-analysis is not compatible with 'adjustment by moderators'?
>
> Best,
> Wolfgang
>
>> -----Original Message-----
>> From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces at r-
>> project.org] On Behalf Of Natan Gosmann
>> Sent: Saturday, 03 March, 2018 20:46
>> To: r-sig-meta-analysis at r-project.org
>> Subject: [R-meta] Violation in non-independece of errors (head to head
>> studies and mutlilevel meta-analysis)?
>>
>> Hello all,
>>
>> We are conducting a large multilevel meta-analysis using the metafor
>> package considering all RCTs that assessed medications vs placebo for
>> psychiatric disorders.
>>
>> We included all available outcomes from each study and therefore, we are
>> considering study and assessment instrument (scale) as random variables
>> in
>> the model. The yi comes from differences in standardized mean change
>> between medication and placebo for each study.
>>
>> We are trying to avoid network meta-analysis, given we want our results
>> to
>> be adjusted by several moderators that affect antidepressant response.
>>
>> However, we have doubts about how we can handle head to head studies
>> (studies with more than one medication) and studies with distinct dosages
>> of the same medication. We were thinking to just calculate differences
>> from
>> placebo (but placebo would be the same group for those studies - would be
>> the contrast group for more then one medication or dosage group).
>> Including
>> study ID as random variable already accounts for violation in
>> non-independence of errors? Is that an appropriate way of doing that?
>>
>> Alternatively, should we select only one medication from head to head
>> trials?
>>
>> I would very much appreciate if you could help us with that.
>>
>> Best regards,
>> Natan
> _______________________________________________
> R-sig-meta-analysis mailing list
> R-sig-meta-analysis at r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis


	[[alternative HTML version deleted]]