[R-meta] Violation in non-independece of errors (head to head studies and mutlilevel meta-analysis)?

Tue Apr 3 11:30:18 CEST 2018

Dear Wolfgang,

thank you very much for the references.

There's always a lot to learn!

Best,

Emily

Am 31.03.2018 um 17:52 schrieb Viechtbauer Wolfgang (SP):
> Convergence problems are difficult to anticipate. But in general, they are more likely to appear when fitting complex models and/or when the dataset is small. It is also possible that one is trying to fit an overparameterized model, that is, a model where certain parameters are not identifiable. The issue of identifiability is complex, but some articles that deal with this are:
>
> Kreutz, C., Raue, A., Kaschek, D., & Timmer, J. (2013). Profile likelihood in systems biology. The FEBS Journal, 280(11), 2564-2571.
>
> Lavielle, M., & Aarons, L. (2016). What do we mean by identifiability in mixed effects models? Journal of Pharmacokinetics and Pharmacodynamics, 43(1), 111-122.
>
> Raue, A., Kreutz, C., Maiwald, T., Bachmann, J., Schilling, M., Klingmuller, U., & Timmer, J. (2009). Structural and practical identifiability analysis of partially observed dynamical models by exploiting the profile likelihood. Bioinformatics, 25(15), 1923-1929.
>
> Raue, A., Kreutz, C., Maiwald, T., Klingmuller, U., & Timmer, J. (2011). Addressing parameter identifiability by model-based experimentation. IET Systems Biolology, 5(2), 120-130.
>
> Wang, W. (2013). Identifiability of linear mixed effects models. Electronic Journal of Statistics, 7, 244-263.
>
> One way of assessing parameter identifiability is to examine/plot profile likelihoods. This is what the profile() function is for. When fitting complex models, I would always recommend to profile all variance/correlation components.
>
> Even if all components are identifiable, it may be difficult to find the ML/REML estimates. Complex models require optimization over a large number of parameters and this is not a trivial task. rma.mv() uses nlminb() by default, but that is not always the best option. One can try many other optimizers (using the control arguments 'optimizer' and 'optmethod'). See the 'Note' section under help(rma.mv).
>
> As for power, there are these two articles:
>
> Hedges, L. V., & Pigott, T. D. (2001). The power of statistical tests in meta-analysis. Psychological Methods, 6(3), 203-217.
> Hedges, L. V., & Pigott, T. D. (2004). The power of statistical tests for moderators in meta-analysis. Psychological Methods, 9(4), 426-445.
>
> But they do not deal with complex models (just standard random/mixed-effects models). Indeed, for complex models, one would need to take a simulation approach.
>
> Best,
> Wolfgang
>
> -----Original Message-----
> From: Emily Finne [mailto:emily.finne at uni-bielefeld.de]
> Sent: Wednesday, 28 March, 2018 10:35
> To: Viechtbauer Wolfgang (SP); r-sig-meta-analysis at r-project.org
> Subject: Re: [R-meta] Violation in non-independece of errors (head to head studies and mutlilevel meta-analysis)?
>
> I really wished, everything WOULD be so obvious to me... !
>
> For this analysis the results turned out to be nearly unchanged when including the crossed random effects, although you guessed right that convergence problems could emerge. These were related to those parameters estimating the covariaces between effects within studies.
>
> I wonder how one can anticipate such problems in advance or rather determine how complex a model can be with given data to have enough power to test (fixed) moderator effects of interest and to make sure that confidence intervals are reliable.
> Is there something like a formal power analysis for meta-analysis or meta-regression? I am aware that this is complex and I think in mixed effects models, in general, one would use simulations.
>
> Any advice on literature  I could read to get prepared for further projects?
>    
> Best,
> Emily
>
> Am 25.03.2018 um 12:16 schrieb Viechtbauer Wolfgang (SP):
> See comments below.
>
> Best,
> Wolfgang
>
> -----Original Message-----
> From: Emily Finne [mailto:emily.finne at uni-bielefeld.de]
> Sent: Saturday, 24 March, 2018 21:41
> To: Viechtbauer Wolfgang (SP); r-sig-meta-analysis at r-project.org
> Subject: Re: [R-meta] Violation in non-independece of errors (head to
> head studies and mutlilevel meta-analysis)?
>
> Dear Wolfgang,
>
> oh yes, many people were  sick during the last weeks, here too. I hope
> you're feeling better by now.
>
> Thanks - not 100% but good enough to catch up on work.
>
> Yes, that is exactly the data structure I have. I completely missed out
> to think of the problem as crossed random effects! Thank you!
>
> I am quite sure that I constructed the var-cov-matrix V right. I used the
> formulas by Gleser & Olkin and James Pustejovsky. I double-checked the
> resulting matrix. Additionally, I used robust estimation, since most
> correlations between outcomes were only a best guess.
>
> Great!
>
> Only to make sure, that I understand correctly the point about the random
> effects: I code two different treatment groups within one study with
> different numbers starting with 1 (for example) and than use the code you
> provided for the crossed random effects. But the numbers given to
> different treatments are arbitrary and don't mean that the group with
> 'treatment = 1' always got the same treatment. It is only to code that
> treatment 1 and 2 within one study are different (say medication A and B
> each compared against a placebo control), not that 1 and 2 always means
> the same thing in different studies (it could also stand for medication B
> and C vs. control in another study). Am I right?
>
> Correct!
>
> On the other hand, if treatment 1 always stood for medication A and treatment 2 always stood for medication B (across all studies), then it would make sense to distiniguish the two and, for example, allow a different tau^2 for treatment 1 vs treatment 2. I assumed that for 'outcome', this is actually the the case (so, for example, outcome 1 always stands for measure X and outcome 2 always stands for measure Y).
>
> One thing I forgot to mention: At the moment, the 'inner' terms in a '~ inner | outer' formula under random must be a character/factor variable. So, if 'outcome' and 'trt' are numeric, then use:
>
> random = list(~ factor(outcome) | study, ~ factor(trt) | study), struct=c("UN","CS")
>
> Or you could code 'outcome' and 'trt' as character variables to begin with.
>
> The treatments we look at in our analysis, in fact, are all different in
> some aspects although pursuing the same goal. We use characteristics of
> the treatments as moderators then and hope to explain differences in
> effect sizes.
>
> Again, spot on. Using this example, it also illustrates why it would make sense to include a fixed effect for 'outcome' (since outcome 1 and 2 are uniquely defined), while it would not make sense to include a fixed effect for 'trt'. For the latter, as you say, we can use characteristics of the treatments as moderators.
>
> Sorry if this is all obvious to you, but having this written down here is useful for future reference. Also, to come back to the Konstantopoulos (2011) and Berkey et al. (1998) examples, this also explains why we would use:
>
> rma.mv(yi, vi, random = ~ factor(study) | district, struct="CS", data=dat)
>
> in the Konstantopoulos example (https://tinyurl.com/ybpzn5ra) (so no fixed effect for 'study' and struct="CS" -- which is actually the default, but added here for clarity)
>
> and
>
> rma.mv(yi, V, mods = ~ outcome - 1, random = ~ outcome | trial, struct="UN", data=dat)
>
> in the Berkey example (https://tinyurl.com/y9yv366v) (so with fixed effect for 'outcome' and struct="UN").
>
> In the first case, the coding of 'study' within 'district' is arbitary. In the second case, the coding of 'outcome' within 'trial' is meaningful.
>
> Again, thank you so much for you detailed help.
>
> I will try, if the model with the crossed effects converge. Otherwise, I
> would stick to the old model (only  random = ~ outcome | study,
> struct="UN") and discuss this as a limitation.
>
> Best,
> Emily

-- 
Dr. Emily Finne, Dipl.-Psych.

Universität Bielefeld
Fakultät für Gesundheitswissenschaften
AG 4: Prävention und Gesundheitsförderung
Postfach 10 01 31

D-33501 Bielefeld

Mail: emily.finne at uni-bielefeld.de
http://www.uni-bielefeld.de/gesundhw/ag4

	[[alternative HTML version deleted]]