[R-meta] Questions about Omnibus tests

[Viechtbauer, Wolfgang (SP)]

I don't think there is an inherent need to 'handle high heterogeneity'. The data might be highly heterogeneous, but that might just be a reflection of the nature of the phenomenon under investigation.

On the other hand, there might be outliers and/or highly influential studies and I think it is useful to try to identify them. rstandard() and cooks.distance() are useful for this. Then one can present results with and without those studies to see how much this impacts the results. It seems like you have done that already.

Best,
Wolfgang

-----Original Message-----
From: Rafael Rios [mailto:biorafaelrm using gmail.com] 
Sent: Thursday, 08 November, 2018 17:50
To: Viechtbauer, Wolfgang (SP)
Cc: r-sig-meta-analysis using r-project.org
Subject: Re: [R-meta] Questions about Omnibus tests

Dear Wolfgang,

Thank you for the clarification! There are just a few more questions that I hope you may help me.

I used your script to calculate I² and found a high heterogeneity in my model (86.63%). 
#I²: http://www.metafor-project.org/doku.php/tips:i2_multilevel_multivariate
W <- diag(1/h_mc$vzf)
X <- model.matrix(model1)
P <- W - W %*% X %*% solve(t(X) %*% W %*% X) %*% t(X) %*% W
100 * sum(meta$sigma2) / (sum(meta$sigma2) + (meta$k-meta$p)/sum(diag(P)))

Do you have suggestions on how to handle with high heterogeneity among effect sizes? How may I conduct sensitivity tests in a multilevel meta-analysis using metafor? I identified (using a funnel plot) and removed outliers to reduce the heterogeneity and redo the model. Then, I compared the results of both models to evaluate if there is any discrepancies. Is this approach suitable to evaluate potential bias in the results? Or are there better alternatives?

Best wishes,

Rafael.
__________________________________________________________

Dr. Rafael Rios Moura
scientia amabilis

Behavioral Ecologist, PhD
Postdoctoral Researcher
Universidade Estadual de Campinas (UNICAMP)
Campinas, São Paulo, Brazil

Currículo Lattes: http://lattes.cnpq.br/4264357546465157
ORCID: http://orcid.org/0000-0002-7911-4734
Research Gate: https://www.researchgate.net/profile/Rafael_Rios_Moura2

Em qui, 8 de nov de 2018 às 13:48, Viechtbauer, Wolfgang (SP) <wolfgang.viechtbauer using maastrichtuniversity.nl> escreveu:
As for the model to use:

In general, you want to use: ~1|studyID/effectsizeID

Using list(~1|effectsizeID, ~1|studyID) may be correct if effectsizeID is unique for every row.

This is essentially a question of 'explicit' versus 'implict' nesting. See also:

https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2018-July/000896.html

Best,
Wolfgang

-----Original Message-----
From: Rafael Rios [mailto:biorafaelrm using gmail.com] 
Sent: Sunday, 04 November, 2018 0:32
To: Viechtbauer, Wolfgang (SP)
Cc: Michael Dewey; r-sig-meta-analysis using r-project.org
Subject: Re: [R-meta] Questions about Omnibus tests

Dear Wolfgang,

Could you please help me again with new questions?

Should I build model1 rather than model2 to control for the dependency among studyID and effectsizeID? 

model1=rma.mv(zf, vzf, mods=~mate_choice, random=list(~1|studyID/effectsizeID, ~1|species1), data = h_mc)
model2=rma.mv(zf, vzf, mods=~mate_choice, random=list(~1|effectsizeID, ~1|studyID, ~1|species1), data = h_mc)

I used your script to calculate I² and found a high heterogeneity in my model (86.63%). 
#I²: http://www.metafor-project.org/doku.php/tips:i2_multilevel_multivariate
W <- diag(1/h_mc$vzf)
X <- model.matrix(model1)
P <- W - W %*% X %*% solve(t(X) %*% W %*% X) %*% t(X) %*% W
100 * sum(meta$sigma2) / (sum(meta$sigma2) + (meta$k-meta$p)/sum(diag(P)))

Do you have suggestions on how to handle with high heterogeneity among effect sizes? How may I conduct sensitivity tests in a multilevel meta-analysis using metafor? I identified (using a funnel plot) and removed outliers to reduce the heterogeneity and redo the model. Is this approach suitable to evaluate potential bias in results? Or are there better alternatives?

Best wishes,

Rafael.
__________________________________________________________

Dr. Rafael Rios Moura
scientia amabilis

Behavioral Ecologist, PhD
Postdoctoral Researcher
Universidade Estadual de Campinas (UNICAMP)
Campinas, São Paulo, Brazil

Currículo Lattes: http://lattes.cnpq.br/4264357546465157
ORCID: http://orcid.org/0000-0002-7911-4734
Research Gate: https://www.researchgate.net/profile/Rafael_Rios_Moura2