[R-meta] Questions about Omnibus tests

[Rafael Rios]

Dear Wolfgang,

Could you please help me again with new questions?

Should I build model1 rather than model2 to control for the dependency
among studyID and effectsizeID?

model1=rma.mv(zf, vzf, mods=~mate_choice,
random=list(~1|studyID/effectsizeID, ~1|species1), data = h_mc)
model2=rma.mv(zf, vzf, mods=~mate_choice, random=list(~1|effectsizeID,
~1|studyID, ~1|species1), data = h_mc)

I used your script to calculate I² and found a high heterogeneity in my
model (86.63%).
#I²: http://www.metafor-project.org/doku.php/tips:i2_multilevel_multivariate
W <- diag(1/h_mc$vzf)
X <- model.matrix(model1)
P <- W - W %*% X %*% solve(t(X) %*% W %*% X) %*% t(X) %*% W
100 * sum(meta$sigma2) / (sum(meta$sigma2) + (meta$k-meta$p)/sum(diag(P)))

Do you have suggestions on how to handle with high heterogeneity among
effect sizes? How may I conduct sensitivity tests in a multilevel
meta-analysis using metafor? I identified (using a funnel plot) and removed
outliers to reduce the heterogeneity and redo the model. Is this approach
suitable to evaluate potential bias in results? Or are there better
alternatives?

Best wishes,

Rafael.
__________________________________________________________

Dr. Rafael Rios Moura
*scientia amabilis*

Behavioral Ecologist, PhD
Postdoctoral Researcher
Universidade Estadual de Campinas (UNICAMP)
Campinas, São Paulo, Brazil

Currículo Lattes: http://lattes.cnpq.br/4264357546465157
ORCID: http://orcid.org/0000-0002-7911-4734
Research Gate: https://www.researchgate.net/profile/Rafael_Rios_Moura2




<http://buscatextual.cnpq.br/buscatextual/visualizacv.do?id=K4244908A8>


Em ter, 30 de out de 2018 às 16:01, Rafael Rios <biorafaelrm using gmail.com>
escreveu:

> Dear Wolfgang,
>
> Thank you for the amazing clarifications! I think I finally have a better
> picture about the meta-analytic procedures.
>
> Best wishes,
>
> Rafael.
> __________________________________________________________
>
> Dr. Rafael Rios Moura
> *scientia amabilis*
>
> Behavioral Ecologist, PhD
> Postdoctoral Researcher
> Universidade Estadual de Campinas (UNICAMP)
> Campinas, São Paulo, Brazil
>
> Currículo Lattes: http://lattes.cnpq.br/4264357546465157
> ORCID: http://orcid.org/0000-0002-7911-4734
> Research Gate: https://www.researchgate.net/profile/Rafael_Rios_Moura2
>
>
>
>
> <http://buscatextual.cnpq.br/buscatextual/visualizacv.do?id=K4244908A8>
>
>
> Em ter, 30 de out de 2018 às 15:28, Viechtbauer, Wolfgang (SP) <
> wolfgang.viechtbauer using maastrichtuniversity.nl> escreveu:
>
>> Dear Rafael,
>>
>> 1. "Does the QM-test, with an intercept in the model, evaluates if the
>> average true outcomes of subgroups differ from the reference level or from
>> 0?"
>>
>> From the reference level.
>>
>> "Why are the graph results so different from the QM-test with an
>> intercept in the model?"
>>
>> Your graph is not correct. It should be:
>>
>> preds <- predict(meta, newmods=rbind(c(0,0), c(1,0), c(0,1)))
>> forest(preds$pred, sei=preds$se, slab=c("female", "male", "mutual"))
>>
>> The differences between the three levels are small.
>>
>> "Should I evaluate results using anova(meta,btt=1:3)?"
>>
>> anova(meta,btt=1:3) tests if all 3 groups have a zero effect. That does
>> not test for differences between groups.
>>
>> "Was the argument linfct=rbind(c(0,0,1)) used to compare the subgroups of
>> female choice (reference level) and male choice?"
>>
>> No, this compares 'mutual' with 'female'.
>>
>> "What am I evaluating by using summary(glht(meta,
>> linfct=rbind(female=c(1,0,0), male=c(0,1,0))), test=Chisqtest())"
>>
>> You are evaluating whether the intercept (and hence the effect for
>> 'female') is 0 and whether there is a difference between 'male' and
>> 'female'.
>>
>> 2. "What is the best approach to measure heterogeneity in a multilevel
>> meta-analysis?"
>>
>> I don't know what is best. The link you posted provides some
>> possibilities for computing I^2-like measures for multilevel/multivariate
>> models.
>>
>> 3. "I used the standard deviation to weight the effect sizes, according
>> to Zaykin (2011). Is variance a better measure of weight than se in a
>> multilevel meta-analysis?"
>>
>> As mentioned by Michael, this article is irrelevant.
>>
>> 4. "An alternative could be to include this potential_sce as a fixed
>> variable."
>>
>> Sure.
>>
>> "Is this model more appropriate?: meta=rma.mv(zf, sezf,
>> mods=~mate_choice+potential_sce, random = list (~1|effectsizeID,
>> ~1|studyID, ~1|species1), data = h_mc)"
>>
>> You should pass the variances to the function:
>>
>> meta=rma.mv(zf, vzf, mods=~mate_choice+potential_sce, random = list
>> (~1|effectsizeID, ~1|studyID, ~1|species1), data = h_mc)
>>
>> Best,
>> Wolfgang
>>
>> -----Original Message-----
>> From: Rafael Rios [mailto:biorafaelrm using gmail.com]
>> Sent: Tuesday, 30 October, 2018 6:16
>> To: Viechtbauer, Wolfgang (SP)
>> Cc: Michael Dewey; r-sig-meta-analysis using r-project.org
>> Subject: Re: [R-meta] Questions about Omnibus tests
>>
>> Dear Wolfgang,
>>
>> Thank you for the very helpful advices! I will be grateful if you could
>> help me again with my new questions. I organized them in the topics bellow.
>>
>> 1. Does the QM-test, with an intercept in the model, evaluates if the
>> average true outcomes of subgroups differ from the reference level or from
>> 0? I found a p>0.05, probably meaning that there is no difference among
>> subgroups. However, if you analyze the graph, there a higher effect size
>> for the subgroup of female choice compared to others. So, I am not sure
>> about the best approach to evaluate differences among outcomes. Why are the
>> graph results so different from the QM-test with an intercept in the model?
>> Should I evaluate results using anova(meta,btt=1:3)?
>>
>> You also suggested that the script for pairwise comparisons was wrong.
>> According to the link that you provided, it can also be drawn
>> as summary(glht(meta, linfct=rbind(c(0,0,1), c(0,1,0), c(0,-1,1))),
>> test=adjusted("none")). Was the argument linfct=rbind(c(0,0,1)) used to
>> compare the subgroups of female choice (reference level) and male choice?
>> What am I evaluating by using summary(glht(meta,
>> linfct=rbind(female=c(1,0,0), male=c(0,1,0))), test=Chisqtest())?
>>
>> 2. Thank you for the correction of I² formula. What is the best approach
>> to measure heterogeneity in a multilevel meta-analysis? Maybe, this one:
>> http://www.metafor-project.org/doku.php/tips:i2_multilevel_multivariate
>>
>> 3. I used the standard deviation to weight the effect sizes, according to
>> Zaykin (2011). Is variance a better measure of weight than se in a
>> multilevel meta-analysis? Reference: D. V. Zaykin, Optimally weighted
>> Z-test is a powerful method for combining probabilities in meta-analysis.
>> J. Evol. Biol. 24, 1836–1841 (2011).
>>
>> 4. Finally, I agree with the exclusion of potential_sce as a random
>> variable. However, I need to control for this variable. An alternative
>> could be to include this potential_sce as a fixed variable. Is this model
>> more appropriate?: meta=rma.mv(zf, sezf,
>> mods=~mate_choice+potential_sce, random = list (~1|effectsizeID,
>> ~1|studyID, ~1|species1), data = h_mc).
>>
>> Thank you again for the help.
>>
>> Best wishes,
>>
>> Rafael.
>> __________________________________________________________
>>
>> Dr. Rafael Rios Moura
>> scientia amabilis
>>
>> Behavioral Ecologist, PhD
>> Postdoctoral Researcher
>> Universidade Estadual de Campinas (UNICAMP)
>> Campinas, São Paulo, Brazil
>>
>> Currículo Lattes: http://lattes.cnpq.br/4264357546465157
>> ORCID: http://orcid.org/0000-0002-7911-4734
>> Research Gate: https://www.researchgate.net/profile/Rafael_Rios_Moura2
>>
>

	[[alternative HTML version deleted]]