[R-meta] Fwd: Overlapping CIs with significant difference among subgroups

[Rafael Rios Moura]

Dear Dr. Wolfgang,

I implemented your suggestions and obtained some different results. I found
a difference in the variable field, for example, which represents whether
the data was collected in a laboratory (no) or in the field (yes).
Therefore, the source of the data may introduce a noise in the estimation
of an average effect size. Since the moderator has only two levels, I
cannot add it among random variables. Or is there a way of including field
as a random variable (e.g. effectsizeID|field)? I can also add it as a
fixed predictor. In this situation, how can I estimate the average effect
size, because the intercept will represent one level of the moderator? I
added the variable field to my dataset. Thank you in advance.

Best wishes,

Rafael.
_______________________________________________________

*Prof. Dr. Rafael Rios Moura*

*scientia amabilis *
Coordenador de Pesquisa e do NEPEE/CNPq
UEMG - Unidade Ituiutaba

ORCID: http://orcid.org/0000-0002-7911-4734
Currículo Lattes: http://lattes.cnpq.br/4264357546465157
<http://orcid.org/0000-0002-7911-4734>
<http://lattes.cnpq.br/4264357546465157>
<http://lattes.cnpq.br/4264357546465157>Research Gate:
https://www.researchgate.net/profile/Rafael_Rios_Moura2
<http://orcid.org/0000-0002-7911-4734>
Rios de Ciência: https://www.youtube.com/channel/UCu2186wIJKji22ai8tvlUfg
<http://orcid.org/0000-0002-7911-4734>


Em ter., 6 de out. de 2020 às 16:17, Viechtbauer, Wolfgang (SP) <
wolfgang.viechtbauer using maastrichtuniversity.nl> escreveu:

> Dear Rafael,
>
> The SEs of the predicted average outcomes for the various levels can be
> quite different than the SEs of the difference between levels.
>
> You can get the predicted average outcome for all four levels with:
>
> pred.r = predict(res, transf=transf.ztor, digits=3, newmods =
> rbind(c(0,0,0), c(1,0,0), c(0,1,0), c(1,1,1)))
> pred.r
>
> And then you can plot them with:
>
> forest(pred.r$pred, ci.lb=pred.r$ci.lb, ci.ub=pred.r$ci.ub,
> slab=c("Without grouping","Temporal grouping","Spatial grouping", "Both"))
>
> Indeed, CIs are wide and overlap. But let's now compute the predicted
> average difference between levels when compared against the "Without
> grouping" level:
>
> pred.r = predict(res, transf=transf.ztor, digits=3, newmods =
> rbind(c(1,0,0) - c(0,0,0), c(0,1,0) - c(0,0,0), c(1,1,1) - c(0,0,0)),
> intercept=FALSE)
> pred.r
>
> forest(pred.r$pred, ci.lb=pred.r$ci.lb, ci.ub=pred.r$ci.ub, slab=c("Diff
> Temporal grouping","Diff Spatial grouping", "Diff Both"))
>
> Now the CIs are quite narrow and exlude 0.
>
> This aside, I would recommend that you include random effects for species
> twice, once without and once with the phylogenetic correlation matrix:
>
> h$speciesIDnon <- h$speciesID
> res <- rma.mv(zf, vzf, mods=~sce_temporal*sce_spatial, random = list
> (~1|effectsizeID, ~1|studyID, ~1|speciesIDnon, ~1|speciesID),
> R=list(speciesID=corr), data=h)
>
> This is model (15) from:
>
> Nakagawa, S., & Santos, E. S. A. (2012). Methodological issues and
> advances in biological meta-analysis. Evolutionary Ecology, 26(5),
> 1253-1274.
>
> Conclusions do not change as far as I can tell, but I would still go with
> that model. A LRT also shows that this model fits significantly better:
>
> res0 <- rma.mv(zf, vzf, mods=~sce_temporal*sce_spatial, random = list
> (~1|effectsizeID, ~1|studyID, ~1|speciesID), R=list(speciesID=corr), data=h)
> anova(res, res0)
>
> Best,
> Wolfgang
>
> >-----Original Message-----
> >From: Rafael Rios Moura [mailto:biorafaelrm using gmail.com]
> >Sent: Tuesday, 06 October, 2020 20:25
> >To: r-sig-meta-analysis using r-project.org; Viechtbauer, Wolfgang (SP)
> >Subject: Fwd: Overlapping CIs with significant difference among subgroups
> >
> >ATTACHMENT(S) REMOVED: dataset.csv | pruned_super-tree.tre | script.R
> >
> >Dear Wolfgang and All,
> >
> >Few months ago, I sent this email about a result obtained from a mixed
> >effects MLMA, controlling for phylogenetic non-independence. I tested the
> >difference between two levels of a moderator and obtained two close means
> >(0.39 and 0.31) with highly overlapping CIs. However, the omnibus test
> >detected a difference between estimates. Could it be a problem with my
> code
> >or the test? Or am I not using the "predict" function correctly? My
> dataset
> >and script are attached. I am grateful for contributions.
> >
> >Best wishes,
> >_______________________________________________________
> >
> >Prof. Dr. Rafael Rios Moura
> >scientia amabilis
> >Coordenador de Pesquisa e do NEPEE/CNPq
> >UEMG - Unidade Ituiutaba
> >
> >ORCID: http://orcid.org/0000-0002-7911-4734
> >Currículo Lattes: http://lattes.cnpq.br/4264357546465157
> >Research Gate: https://www.researchgate.net/profile/Rafael_Rios_Moura2
> >Rios de Ciência: https://www.youtube.com/channel/UCu2186wIJKji22ai8tvlUfg
> >
> >---------- Forwarded message ---------
> >De: Rafael Rios <biorafaelrm using gmail.com>
> >Date: seg., 1 de jun. de 2020 às 16:53
> >Subject: Overlapping CIs with significant difference among subgroups
> >To: <r-sig-meta-analysis using r-project.org>, Viechtbauer Wolfgang (SP)
> ><wolfgang.viechtbauer using maastrichtuniversity.nl>
> >
> >Dear Wolfgang and All,
> >
> >I conducted a multilevel mixed-effects meta-analysis and found differences
> >between levels of two moderators. I was expecting to find non-overlapped
> >confidence intervals. However, I obtained overlapped confidence intervals
> >for all subgroups. How can I interpret these results? In such situation,
> >should I trust in the Q-test or in the CIs? I controlled for phylogenetic
> >non-independence. Is there a chance of this approach affect the estimation
> >of CIs using predict function? My dataset and script are attached.
> >
> >Best wishes,
> >_______________________________________________________
> >
> >Prof. Dr. Rafael Rios Moura
> >Coordenador de Pesquisa e do NEPEE/CNPq
> >Laboratório de Ecologia e Zoologia (LEZ)
> >UEMG - Unidade Ituiutaba
> >
> >ORCID: http://orcid.org/0000-0002-7911-4734
> >Currículo Lattes: http://lattes.cnpq.br/4264357546465157
> >Research Gate: https://www.researchgate.net/profile/Rafael_Rios_Moura2
> >Rios de Ciência: https://www.youtube.com/channel/UCu2186wIJKji22ai8tvlUfg
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://stat.ethz.ch/pipermail/r-sig-meta-analysis/attachments/20201011/73f26234/attachment-0001.html>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: pruned_super-tree.tre
Type: application/octet-stream
Size: 15520 bytes
Desc: not available
URL: <https://stat.ethz.ch/pipermail/r-sig-meta-analysis/attachments/20201011/73f26234/attachment-0002.obj>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: dataset.csv
Type: text/csv
Size: 185128 bytes
Desc: not available
URL: <https://stat.ethz.ch/pipermail/r-sig-meta-analysis/attachments/20201011/73f26234/attachment-0001.csv>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: script_test.R
Type: application/octet-stream
Size: 2004 bytes
Desc: not available
URL: <https://stat.ethz.ch/pipermail/r-sig-meta-analysis/attachments/20201011/73f26234/attachment-0003.obj>