<div dir="ltr">Dear James,<div>Thank you!</div><div><br></div><div>Attached are the code and the database.</div><div>And here is some results </div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div><font color="#9900ff">> summary(based_inf)</font></div><div><font color="#9900ff">Multivariate Meta-Analysis Model (k = 17; method: REML)</font></div><div><font color="#9900ff"> logLik Deviance AIC BIC AICc </font></div><div><font color="#9900ff">-14.2991 28.5983 34.5983 36.9161 36.5983 </font></div><div><font color="#9900ff"><br></font></div><div><font color="#9900ff">Variance Components:</font></div><div><font color="#9900ff"> estim sqrt nlvls fixed factor </font></div><div><font color="#9900ff">sigma^2.1 0.0000 0.0000 12 no ID_study </font></div><div><font color="#9900ff">sigma^2.2 4.1629 2.0403 5 no ID_database </font></div><div><font color="#9900ff"><br></font></div><div><font color="#9900ff">Test for Heterogeneity:</font></div><div><font color="#9900ff">Q(df = 16) = 48.1411, p-val < .0001</font></div><div><font color="#9900ff"><br></font></div><div><font color="#9900ff">Model Results:</font></div><div><font color="#9900ff">estimate se tval pval <a href="http://ci.lb" target="_blank">ci.lb</a> ci.ub </font></div><div><font color="#9900ff"> 2.0905 0.9704 2.1542 0.0468 0.0333 4.1478 * </font></div><div><font color="#9900ff"><br></font></div><div><font color="#9900ff"><br></font></div><div><font color="#9900ff">> coef_test(based_inf, vcov = "CR2", </font></div><div><font color="#9900ff">+ cluster = wb$ID_database)</font></div><div><font color="#9900ff"> Coef. Estimate SE t-stat d.f. p-val (Satt) Sig.</font></div><div><font color="#9900ff">1 intrcpt 2.09 0.954 2.19 3.91 0.0952 .</font></div></blockquote><div><font color="#9900ff"><br></font></div><div><font color="#000000">I think</font> I found out where our mistake was. </div><div><br></div><div>The sandwich correction doesn't calculate Conf Intervals, so we calculated them using formula: SE*1.96</div><div>Stupid, I know. </div><div>Still, even now I am not sure how to correctly calculate CI here - could you please explain?</div><div><br></div><div>And another question</div><div>There are several methods for outliers detection: Cook distance, residuals, hat values. Rather often a study is problematic with one method but OK with others. Are there any guidelines which studies should be removed -- i.e., when at least two methods indicate it as outliers?</div><div><br></div><div>Best,</div><div>Valeria</div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Dec 1, 2020 at 9:10 PM James Pustejovsky <<a href="mailto:jepusto@gmail.com" target="_blank">jepusto@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Valeria,<div><br><div>These are indeed perplexing results. Based on the information you've provided, it's hard to say what could be going on. Could you provide examples of the code you're using and the results of your analyses? Doing so will help to identify potential problems or coding errors. </div></div><div><br></div><div>Kind Regards,</div><div>James</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Dec 1, 2020 at 10:45 AM Valeria Ivaniushina <<a href="mailto:v.ivaniushina@gmail.com" target="_blank">v.ivaniushina@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Dear list members,<br>
<br>
We are conducting several meta-analyses using the metafor package in R<br>
(Viechtbauer 2010) because of 3-level data structure, followed by<br>
sandwich-type estimator with a small-sample adjustment to get cluster<br>
robust standard errors.<br>
<br>
There are some things that puzzle me, and I hope to get answers from the<br>
community.<br>
<br>
1. We calculate 95% CI for our mean effect size, and p-value is calculated<br>
as a part of the output. While CI always indicate significant mean effect<br>
size, p-values are often > 0.05<br>
- Should I report both CI and p-value?<br>
- How to interpret such discrepancy?<br>
<br>
2. When I draw a forest plot for a meta-analysis of 8 models, I can see<br>
that 95% CIs for every coefficient contain zero (for example, -0.40 -<br>
0.84). However, the 95% CI for the mean coefficient is well above zero<br>
(0,28 - 0,45). How is it possible?<br>
<br>
3. Theoretically, the data has a 3-level structure (model; article;<br>
database). But sometimes I see that there is no variance on one or two of<br>
the levels. Should I repeat the analysis with only 2 or 1 level, according<br>
to the variance distribution?<br>
<br>
Best, Valeria<br>
<br>
[[alternative HTML version deleted]]<br>
<br>
_______________________________________________<br>
R-sig-meta-analysis mailing list<br>
<a href="mailto:R-sig-meta-analysis@r-project.org" target="_blank">R-sig-meta-analysis@r-project.org</a><br>
<a href="https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis" rel="noreferrer" target="_blank">https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis</a><br>
</blockquote></div>
</blockquote></div>