[R-meta] Negative r value for effect sizes within studies

[Stefanou Revesz]

Dear Wolfgang, Reza and James,

Thank you so much, I think I got really great, complementary advice.
I'm convinced that I should reverse the sign of effect sizes on one of
my outcomes to avoid all the complexities that you all brought up
(thank you for the last example).

Stefanou


On Fri, Sep 3, 2021 at 12:59 PM Reza Norouzian <rnorouzian using gmail.com> wrote:
>
> I should add that it is entirely "possible" to encounter outcomes that
> are measured in opposite directions across studies and that can be
> meaningfully part of the same multivariate meta-analysis. But like I
> mentioned, syncing the outcomes' directions often resolves the issue.
>
> Here is a substantive example.
>
> Studies that research the efficacy of teacher's feedback on students'
> (English Language Learners) writing competence often focus on various
> related linguistic features (e.g., verbs, articles etc.). However, due
> to the debates over how these linguistic features ought to be
> measured, these studies tend to measure some linguistic features "by
> tradition" using the number of errors, but others using the number of
> correct usage (I'm simplifying the actual measures to make this
> discussion more general, but the directions of outcomes run opposite
> to one another for different linguistic features).
>
> The fact that these outcome levels are related is of great interest,
> hence motivating a multivariate meta-analysis. On the other hand,
> almost always these studies are multi-group to compare different types
> of feedback and longitudinal to measure the durability of feedback's
> effect on developing students' writing competence (to make matters
> even more complex, they even use more than one comparison/control
> group).
>
> In a situation like what I described above (and more generally as
> described in my previous email), if not synced, these opposing
> outcomes could create a difficult situation where each study could
> have a combination of positively and negatively correlated estimates
> of effect size in it.
>
> So, synching the outcome directions, at least for me, is inevitable.
>
> Reza
>
> On Fri, Sep 3, 2021 at 12:19 PM James Pustejovsky <jepusto using gmail.com> wrote:
> >
> > To add to Wolfgang's comments, yes it is possible in principle. However, on
> > a practical level, it seems like it would be a very odd situation where you
> > would include two outcomes in the same meta-analysis that are *negatively*
> > correlated with each other. Say that one intervention study reports a
> > negative effect size estimate for depression and a positive effect size for
> > positive affect, two outcomes which are negatively correlated. It would not
> > make much practical sense to average the negative ES for depression and the
> > positive ES for positive affect together, which suggests that it would be
> > very odd to include both in a single meta-analysis.
> >
> > On Fri, Sep 3, 2021 at 12:09 PM Viechtbauer, Wolfgang (SP) <
> > wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
> >
> > > Dear Stefanou,
> > >
> > > Would be unusual, but in principle, yes, it can. However, only certain
> > > values of r are actually possible. Let's say there is a study with 5
> > > estimates and you assume r = -0.6. Then the correlation matrix for this
> > > study would be:
> > >
> > > v <- matrix(-.6, 5, 5); diag(v) <- 1
> > >
> > > which is not positive definite, which we can see by examining its
> > > eigenvalues:
> > >
> > > round(eigen(v)$values, 4)
> > >
> > > (note the last eigenvalue is negative). The largest negative correlation
> > > possible when p=5 (to get at least a positive semi-definite matrix) is
> > > -0.25:
> > >
> > > v <- matrix(-.25, 5, 5); diag(v) <- 1
> > > round(eigen(v)$values, 4)
> > >
> > > (the last eigenvalue is now essentially 0).
> > >
> > > More generally, the lower bound is:
> > >
> > > -1/(p-1)
> > >
> > > So, you need to determine which study has the largest number of estimates
> > > and based on that, you know what the lower bound for r is (at least, when
> > > we assume that the correlation matrix is compound symmetric - as above).
> > >
> > > Best,
> > > Wolfgang
> > >
> > > >-----Original Message-----
> > > >From: R-sig-meta-analysis [mailto:
> > > r-sig-meta-analysis-bounces using r-project.org] On
> > > >Behalf Of Stefanou Revesz
> > > >Sent: Friday, 03 September, 2021 18:47
> > > >To: R meta
> > > >Subject: [R-meta] Negative r value for effect sizes within studies
> > > >
> > > >Dear All,
> > > >
> > > >This may be too basic to ask. But can the r value chosen to represent
> > > >the common correlation among effect sizes within studies be a negative
> > > >one?
> > > >
> > > >For example, is there a situation where I would need to use the following?
> > > >
> > > >impute_covariance_matrix(..., r = -.6)
> > > >
> > > >Thank you,
> > > >Stefanou
> > >
> > > _______________________________________________
> > > R-sig-meta-analysis mailing list
> > > R-sig-meta-analysis using r-project.org
> > > https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
> > >
> >
> >         [[alternative HTML version deleted]]
> >
> > _______________________________________________
> > R-sig-meta-analysis mailing list
> > R-sig-meta-analysis using r-project.org
> > https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>
> _______________________________________________
> R-sig-meta-analysis mailing list
> R-sig-meta-analysis using r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis