[R-meta] Meta-analysis when sampling covariance matrices are missing

[Philipp Doebler]

Dear Wolfgang and Célia,

it is an interesting thought to see this as an errors in variables problem.
Your intuition that there is a negative bias can probably be made more
precise: From the literature on attenutation of variances, covariances and
regression coefficients for insufficient reliability, I would assume the
absolute value is biased towards zero, i.e. a negative bias in the absolute
value of the regression coefficient. In this sense, I would agree with
Wolfgang: If one manages to find a significant (but attenuated)
relationship, then there is one.

I would conjecture that the factor should be similar to Var(d)/(Var(d) +
\bar nu^2)), where \bar nu^2 is the average squared standard error, or
around the square-root of this for a standardized regression coefficient.
Caution that people show results in this case assuming that the variance is
homogeneous, but it would surprise me if results differed dramatically in
the heterogeneous case.

Kind wishes,
  Philipp



On Tue, Jan 16, 2018 at 6:18 PM, Viechtbauer Wolfgang (SP) <
wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:

> Dear Célia,
>
> If I understand you correctly, the general question here is how to analyze
> the relationship between two effects measured on the same subjects. So, for
> each study, we have [y_i1, y_i2] and corresponding sampling variances
> [v_i1, v_i2]. Ideally, we also have the covariance between [y_i1, y_i2], so
> we have the 2x2 var-cov matrix of the sampling errors for each study. In
> that case, one can fit a multivariate (or rather: bivariate) model along
> the lines of Berkey et al. (1998):
>
> http://www.metafor-project.org/doku.php/analyses:berkey1998
>
> And then, based on the var-cov matrix of the random effects, we can
> estimate the correlation of the underlying true effects (the correlation is
> given directly in the output). One can even estimate the regression line
> that describes the linear relationship between the underlying true effects.
> See, for example:
>
> van Houwelingen, H. C., Arends, L. R., & Stijnen, T. (2002). Advanced
> methods in meta-analysis: Multivariate approach and meta-regression.
> Statistics in Medicine, 21(4), 589-624.
>
> Page 601 is the most relevant here.
>
> If you do not know the covariances (and one would assume them to be 0),
> then this approach is going to give you biased results, so I would not
> recommended this (and cluster robust methods are not going to help you
> here).
>
> An easier approach would be to simply treat one of the effects as your
> outcome and the other as a predictor. Technically, this isn't quite right,
> since the predictor is measured with error. This will lead to bias of the
> underlying true relationship (e.g., https://en.wikipedia.org/wiki/
> Errors-in-variables_models). Things are a bit more complex compared to
> the 'standard' regression context, since the amount of error actually
> varies across studies, but it's the same fundamental issue. I haven't given
> this a lot of thought, but I would assume that the bias will again be
> negative, so the strength of the relationship will tend to be
> underestimated. Unless somebody has a better idea, one could argue that if
> a relationship is still found, then this provides evidence that a
> relationship does exist, although the actual strength is uncertain.
>
> Best,
> Wolfgang
>
> -----Original Message-----
> From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-
> bounces at r-project.org] On Behalf Of Célia Sofia Moreira
> Sent: Wednesday, 10 January, 2018 1:04
> To: r-sig-meta-analysis at r-project.org
> Subject: Re: [R-meta] Meta-analysis when sampling covariance matrices are
> missing
>
> Dear all,
>
> I have been reading some of the messages in the list related to my problem,
> and I realized that “unknown correlations” is an old topic. Special thanks
> to Prof. Wolfgang, James Pustejovsky, and Isabel Schlegel in
> https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2017-August/000127.html
> .
> Really helpful!
>
> I also understood that my attempt to perform the multilevel model in my
> previous message was wrong. Please, forget the previous questions.
>
> My data are multivariate and so outcomes should be analyzed as such.
> However, I only have sampling means and SD, and thus, only variances of the
> effects (SMD) are available. So, I will input a covariance matrix
> (clubSandwich) and/or RVE (robumeta), always with the indispensable rma.mv
> (metafor).
>
> Nevertheless, I would like to investigate a regression between two effects.
> However, due to the previous limitation, I have no idea if it is possible,
> and, in affirmative case, how to do it. Thus, any recommendations /
> suggestions will be much appreciated. In negative case, can the
> unstructured correlation matrix obtained with the rma.mv output be used to
> assess the strength of the relationship between these effects?
>
> Please, tell me your opinion!
>
> Kind regards
> _______________________________________________
> R-sig-meta-analysis mailing list
> R-sig-meta-analysis at r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>



-- 
Prof. Dr. Philipp Doebler
Technische Universität Dortmund
Fakultät Statistik
Vogelpothsweg 87
44227 Dortmund

Tel.: +49 231-755 8259
Fax: +49 231-755 3918
doebler at statistik.tu-dortmund.de
www.statistik.tu-dortmund.de/1261.html


Wichtiger Hinweis: Die Information in dieser E-Mail ist vertraulich. Sie
ist ausschließlich für den Adressaten bestimmt. Sollten Sie nicht der
für diese E-Mail bestimmte Adressat sein, unterrichten Sie bitte den
Absender und vernichten Sie diese Mail. Vielen Dank.
Unbeschadet der Korrespondenz per E-Mail, sind unsere Erklärungen
ausschließlich final rechtsverbindlich, wenn sie in herkömmlicher
Schriftform (mit eigenhändiger Unterschrift) oder durch Übermittlung
eines solchen Schriftstücks per Telefax erfolgen.

Important note: The information included in this e-mail is confidential.
It is solely intended for the recipient. If you are not the intended
recipient of this e-mail please contact the sender and delete this
message. Thank you.
Without prejudice of e-mail correspondence, our statements are only
legally binding when they are made in the conventional written form
(with personal signature) or when such documents are sent by fax.

	[[alternative HTML version deleted]]