[R-meta] Meta-analysis when sampling covariance matrices are missing

[Célia Sofia Moreira]

Dear Prof. Wolfgang and Prof. Doebler,

Thank you very much for your valuable recommendations to my problem.
However, given some complexities in 'my' outcomes, it would be difficult to
apply the "errors-in-variables" model. Moreover, without a reliable
published result regarding the bias and the significance of the
relationship, it would be difficult to publish my results. So, I will try a
different approach, which I will explain in another message.

Once more, thank you very much for your attention and your interesting
comments. I think you have a theorem to prove about your joint conjecture
on the bias and the significance of the relationship in the
errors-in-variables model!

Kind regards,
 celia

2018-01-16 19:09 GMT+00:00 Philipp Doebler <doebler at statistik.tu-dortmund.de
>:

> Dear Wolfgang and Célia,
>
> it is an interesting thought to see this as an errors in variables
> problem. Your intuition that there is a negative bias can probably be made
> more precise: From the literature on attenutation of variances, covariances
> and regression coefficients for insufficient reliability, I would assume
> the absolute value is biased towards zero, i.e. a negative bias in the
> absolute value of the regression coefficient. In this sense, I would agree
> with Wolfgang: If one manages to find a significant (but attenuated)
> relationship, then there is one.
>
> I would conjecture that the factor should be similar to Var(d)/(Var(d) +
> \bar nu^2)), where \bar nu^2 is the average squared standard error, or
> around the square-root of this for a standardized regression coefficient.
> Caution that people show results in this case assuming that the variance is
> homogeneous, but it would surprise me if results differed dramatically in
> the heterogeneous case.
>
> Kind wishes,
>   Philipp
>
>
>
> On Tue, Jan 16, 2018 at 6:18 PM, Viechtbauer Wolfgang (SP) <
> wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:
>
>> Dear Célia,
>>
>> If I understand you correctly, the general question here is how to
>> analyze the relationship between two effects measured on the same subjects.
>> So, for each study, we have [y_i1, y_i2] and corresponding sampling
>> variances [v_i1, v_i2]. Ideally, we also have the covariance between [y_i1,
>> y_i2], so we have the 2x2 var-cov matrix of the sampling errors for each
>> study. In that case, one can fit a multivariate (or rather: bivariate)
>> model along the lines of Berkey et al. (1998):
>>
>> http://www.metafor-project.org/doku.php/analyses:berkey1998
>>
>> And then, based on the var-cov matrix of the random effects, we can
>> estimate the correlation of the underlying true effects (the correlation is
>> given directly in the output). One can even estimate the regression line
>> that describes the linear relationship between the underlying true effects.
>> See, for example:
>>
>> van Houwelingen, H. C., Arends, L. R., & Stijnen, T. (2002). Advanced
>> methods in meta-analysis: Multivariate approach and meta-regression.
>> Statistics in Medicine, 21(4), 589-624.
>>
>> Page 601 is the most relevant here.
>>
>> If you do not know the covariances (and one would assume them to be 0),
>> then this approach is going to give you biased results, so I would not
>> recommended this (and cluster robust methods are not going to help you
>> here).
>>
>> An easier approach would be to simply treat one of the effects as your
>> outcome and the other as a predictor. Technically, this isn't quite right,
>> since the predictor is measured with error. This will lead to bias of the
>> underlying true relationship (e.g., https://en.wikipedia.org/wiki/
>> Errors-in-variables_models). Things are a bit more complex compared to
>> the 'standard' regression context, since the amount of error actually
>> varies across studies, but it's the same fundamental issue. I haven't given
>> this a lot of thought, but I would assume that the bias will again be
>> negative, so the strength of the relationship will tend to be
>> underestimated. Unless somebody has a better idea, one could argue that if
>> a relationship is still found, then this provides evidence that a
>> relationship does exist, although the actual strength is uncertain.
>>
>> Best,
>> Wolfgang
>>
>> -----Original Message-----
>> From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bo
>> unces at r-project.org] On Behalf Of Célia Sofia Moreira
>> Sent: Wednesday, 10 January, 2018 1:04
>> To: r-sig-meta-analysis at r-project.org
>> Subject: Re: [R-meta] Meta-analysis when sampling covariance matrices are
>> missing
>>
>> Dear all,
>>
>> I have been reading some of the messages in the list related to my
>> problem,
>> and I realized that “unknown correlations” is an old topic. Special thanks
>> to Prof. Wolfgang, James Pustejovsky, and Isabel Schlegel in
>> https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2017-
>> August/000127.html.
>> Really helpful!
>>
>> I also understood that my attempt to perform the multilevel model in my
>> previous message was wrong. Please, forget the previous questions.
>>
>> My data are multivariate and so outcomes should be analyzed as such.
>> However, I only have sampling means and SD, and thus, only variances of
>> the
>> effects (SMD) are available. So, I will input a covariance matrix
>> (clubSandwich) and/or RVE (robumeta), always with the indispensable
>> rma.mv
>> (metafor).
>>
>> Nevertheless, I would like to investigate a regression between two
>> effects.
>> However, due to the previous limitation, I have no idea if it is possible,
>> and, in affirmative case, how to do it. Thus, any recommendations /
>> suggestions will be much appreciated. In negative case, can the
>> unstructured correlation matrix obtained with the rma.mv output be used
>> to
>> assess the strength of the relationship between these effects?
>>
>> Please, tell me your opinion!
>>
>> Kind regards
>> _______________________________________________
>> R-sig-meta-analysis mailing list
>> R-sig-meta-analysis at r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>>
>
>
>
> --
> Prof. Dr. Philipp Doebler
> Technische Universität Dortmund
> Fakultät Statistik
> Vogelpothsweg 87
> <https://maps.google.com/?q=Vogelpothsweg+87+44227+Dortmund&entry=gmail&source=g>
> 44227 Dortmund
> <https://maps.google.com/?q=Vogelpothsweg+87+44227+Dortmund&entry=gmail&source=g>
>
> Tel.: +49 231-755 8259 <+49%20231%207558259>
> Fax: +49 231-755 3918 <+49%20231%207553918>
> doebler at statistik.tu-dortmund.de
> www.statistik.tu-dortmund.de/1261.html
>
>
> Wichtiger Hinweis: Die Information in dieser E-Mail ist vertraulich. Sie
> ist ausschließlich für den Adressaten bestimmt. Sollten Sie nicht der
> für diese E-Mail bestimmte Adressat sein, unterrichten Sie bitte den
> Absender und vernichten Sie diese Mail. Vielen Dank.
> Unbeschadet der Korrespondenz per E-Mail, sind unsere Erklärungen
> ausschließlich final rechtsverbindlich, wenn sie in herkömmlicher
> Schriftform (mit eigenhändiger Unterschrift) oder durch Übermittlung
> eines solchen Schriftstücks per Telefax erfolgen.
>
> Important note: The information included in this e-mail is confidential.
> It is solely intended for the recipient. If you are not the intended
> recipient of this e-mail please contact the sender and delete this
> message. Thank you.
> Without prejudice of e-mail correspondence, our statements are only
> legally binding when they are made in the conventional written form
> (with personal signature) or when such documents are sent by fax.
>

	[[alternative HTML version deleted]]