[R-meta] extracting variances
Gram, Gil (IITA)
G@Gr@m @end|ng |rom cg|@r@org
Tue Aug 18 10:58:57 CEST 2020
Dear Wolfgang,
As always, thanks for your reply.
Even in effect size units, the response variance 0.00775 is extremely small.
Otherwise last questions,
1. does the approach seem correct to you? (extracting the vcov matrix H in order to compute the Control, MR, OR, and ORMR variance responses over time)
2. where can I find the residual variance in the output below?
Multivariate Meta-Analysis Model (k = 2943; method: REML)
logLik Deviance AIC BIC AICc
-342.3623 684.7246 752.7246 956.0338 753.5493
Variance Components:
estim sqrt nlvls fixed factor
sigma^2.1 0.0924 0.3040 40 no ref
sigma^2.2 0.0251 0.1583 2943 no idRow
outer factor: idSite (nlvls = 71)
inner factor: treatment (nlvls = 4)
estim sqrt k.lvl fixed level
tau^2.1 0.1865 0.4319 275 no Control
tau^2.2 0.0922 0.3036 374 no MR
tau^2.3 0.1035 0.3217 1039 no OR
tau^2.4 0.0640 0.2530 1255 no ORMR
rho 0.8154 no
outer factor: idSite.time (nlvls = 271)
inner factor: treatment (nlvls = 4)
estim sqrt k.lvl fixed level
gamma^2.1 0.1072 0.3274 275 no Control
gamma^2.2 0.1429 0.3780 374 no MR
gamma^2.3 0.1129 0.3360 1039 no OR
gamma^2.4 0.1405 0.3748 1255 no ORMR
phi 0.9435 no
Test for Residual Heterogeneity:
QE(df = 2921) = 1068664.4274, p-val < .0001
Test of Moderators (coefficients 2:22):
QM(df = 21) = 1259.6581, p-val < .0001
Thanks and have a good day!
Gil
Message: 1
Date: Thu, 6 Aug 2020 12:07:57 +0000
From: "Viechtbauer, Wolfgang (SP)"
<wolfgang.viechtbauer using maastrichtuniversity.nl<mailto:wolfgang.viechtbauer using maastrichtuniversity.nl>>
To: "Gram, Gil (IITA)" <G.Gram using cgiar.org<mailto:G.Gram using cgiar.org>>,
"r-sig-meta-analysis using r-project.org<mailto:r-sig-meta-analysis using r-project.org>"
<r-sig-meta-analysis using r-project.org<mailto:r-sig-meta-analysis using r-project.org>>
Subject: Re: [R-meta] extracting variances
Message-ID: <da3972b28cf14339b934c848f40a7c30 using UM-MAIL3214.unimaas.nl<mailto:da3972b28cf14339b934c848f40a7c30 using UM-MAIL3214.unimaas.nl>>
Content-Type: text/plain; charset="utf-8"
Dear Gil,
You seem to interpret 0.00775 as 0.77% but the variances (or contrasts thereof) are not percentages. They are variances (in the units of whatever effect size / outcome measure you are using).
Best,
Wolfgang
-----Original Message-----
From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org]
On Behalf Of Gram, Gil (IITA)
Sent: Tuesday, 14 July, 2020 11:38
To: r-sig-meta-analysis using r-project.org<mailto:r-sig-meta-analysis using r-project.org>
Subject: [R-meta] extracting variances
Dear all,
I have a question regarding extracting the variances from my model.
Say I want to analyse the yields (tonnes per hectare) of 4 treatments
(control, OR, MR, ORMR) running across different sites and times. A
simplified version of my model would then be:
dat = escalc(measure="MN", mi=yield, sdi=sdYield, ni=nRep, data=temp)
dat$yi = sqrt(dat$yi) # sqrt transformation
dat$vi = dat$vi/(4*dat$yi) # variance adjustment to sqrt transformation
mod = rma.mv(yi, as.matrix(vi), method = 'REML', struct="HCS", sparse=TRUE,
data=dat,
mods = ~ rateORone + kgMN + I(rateORone^2) +
I(kgMN^2)
+ rateORone:kgMN + I(rateORone^2):I(kgMN^2) +
[…],
random = list(~1|ref, ~1|idRow, ~ treatment |
idSite, ~ treatment | idSite.time))
I’m interested in the yield variance responses over time, of OR and ORMR
versus control. So I extract the variance-covariance matrix H = mod$H:
Control MR OR ORMR
Control 0.1098190 0.1179042 0.1055471 0.1216751
MR 0.1179042 0.1360579 0.1174815 0.1354332
OR 0.1055471 0.1174815 0.1090329 0.1212389
ORMR 0.1216751 0.1354332 0.1212389 0.1449001
The variance responses I then calculate with e.g. responseOR = varianceOR +
varianceControl - 2*covar(OR, Control):
resOR
= (H['OR','OR'] + H['Control','Control'] - 2*H['Control','OR'])
= 0.1090329 + 0.1098190 - 2* 0.1055471
~ 0.00775
resORMR
~ 0.0114
I understand therefore that the variance responses over time for treatments
OR and ORMR are about 0.77% and 1.1%. These values are extremely small,
hence my questions:
- Am I correct that this was the correct way to estimate the yield
variability (responses) over time?
If this is all correct, then this means that there is hardly any variability
associated with these components. And one could start wondering what the
point is of even looking at this. I tried looking at the values of the other
components, and see whether these are larger.
- Keeping in mind the original data was sqrt transformed, can these values
still be considered as variances? or as standard deviations instead?
- If this makes up so little variance, then where is the variance coming
from? How much variability is associated with the error term? Or the other
components. Are these then magnitudes larger? How do I check if the sum of
all variance components equals 100% with the model output below?
I hope my questions are clear…
Thanks a lot in advance for your help,
Gil
------
Multivariate Meta-Analysis Model (k = 1161; method: REML)
Variance Components:
estim sqrt nlvls fixed factor
sigma^2.1 0.0604 0.2458 40 no ref
sigma^2.2 0.0285 0.1688 1161 no idRow
outer factor: idSite (nlvls = 71)
inner factor: treatment (nlvls = 4)
estim sqrt k.lvl fixed level
tau^2.1 0.1285 0.3584 275 no Control
tau^2.2 0.0952 0.3086 374 no MR
tau^2.3 0.1217 0.3488 234 no OR
tau^2.4 0.0711 0.2666 278 no ORMR
rho 0.7172 no
outer factor: idSite.time (nlvls = 271)
inner factor: treatment (nlvls = 4)
estim sqrt k.lvl fixed level
gamma^2.1 0.1098 0.3314 275 no Control
gamma^2.2 0.1361 0.3689 374 no MR
gamma^2.3 0.1090 0.3302 234 no OR
gamma^2.4 0.1449 0.3807 278 no ORMR
phi 0.9646 no
Test for Residual Heterogeneity:
QE(df = 1151) = 501266.0717, p-val < .0001
Test of Moderators (coefficients 2:10):
QM(df = 9) = 441.0373, p-val < .0001
Model Results:
estimate se zval pval ci.lb
ci.ub
intrcpt 1.2855 0.0691 18.6010 <.0001 1.1501
1.4210 ***
rateORone 0.0059 0.0007 8.5224 <.0001 0.0045
0.0072 ***
kgMN 0.0096 0.0009 10.5108 <.0001 0.0078
0.0114 ***
I(rateORone^2) -0.0000 0.0000 -5.2103 <.0001 -0.0000 -
0.0000 ***
I(kgMN^2) -0.0000 0.0000 -6.6753 <.0001 -0.0000 -
0.0000 ***
[…]
rateORone:kgMN -0.0000 0.0000 -3.7035 0.0002 -0.0000 -
0.0000 ***
I(rateORone^2):I(kgMN^2) 0.0000 0.0000 2.5775 0.0100 0.0000
0.0000 **
------------------------------
Message: 2
Date: Thu, 6 Aug 2020 12:22:49 +0000
From: "Viechtbauer, Wolfgang (SP)"
<wolfgang.viechtbauer using maastrichtuniversity.nl<mailto:wolfgang.viechtbauer using maastrichtuniversity.nl>>
To: Thao Tran <thaobrawn using gmail.com<mailto:thaobrawn using gmail.com>>,
"r-sig-meta-analysis using r-project.org<mailto:r-sig-meta-analysis using r-project.org>"
<r-sig-meta-analysis using r-project.org<mailto:r-sig-meta-analysis using r-project.org>>
Subject: Re: [R-meta] Correction for sample overlap in a meta-analysis
of prevalence
Message-ID: <4e330b7a5b1a4fddb65030b4f938638c using UM-MAIL3214.unimaas.nl<mailto:4e330b7a5b1a4fddb65030b4f938638c using UM-MAIL3214.unimaas.nl>>
Content-Type: text/plain; charset="utf-8"
Dear Thao,
I do not know these papers, so I cannot comment on what methods they describe and whether those could be implemented using metafor.
Obviously, the degree of dependence between overlapping estimates depends on the degree of overlap. Say there are two diseases (as in your example). Then if we had the raw data, we could count the number of individuals that:
x1: have only disease 1
x2: have only disease 2
x12: have both disease 1 and 2
x0: have neither disease
Let n = x1 + x2 + x12 + x0. Then you have p1 = (x1+x12) / n and p2 = (x2+x12) / n as the two prevalences. One could easily work out the covariance (I am too lazy to do that right now), but in the end this won't help, because computing this will require knowing all the x's, not just p1 and p2 and n. And I assume no information is reported on the degree of overlap. One could maybe make some reasonable 'guestimates' and then compute the covariances followed by a sensitivity analysis.
Alternatively, you could use the 'sandwich' method (cluster-robust inference). This has been discussed on this mailing list extensively in the past (not in the context of overlap in such estimates, but the principle is all the same).
Best,
Wolfgang
-----Original Message-----
From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org]
On Behalf Of Thao Tran
Sent: Tuesday, 04 August, 2020 15:26
To: r-sig-meta-analysis using r-project.org<mailto:r-sig-meta-analysis using r-project.org>
Subject: [R-meta] Correction for sample overlap in a meta-analysis of
prevalence
Dear all,
I want to conduct a meta-analysis of around 30 studies (from a systematic
review).
Some background of the studies: The quantity of interest is the prevalence
of RSV infection. Different studies reported RSV prevalence for different
risk groups. Since, it is quite often that some people might suffer from
multiple comorbidities (for example, an individual might have both cardiac
disease and lung disease), and it was not stated clearly in the reported
data if these two sub-populations (cardia disease patients, and lung
disease patients) are mutually exclusive. In the end, I want to have an
overall estimate across all risk groups. Given the fact stated above, it is
likely that some of the data (from two or more risk groups) might share a
proportion of the population. For example, John's study reported data on
cardiac disease as well as lung disease. These two risk groups were
included in the meta-analysis. However, we need to take into account the
fact that, the two sub-populations might share some proportions of
participants.
I was searching on the internet methods to account for the overlap samples
while conducting meta-analysis. There are two papers that address this
problem:
1. https://academic.oup.com/bioinformatics/article/33/24/3947/3980249 The
authors proposed FOLD, a method to optimize power in a meta-analysis of
genetic associations studies with overlapping subjects.
2.
http://www.stiftung.at/wp-content/uploads/2015/04/BomPaper_Oct_2014.pdf
In
this paper, the author compared generalized weights and inverse-variance
weights meta-estimates to account for overlap sample.
My question is:
Are these approaches incorporated into the *metafor* package?
Thanks for your input.
Best,
Thao
--
*Trần Mai Phương Thảo*
Master Student - Master of Statistics
Hasselt University - Belgium.
Email: Thaobrawn using gmail.com<mailto:Thaobrawn using gmail.com> / maiphuongthao.tran using student.uhasselt.be<mailto:maiphuongthao.tran using student.uhasselt.be>
Phone number: + 84 979 397 410+ 84 979 397 410 / 0032 488 0358430032 488
035843
------------------------------
Message: 3
Date: Thu, 6 Aug 2020 12:29:48 +0000
From: "Viechtbauer, Wolfgang (SP)"
<wolfgang.viechtbauer using maastrichtuniversity.nl<mailto:wolfgang.viechtbauer using maastrichtuniversity.nl>>
To: Tzlil Shushan <tzlil21092 using gmail.com<mailto:tzlil21092 using gmail.com>>,
"r-sig-meta-analysis using r-project.org<mailto:r-sig-meta-analysis using r-project.org>"
<r-sig-meta-analysis using r-project.org<mailto:r-sig-meta-analysis using r-project.org>>
Subject: Re: [R-meta] Performing a multilevel meta-analysis
Message-ID: <024f9bc096534c129decb63836a59f1f using UM-MAIL3214.unimaas.nl<mailto:024f9bc096534c129decb63836a59f1f using UM-MAIL3214.unimaas.nl>>
Content-Type: text/plain; charset="utf-8"
Dear Tzlil,
Unless you have good reasons to do so, do not use custom weights. rma.mv() uses weights and the default ones are usually fine.
weights(res, type="rowsum") will only (currently) work in the 'devel' version of metafor, which you can install as described here:
https://wviechtb.github.io/metafor/#installation
I can't really comment on the second question, because answering this would require knowing all details of what is being computed/reported.
As for the last question ("is there a straightforward way in metafor to specify the analysis with Chi-square values"): No, chi-square values are test statistics, not an effect size / outcome measure, so they cannot be used for a meta-analysis (at least not with metafor).
Best,
Wolfgang
-----Original Message-----
From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org]
On Behalf Of Tzlil Shushan
Sent: Wednesday, 05 August, 2020 5:45
To: r-sig-meta-analysis using r-project.org
Subject: [R-meta] Performing a multilevel meta-analysis
Hi R legends!
My name is Tzlil and I'm a PhD candidate in Sport Science - Human
performance science and sports analytics
I'm currently working on a multilevel meta-analysis using the metafor
package.
My first question is around the methods used to assign weights within rma.mv
models.
I'd like to know if there is a conventional or 'most conservative' approach
to continue with. Since I haven't found a consistent methodology within the
multilevel meta-analyses papers I read, I originally applied a weight which
pertains to variance (vi) and number of effect sizes from the same study. I
found this method in a lecture by Joshua R. Polanin
https://www.youtube.com/watch?v=rJjeRRf23L8&t=1719s from 28:00.
W = 1/vi, then divided by the number of ES for a study
for example, a study with vi = 0.0402 and 2 different ES will weight as
follow;
1/0.0402 = 24.88, then 24.88/2 = 12.44 (finally, converting into
percentages based on the overall weights in the analysis)
After I've read some of the great posts provided in last threads here such
as;
http://www.metafor-project.org/doku.php/tips:weights_in_rma.mv_models and
https://www.jepusto.com/weighting-in-multivariate-meta-analysis/
I wonder if it is not correct and I need to modify the way I use weights in
my model..
For some reason, I tried to imitate the approach used in the first link
above. However, for some reason I get an error every time I tried to
specify weights(res, type="rowsum") *Error in match.arg(type, c("diagonal",
"matrix")) : 'arg' should be one of “diagonal”, “matrix”*
My second question is related to the way I meta-analyse a specific ES. My
meta-analysis involves the reliability and convergent validity of heart
rate during a specific task, which is measured in relative values (i.e.
percentages). Therefore, my meta-analysis includes four different ESs
parameters (mean difference; MD, interclass correlation; ICC, standard
error of measurement; SEM, and correlation coefficient; r).
I wonder how I need to use SEM before starting the analysis. I've seen some
papers which squared and log transformed the SEM before performing a
meta-analysis, while others converted the SEM into CV%. Due to the original
scale of our ES (which is already in percentages) I'd like to perform the
analysis without converting it into CV% values. Should I use the SEM as the
reported values? only log transformed it? Further, is there a
straightforward way in metafor to specify the analysis with Chi-square
values (as "ZCOR" in correlations)?
Thanks in advance!
Kind regards,
Tzlil Shushan | Sport Scientist, Physical Preparation Coach
BEd Physical Education and Exercise Science
MSc Exercise Science - High Performance Sports: Strength &
Conditioning, CSCS
PhD Candidate Human Performance Science & Sports Analytics
------------------------------
Subject: Digest Footer
_______________________________________________
R-sig-meta-analysis mailing list
R-sig-meta-analysis using r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
------------------------------
End of R-sig-meta-analysis Digest, Vol 39, Issue 2
**************************************************
[[alternative HTML version deleted]]
More information about the R-sig-meta-analysis
mailing list