[R-meta] specification of phi in vcalc
Danielle Hiam
d@n|e||e@h|@m @end|ng |rom de@k|n@edu@@u
Sun Aug 14 02:21:17 CEST 2022
Hi James,
Very happy to share the data. To give you a brief overview, I am trying to run a correlated and hierarchical meta-analysis to investigate changes in microRNA expression after a acute bout of exercise. Each of the primary studies have differing time points so I have picked the most “common” timepoints (PRE, POST, 1-2hour post exercise and 24 hours post exercise) and dummy coded them as follows -1 (PRE), 0(POST), 1(within 1-2HP) and 2 (24HP). I have chosen to use the fold change from PRE as values as these would then be standard measurement across all the studies.
* How many studies you've got:
I am doing multiple meta-analysis of several microRNAs. To give you an idea miRNA-1 has 14 studies, miR-133a 16 studies etc...
* How many effect sizes per study (on average and the range of effect sizes per study; quartiles would be great too): miRNA-1: mean: 2.30 (total 46obs). miR-133a= mean 2.22 (total 51obs)
Average, median and IQR for miR-133a
Time mean MEDIAN IQR
1 -1 1 1 0
2 0 2.23 1.2 1.71
3 1 1.54 1.49 1.08
4 2 1.42 1.27 1.06
* Your model specification, including both what moderators are included and how you've specified the random effects:
This is my code:
V <- vcalc(FC_SD, cluster=cohort.in.study,
time1=Time_NUM, data = dat2, phi=0.6)
rma.mv(yi = FC_MEAN,
V = V,
data = dat2,
mods = ~ factor(Time_NUM),
random = list (~ Time_NUM|cohort.in.study),
struct = "CAR")
As the timepoints are not evenly spaced but are dependent (repeated measures) I ran CAR for the struct argument.
Cohort.in.study is a unique ID to indicate that the timepoints within the cohort are dependent while timepoints outside this cohort are independent.
* For any included moderators, whether they are study-level characteristics (constant across the effect sizes for a given study) or effect-level characteristics (that vary within study).
I have chosen timepoints that are common across the studies (PRE, POST, 1-2hour post exercise and 24 hours post exercise). For example I dummy coded them as follows -1 (PRE), 0(POST), 1(within 1-2HP) and 2 (24HP)
* If you're open to sharing, the results of fitting the model with different values of phi, to provide a sense of how much the estimates change.
No worries see below for the miRNA-133a results. You will see while estimates don’t change drastically, the results from phi 0.4, 0.5 and 0.6 are borderline significant and when I change the phi to be above 0.7 they become significant. The QE also changes but it remains significant for all values of phi.
Phi 0.4 estim sqrt fixed
tau^2 1.791 1.338 no
rho 0.000 no
Test for Residual Heterogeneity:
QE(df = 47) = 192.076, p-val < .001
Model Results:
estimate se¹ tval¹ df¹ pval¹ ci.lb¹ ci.ub¹ <U+200B>
intrcpt 0.983 0.040 24.415 12.72 <.001 0.896 1.070 ***
factor(Time_NUM)0 1.353 0.571 2.368 16.51 0.030 0.145 2.560 *
factor(Time_NUM)1 0.773 0.226 3.416 11.63 0.005 0.278 1.268 **
factor(Time_NUM)2 0.519 0.242 2.146 7.06 0.069 -0.052 1.089 .
Phi 0.5 estim sqrt fixed
tau^2 1.824 1.351 no
rho 0.000 no
Test for Residual Heterogeneity:
QE(df = 47) = 213.811, p<0.001
Model Results:
estimate se¹ tval¹ df¹ pval¹ ci.lb¹ ci.ub¹ <U+200B>
intrcpt 0.978 0.050 19.485 12.76 <.001 0.870 1.087 ***
factor(Time_NUM)0 1.361 0.580 2.348 16.45 0.032 0.135 2.587 *
factor(Time_NUM)1 0.800 0.235 3.409 11.56 0.005 0.286 1.313 **
factor(Time_NUM)2 0.541 0.240 2.257 7.05 0.058 -0.025 1.106 .
Phi 0.6 estim sqrt fixed
tau^2 1.865 1.366 no
rho 0.000 no
Test for Residual Heterogeneity:
QE(df = 47) = 249.228, p<0.001
Model Results:
estimate se¹ tval¹ df¹ pval¹ ci.lb¹ ci.ub¹ <U+200B>
intrcpt 0.974 0.060 16.213 12.82 <.001 0.844 1.104 ***
factor(Time_NUM)0 1.369 0.588 2.328 16.39 0.033 0.125 2.614 *
factor(Time_NUM)1 0.822 0.244 3.376 11.48 0.006 0.289 1.356 **
factor(Time_NUM)2 0.564 0.239 2.358 7.01 0.050 -0.001 1.130 .
Phi 0.8 estim sqrt fixed
tau^2 1.965 1.402 no
rho 0.000 no
Test for Residual Heterogeneity:
QE(df = 47) = 439.477, p<0.001
Model Results:
estimate se¹ tval¹ df¹ pval¹ ci.lb¹ ci.ub¹ <U+200B>
intrcpt 0.965 0.079 12.157 12.95 <.001 0.794 1.137 ***
factor(Time_NUM)0 1.387 0.606 2.291 16.27 0.036 0.105 2.670 *
factor(Time_NUM)1 0.857 0.262 3.265 11.24 0.007 0.281 1.433 **
factor(Time_NUM)2 0.612 0.246 2.493 6.90 0.042 0.030 1.194 *
-----Original Message-----
From: R-sig-meta-analysis <r-sig-meta-analysis-bounces using r-project.org> On Behalf Of James Pustejovsky
Sent: Thursday, 4 August 2022 1:12 PM
To: Lukasz Stasielowicz <lukasz.stasielowicz using uni-osnabrueck.de>
Cc: R meta <r-sig-meta-analysis using r-project.org>
Subject: Re: [R-meta] specification of phi in vcalc
Hi Danielle,
Just to add a little to Lukasz's suggestions (which I think are excellent), it is a little unusual and surprising that your results are very sensitive to changing the value of phi from 0.6 to 0.8. If you can provide a bit more detail about the structure of your data and model, I may be able to offer some suggestions on how to interpret this sensitivity. Specifically, it would be useful to know:
* How many studies you've got
* How many effect sizes per study (on average and the range of effect sizes per study; quartiles would be great too)
* Your model specification, including both what moderators are included and how you've specified the random effects
* For any included moderators, whether they are study-level characteristics (constant across the effect sizes for a given study) or effect-level characteristics (that vary within study).
* If you're open to sharing, the results of fitting the model with different values of phi, to provide a sense of how much the estimates change.
Incidentally, this issue is closely related to something I'm studying right
now: https://www.jepusto.com/talk/srsm-2022-matter-of-emphasis/
James
On Wed, Aug 3, 2022 at 10:14 AM Lukasz Stasielowicz < lukasz.stasielowicz using uni-osnabrueck.de> wrote:
> Dear Danielle,
>
> vcalc documentation contains a potentially helpful tip: "Argument phi
> must then also be specified to indicate the autocorrelation among the
> sampling errors of two effect sizes that differ by one unit on the
> time1 variable. As above, the autocorrelation of the measurements
> themselves can be used here as a proxy."
> https://rdrr.io/github/wviechtb/metafor/man/vcalc.html
>
> Perhaps some tables in primary studies show correlations between
> neighboring time points?
>
> Another option: If raw data are available for some primary studies,
> then one could estimate the correlation for several data sets and use
> it as "phi" input for vcalc. If different data sets lead to different
> values, then one could test different values in sensitivity analyses.
>
>
>
>
> Best,
> Lukasz
> --
> Lukasz Stasielowicz
> Osnabrück University
> Institute for Psychology
> Research methods, psychological assessment, and evaluation
> Seminarstraße 20
> 49074 Osnabrück (Germany)
>
> See also:
> On 03.08.2022 12:00, r-sig-meta-analysis-request using r-project.org wrote:
> > Send R-sig-meta-analysis mailing list submissions to
> > r-sig-meta-analysis using r-project.org
> >
> > To subscribe or unsubscribe via the World Wide Web, visit
> > https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
> > or, via email, send a message with subject or body 'help' to
> > r-sig-meta-analysis-request using r-project.org
> >
> > You can reach the person managing the list at
> > r-sig-meta-analysis-owner using r-project.org
> >
> > When replying, please edit your Subject line so it is more specific
> > than "Re: Contents of R-sig-meta-analysis digest..."
> >
> >
> > Today's Topics:
> >
> > 1. specification of phi in vcalc (Danielle Hiam)
> >
> > --------------------------------------------------------------------
> > --
> >
> > Message: 1
> > Date: Wed, 3 Aug 2022 06:54:48 +0000
> > From: Danielle Hiam <danielle.hiam using deakin.edu.au>
> > To: "r-sig-meta-analysis using r-project.org"
> > <r-sig-meta-analysis using r-project.org>
> > Subject: [R-meta] specification of phi in vcalc
> > Message-ID:
> > <
> ME3PR01MB54642B304670F1BF825E6F60B79C9 using ME3PR01MB5464.ausprd01.prod.out
> look.com
> >
> >
> > Content-Type: text/plain; charset="iso-8859-1"
> >
> > Hello,
> >
> > I am running a meta-analysis that has nested data and dependent
> > effect
> sizes (multiple timepoints). Therefore, I need to take into account
> effect sizes that are correlated and also cluster within studies. I
> will run vcalc, then the rma.mv function and then use robust with
> clubsandwich set to true to "correct" for any mis-specification of the
> model. However, I am looking for some clarification regarding how to
> "guess" the value for phi in the vcalc function, R code: V <-
> vcalc(FC_SD, cluster, time1=Time_NUM, data, phi=0.6).
> >
> > When I change phi from 0.6 to 0.8 in vcalc I get very different
> > results
> from the meta-analysis (rma.mv followed by robust(.... Clubsanwich=T).
> I have read James Pustejovsky paper (DOI:
> https://doi.org/10.1007/s11121-021-01246-3) on this, where he suggests
> that in situations where no information is available, the meta-analyst
> might pick a plausible value and then conduct sensitivity analysis
> across a range reasonable values. How would I go about this?
> >
> > Any information regarding specification of phi would be greatly
> appreciated
> >
> > Thanks,
> > Danielle
> >
> > Deakin University
> > Melbourne Burwood Campus, 221 Burwood Highway, Burwood VIC 3125
> > danielle.hiam using deakin.edu.au ipan.deakin.edu.au Deakin University
> > CRICOS Provider Code: 00113B
> >
> > Stay connected with IPAN
> > Facebook Twitter
> >
> > Important Notice: The contents of this email are intended solely for
> > the
> named addressee and are confidential; any unauthorised use,
> reproduction or storage of the contents is expressly prohibited.
> > If you have received this email in error, please delete it and any
> attachments immediately and advise the sender by return email or telephone.
> Deakin University does not warrant that this email and any attachments
> are error or virus free.
> >
> >
> >
> > Important Notice: The contents of this email are intended solely for
> > the
> named addressee and are confidential; any unauthorised use,
> reproduction or storage of the contents is expressly prohibited. If
> you have received this email in error, please delete it and any
> attachments immediately and advise the sender by return email or telephone.
> >
> > Deakin University does not warrant that this email and any
> > attachments
> are error or virus free.
> >
> >
> >
> >
> > ------------------------------
> >
> > Subject: Digest Footer
> >
> > _______________________________________________
> > R-sig-meta-analysis mailing list @ R-sig-meta-analysis using r-project.org
> > To manage your subscription to this mailing list, go to:
> > https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
> >
> >
> > ------------------------------
> >
> > End of R-sig-meta-analysis Digest, Vol 63, Issue 1
> > **************************************************
>
> _______________________________________________
> R-sig-meta-analysis mailing list @ R-sig-meta-analysis using r-project.org
> To manage your subscription to this mailing list, go to:
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>
[[alternative HTML version deleted]]
_______________________________________________
R-sig-meta-analysis mailing list @ R-sig-meta-analysis using r-project.org To manage your subscription to this mailing list, go to:
https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
Important Notice: The contents of this email are intended solely for the named addressee and are confidential; any unauthorised use, reproduction or storage of the contents is expressly prohibited. If you have received this email in error, please delete it and any attachments immediately and advise the sender by return email or telephone.
Deakin University does not warrant that this email and any attachments are error or virus free.
More information about the R-sig-meta-analysis
mailing list