[R-meta] specification of phi in vcalc

Thu Aug 4 05:11:34 CEST 2022

Hi Danielle,

Just to add a little to Lukasz's suggestions (which I think are excellent),
it is a little unusual and surprising that your results are very sensitive
to changing the value of phi from 0.6 to 0.8. If you can provide a bit more
detail about the structure of your data and model, I may be able to offer
some suggestions on how to interpret this sensitivity. Specifically, it
would be useful to know:
* How many studies you've got
* How many effect sizes per study (on average and the range of effect sizes
per study; quartiles would be great too)
* Your model specification, including both what moderators are included and
how you've specified the random effects
* For any included moderators, whether they are study-level characteristics
(constant across the effect sizes for a given study) or effect-level
characteristics (that vary within study).
* If you're open to sharing, the results of fitting the model with
different values of phi, to provide a sense of how much the estimates
change.

Incidentally, this issue is closely related to something I'm studying right
now: https://www.jepusto.com/talk/srsm-2022-matter-of-emphasis/

James

On Wed, Aug 3, 2022 at 10:14 AM Lukasz Stasielowicz <
lukasz.stasielowicz using uni-osnabrueck.de> wrote:

> Dear Danielle,
>
> vcalc documentation contains a potentially helpful tip: "Argument phi
> must then also be specified to indicate the autocorrelation among the
> sampling errors of two effect sizes that differ by one unit on the time1
> variable. As above, the autocorrelation of the measurements themselves
> can be used here as a proxy."
> https://rdrr.io/github/wviechtb/metafor/man/vcalc.html
>
> Perhaps some tables in primary studies show correlations between
> neighboring time points?
>
> Another option: If raw data are available for some primary studies, then
> one could estimate the correlation for several data sets and use it as
> "phi" input for vcalc. If different data sets lead to different values,
> then one could test different values in sensitivity analyses.
>
>
>
>
> Best,
> Lukasz
> --
> Lukasz Stasielowicz
> Osnabrück University
> Institute for Psychology
> Research methods, psychological assessment, and evaluation
> Seminarstraße 20
> 49074 Osnabrück (Germany)
>
> See also:
> On 03.08.2022 12:00, r-sig-meta-analysis-request using r-project.org wrote:
> > Send R-sig-meta-analysis mailing list submissions to
> >       r-sig-meta-analysis using r-project.org
> >
> > To subscribe or unsubscribe via the World Wide Web, visit
> >       https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
> > or, via email, send a message with subject or body 'help' to
> >       r-sig-meta-analysis-request using r-project.org
> >
> > You can reach the person managing the list at
> >       r-sig-meta-analysis-owner using r-project.org
> >
> > When replying, please edit your Subject line so it is more specific
> > than "Re: Contents of R-sig-meta-analysis digest..."
> >
> >
> > Today's Topics:
> >
> >     1. specification of phi in vcalc (Danielle Hiam)
> >
> > ----------------------------------------------------------------------
> >
> > Message: 1
> > Date: Wed, 3 Aug 2022 06:54:48 +0000
> > From: Danielle Hiam <danielle.hiam using deakin.edu.au>
> > To: "r-sig-meta-analysis using r-project.org"
> >       <r-sig-meta-analysis using r-project.org>
> > Subject: [R-meta] specification of phi in vcalc
> > Message-ID:
> >       <
> ME3PR01MB54642B304670F1BF825E6F60B79C9 using ME3PR01MB5464.ausprd01.prod.outlook.com
> >
> >
> > Content-Type: text/plain; charset="iso-8859-1"
> >
> > Hello,
> >
> > I am running a meta-analysis that has nested data and dependent effect
> sizes (multiple timepoints). Therefore, I need to take into account effect
> sizes that are correlated and also cluster within studies. I will run
> vcalc, then the rma.mv function and then use robust with clubsandwich set
> to true to "correct" for any mis-specification of the model. However, I am
> looking for some clarification regarding how to "guess" the value for phi
> in the vcalc function, R code: V <- vcalc(FC_SD, cluster, time1=Time_NUM,
> data, phi=0.6).
> >
> > When I change phi from 0.6 to 0.8 in vcalc I get very different results
> from the meta-analysis (rma.mv followed by robust(.... Clubsanwich=T). I
> have read James Pustejovsky paper (DOI:
> https://doi.org/10.1007/s11121-021-01246-3) on this, where he suggests
> that in situations where no information is available, the meta-analyst
> might pick a plausible value and then conduct sensitivity analysis across a
> range reasonable values. How would I go about this?
> >
> > Any information regarding specification of phi would be greatly
> appreciated
> >
> > Thanks,
> > Danielle
> >
> > Deakin University
> > Melbourne Burwood Campus, 221 Burwood Highway, Burwood VIC 3125
> > danielle.hiam using deakin.edu.au
> > ipan.deakin.edu.au
> > Deakin University CRICOS Provider Code: 00113B
> >
> > Stay connected with IPAN
> > Facebook        Twitter
> >
> > Important Notice: The contents of this email are intended solely for the
> named addressee and are confidential; any unauthorised use, reproduction or
> storage of the contents is expressly prohibited.
> > If you have received this email in error, please delete it and any
> attachments immediately and advise the sender by return email or telephone.
> Deakin University does not warrant that this email and any attachments are
> error or virus free.
> >
> >
> >
> > Important Notice: The contents of this email are intended solely for the
> named addressee and are confidential; any unauthorised use, reproduction or
> storage of the contents is expressly prohibited. If you have received this
> email in error, please delete it and any attachments immediately and advise
> the sender by return email or telephone.
> >
> > Deakin University does not warrant that this email and any attachments
> are error or virus free.
> >
> >
> >
> >
> > ------------------------------
> >
> > Subject: Digest Footer
> >
> > _______________________________________________
> > R-sig-meta-analysis mailing list @ R-sig-meta-analysis using r-project.org
> > To manage your subscription to this mailing list, go to:
> > https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
> >
> >
> > ------------------------------
> >
> > End of R-sig-meta-analysis Digest, Vol 63, Issue 1
> > **************************************************
>
> _______________________________________________
> R-sig-meta-analysis mailing list @ R-sig-meta-analysis using r-project.org
> To manage your subscription to this mailing list, go to:
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>

	[[alternative HTML version deleted]]