[R-meta] variance of predicted effect sizes

Viechtbauer, Wolfgang (SP) wolfg@ng@viechtb@uer @ending from m@@@trichtuniver@ity@nl
Tue Jan 8 13:56:59 CET 2019

Hi Hugh,

Clear. Unfortunately, doing so would take me more time than I can invest right now. Maybe somebody else is willing to work with you on that.

When constructing the right 'V' matrix is complicated, an alternative (that has been discussed on this mailing list on various occasions) is to make use of cluster-robust inference methods. So, you use a 'working' model that is a decent approximation (but might not quite capture all dependencies fully in the 'V' matrix) and then use robust() from metafor or coef_test() and conf_int() from clubSandwich. This would be my suggested approach instead of (or in addition to) the sensitivity analyses you propose.


>-----Original Message-----
>From: Crean, Hugh [mailto:hugh_crean using URMC.Rochester.edu]
>Sent: Monday, 07 January, 2019 23:19
>To: Viechtbauer, Wolfgang (SP); 'r-sig-meta-analysis using r-project.org'
>Subject: RE: variance of predicted effect sizes
>Hi Wolfgang et al.,
>Yes, it is the working out the correct equations that have myself and
>colleagues stumped right now.
>Thanks again,
>-----Original Message-----
>From: Viechtbauer, Wolfgang (SP)
><wolfgang.viechtbauer using maastrichtuniversity.nl>
>Sent: Monday, January 07, 2019 10:52 AM
>To: Crean, Hugh <hugh_crean using URMC.Rochester.edu>; 'r-sig-meta-analysis using r-
>project.org' <r-sig-meta-analysis using r-project.org>
>Subject: RE: variance of predicted effect sizes
>Hi Hugh,
>"Is there a way of incorporating both sets of dependencies into the V
>matrix?" Probably, but one would have to work out the correct equations
>for the covariances (I am not aware of anybody who has done this
>>-----Original Message-----
>>From: Crean, Hugh [mailto:hugh_crean using URMC.Rochester.edu]
>>Sent: Wednesday, 02 January, 2019 22:39
>>To: Viechtbauer, Wolfgang (SP); 'r-sig-meta-analysis using r-project.org'
>>Subject: RE: variance of predicted effect sizes
>>Hello Wolfgang et al.,
>>Yes to the below and thanks again so much.  To start, I have been
>>trying to implement the simpler approach of just using the weighted
>>average control group effects for those studies not having a control
>>group and as you mention below, using the squared SE as the sampling
>>variance.  I have an additional wrinkle, however, and I cannot find
>>much guidance in the literature.  A few of the studies have their own
>>dependencies (multiple treatment arms in the same study either with or
>without a control group).
>>Is there a way of incorporating both sets of dependencies into the V
>>matrix? For now, I am thinking along the lines of using a sensitivity
>>analyses where two estimates are computed -- the first would just add
>>the two estimates together as this would provide a high estimate of the
>>possible covariance and the second would just take the higher of the
>>two covariances.  Neither feels satisfying (aside from difficulties
>>with non- positive matrices) but does at least attempt to recognize
>>these dependent effects.
>>Best and hope all had wonderful Holidays,
>>-----Original Message-----
>>Dear Hugh,
>>It sounds to me that you want to do something like Becker (1988)
>>describes in section 5.2. Then you would use the squared standard error
>>of the predicted value (from the meta-regression model) as the sampling
>>variance of the control group estimate of the standardized mean change.
>>There is an additional complication that there is then a dependency
>>between the effect sizes (i.e., the difference in the standardized mean
>>change between the treatment and control group) for studies with both
>>treatment and control groups and effect sizes for studies with just a
>>treatment group (and also between the effect sizes from studies with
>>treatment groups only). These covariances can be computed as described
>>5.2 and would need to be put into the 'V' matrix of rma.mv() (if you
>>intend to use metafor). Actually implemting this would require a bit of
>>work though. Showing how to do this with metafor would be a nice little
>>exercise/project for a motivated student.
>>-----Original Message-----
>>From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-
>>project.org] On Behalf Of Crean, Hugh
>>Sent: Sunday, 11 November, 2018 23:22
>>To: 'r-sig-meta-analysis using r-project.org'
>>Subject: [R-meta] variance of predicted effect sizes
>>Colleagues and myself are working on a meta-analysis of sleep
>>interventions.  Many of the studies are only single arm pre-post
>>studies and we are following the advice of Becker (1988) and Morris and
>>(2002) to impute missing control group effect sizes.  We are planning
>>on using meta regression to compute predicted effect sizes for those
>>studies missing control information.  However, I cannot quite figure
>>how to compute and/or get the standard error for this estimate.  Would
>>one run a simple meta- analysis on the predicted scores for those with
>>the data and use the provided se (and variance)?
>>Thanks in advance,
>>Hugh F. Crean, Ph.D.
>>School of Nursing
>>University of Rochester
>>601 Elmwood Avenue
>>Rochester, New York  14620
>>(585) 276-5575

More information about the R-sig-meta-analysis mailing list