[R-meta] Calculation of variance of ratio of means from summary statistics

Wed Oct 5 16:36:42 CEST 2022

Hi Mick,

Wolfgang's response is spot on, but I'll add a few other notes.

Wolfgang noted that degree to which the mean of ratios (MoR) corresponds to
the ratio of means (RoM) depends on how far away the means are from zero.
That is not the only factor--the correspondence also depends on the
variation in the individual ratios (i.e., the variance of the LSIs, in your
context) and the correlation between the ratio and the baseline level. If
you call the measurements for an individual participant A and B, and let R
= B / A denote the individual ratio, then
E(B) / E(A) = E(R x A) / E(A) = E(R) + Cov(R, A) / E(A) = E(R) + cor(R,A) x
SD(R) x SD(A) / E(A).
Thus, if there is individual-level heterogeneity in the LSIs--so that SD(R)
is big--and if the LSIs correlate with the individual measurements, then
RoM will differ from MoR. So, all the more reason to use moderator analysis
to account for the difference in metrics.

Besides moderator analysis, another approach would be possible if you have
a substantial number of studies where you can compute BOTH the RoM and the
MoR. If you can get both measures for enough studies, then you could treat
the pair as a bivariate ES measure and use a bivariate meta-analysis to
account for the difference between metrics. Here, you need to have enough
studies with both RoM and MoR in order to estimate the average difference
between the metrics and the correlation between the metrics. But then you
can still include other studies that only report RoM or only report MoR.

James

On Tue, Oct 4, 2022 at 6:46 AM Mick Girdwood <M.Girdwood using latrobe.edu.au>
wrote:

> Dear Wolfgang,
>
> Thank you so much for this it is super helpful and much more eloquently
> explained than I did - I think I can see where I got stuck too.
>
> The paired data was something I knew was an issue, but didn’t see that
> there was an effect like ROMC built into escalc, and don’t know how to
> account for before this. Using this has resulted in much greater precision
> (i.e. tighter confidence intervals).
>
> I had actually tried the calculation for vi you mentioned but it again
> seemed too precise, but that was because of my incorrect use of ROM!. Now
> accounting for the correlated pairs with the RoM calculation they are very
> similar, thank you.
>
> Thank you also for the suggestion of the moderator analysis - I had
> planned sensitivity analyses given the slightly different methods, and
> agree your suggestions is a good one I will use. I will go and answer my
> own question pointing to this answer there incase anyone else happens to be
> in a similar position.
>
> Have a great day!
> Mick
>
>
> On 4 Oct 2022, at 18:36, Viechtbauer, Wolfgang (NP) <
> wolfgang.viechtbauer using maastrichtuniversity.nl<mailto:
> wolfgang.viechtbauer using maastrichtuniversity.nl>> wrote:
>
> Dear Mick,
>
> So for some studies you have means and SDs for each side and can compute
> the (log-transformed) ratio of means (RoM) and corresponding sampling
> variances with:
>
> yi = log(m1/m2i)
> vi = sd1i^2/(ni*m1i^2) + sd2i^2/(ni*m2i^2) - 2*ri*sd1i*sd2i/(m1i*m2i*ni)
>
> where ni refers to the number of individuals. Note that you have paired
> data, so the equation for computing the sampling variance must incorporate
> the correlation between the measurements for the two sides (ri). This is
> measure="ROMC" in escalc(). If you do not know this correlation (because
> presumably it is not reported), then you could try to make an educated
> guestimate.
>
> But for other studies, you have the mean of the ratios (MoR),
> mean(x1i/x2i), and I assume the corresponding standard deviation of the
> ratios (SDoR), sd(x1i/x2i)).
>
> To what extent MoR approximates RoM depends on how close the true means
> are to 0. To demonstrate:
>
> x1 <- rnorm(1000000, 10, 1)
> x2 <- rnorm(1000000, 5, 1)
>
> mean(x1/x2)
> mean(x1)/mean(x2)
>
> x1 <- rnorm(1000000, 100, 1)
> x2 <- rnorm(1000000, 50, 1)
>
> mean(x1/x2)
> mean(x1)/mean(x2)
>
> Assuming you are in a situation where the means are far enough away from 0
> so that this doesn't matter, then how can we incorporate MoR studies into
> the meta-analysis? It would be:
>
> yi = log(mean(x1i/x2i))
> vi = sd(x1i/x2i))^2 / (ni * mean(x1i/x2i)^2)
>
> or using the abbreviations above
>
> yi = log(MoR)
> vi = SDoR^2 / (ni * MoR^2)
>
> where ni is again the sample size. The vi equation comes from a simple
> application of the delta method (
> https://en.wikipedia.org/wiki/Delta_method<
> https://en.wikipedia.org/wiki/Delta_method>).
>
> I would recommend to code for each yi value whether it is a (log
> transformed) RoM or MoR value and examine via a moderator analysis whether
> there is a systematic difference between the two sets.
>
> P.S.: Thanks for pointing out that this was also posted on Cross Validated.
>
> Best,
> Wolfgang
>
> >-----Original Message-----
> >From: R-sig-meta-analysis [mailto:
> r-sig-meta-analysis-bounces using r-project.org] On
> >Behalf Of Mick Girdwood
> >Sent: Tuesday, 04 October, 2022 3:01
> >To: r-sig-meta-analysis using r-project.org<mailto:
> r-sig-meta-analysis using r-project.org>
> >Subject: [R-meta] Calculation of variance of ratio of means from summary
> >statistics
> >
> >Thank you for this message board, it’s always so helpful, and sorry for
> the
> >slightly long question. I am planning a meta-analysis where we are using
> the
> >ratio of means (RoM) effect measure for comparing outcomes from two sides
> of the
> >body (i.e. the affected side and unaffected side). E.g. we are looking at
> a
> >particular muscle strength outcome, this is reported for each side. Most
> studies
> >we are concerned with will report a mean and sd for each side of the
> body. We can
> >then use these for the RoM calculations. In some cases authors will
> report only
> >a 'limb symmetry index ' - e.g. for each participant the LSI = the
> affected /
> >non-affected side. These are then summarised and reported for the group
> as the
> >mean(LSI). In these cases we cannot compute the RoM effect measure, as we
> don’t
> >have the summaries for each side. My issue is about how to incorporate
> the limb
> >symmetry index (LSI) data into the meta-analysis.
> >I am aware that the RoM is essentially mean[affected] /
> mean[non-affected], which
> >is slightly different to LSI - mean(affected/nonaffected). The LSI is a
> good
> >almost perfect approximation of the RoM (I have checked this with data
> where
> >studies reported both). So this is easy to just use log(LSI) to get an
> estimate
> >for yi.
> >My issue is around how to determine a precision parameter (vi sei etc)
> i.e. how
> >can I estimate the variance from the LSI standard deviation provided. I am
> >struggling with this especially as RoM meta-analysis is performed with log
> >transformation, so am not sure how to use the arithmetic SD in this
> situation. I
> >have a large number of studies where only the LSI mean and SD are provide
> (and so
> >cannot calculate the RoM from the data for each side). I have done an
> extensive
> >amount of reading and haven't been able to find anything that works well.
> >I first tried converting the SD to confidence intervals, then log
> transforming
> >these to use, however they overestimate the precision by about a factor
> of ~2, a
> >problem with using inverse variance methods. I also tried methods from
> this paper
> >by Quan 2003 (equation 5)
> >https://onlinelibrary.wiley.com/doi/abs/10.1002/sim.1525<
> https://onlinelibrary.wiley.com/doi/abs/10.1002/sim.1525> however this
> majorly
> >underestimates the precision.
> >The best solution I've found was by accident - I was looking at using the
> formula
> >for sample variance SD^2 / (n-1) and accidentally forgot to square the SD
> in my
> >code, so in effect my formula became sd / (n-1). This was very
> good/comparable
> >when comparing to the CIs generated by RoM calculations. However this is
> an
> >obvious accident so I don’t have any justification to proceed with this
> >My questions are How can I use the LSI data and convert these
> (specifically the
> >variance) for use in analyses In the case of my accidental approach
> SD/(n-1) -
> >why does this work well? Is there any justification I could call on to
> use this?
> >Or is it just a major co-incidence that it provides a good estimation? I
> have a
> >dataset of n=40 on which I tested this on and all were very reasonable
> >approximations. Thank you for your help!
> >Mick Girdwood
> >La Trobe University | Australia
> >
> >n.b. I have posted a similar question here -
> >
> https://stats.stackexchange.com/questions/590611/use-of-ratio-of-means-effect-
> <
> https://stats.stackexchange.com/questions/590611/use-of-ratio-of-means-effect-
> >
> >measure-and-estimation-of-variance-from-mean-of-rat
>
>
>         [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-meta-analysis mailing list @ R-sig-meta-analysis using r-project.org
> To manage your subscription to this mailing list, go to:
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>

	[[alternative HTML version deleted]]