[R-meta] calculate effect size and variance for prepost proportion data

[James Pustejovsky]

Hi Zone,

I think it is less common to use pre-post effect size measures with binary
outcomes. In principle, it can be done, but my sense is that there is less
benefit (in terms of precision improvement) from using a binary pre-test
than there is from accounting for pre-tests with continuous outcomes.

Wei and Higgins (2013; https://doi.org/10.1002/sim.5679) discuss the
covariance between log odds ratios computed for different binary outcomes,
which is closely related to the case you're looking at. In order to get an
accurate estimate of the sampling variance of the pre-post log odds ratio,
you will need to know the correlation between the pre-test outcome and the
post-test outcome or, equivalently, the number of participants with the
positive outcome at both pre-test and post-test.

Say that that the outcome is school suspension (at any time) during 6th
grade (pre) and 7th grade (post). Let P6 be the overall proportion of
students suspended during 6th grade, P7 be the overall proportion of
students suspended during 7th grade, and B be the proportion of students
suspended during both 6th and 7th grades. Let N be the total sample size
(which I'm assuming to be the same at both time points). The pre-post LOR is

LOR = log[P7 / (1 - P7)] - log[P6 / (1 - P6)]

And an estimate of its sampling variance is

V(LOR) = [P6 (1 - P6) + P7 (1 - P7) - 2 (B - P6 * P7)] / [N P6 (1 - P6) P7
(1 - P7)]

As you can see, you'll need to know B to compute this. If this is not
reported, you could use a conservative estimate (i.e., a probable
over-estimate) of the sampling variance based on the assumption that the
outcomes are independent (in which case B = P6 * P7 and the last term in
the numerator drops out) but I'm not sure how useful that would be in your
application.

James

On Mon, Jan 2, 2023 at 7:40 AM Liu Sicong <64zone using gmail.com> wrote:

> Happy 2023 All!
>
> I have some prepost proportion data. For instance, some clinical trials
> may intervene on patients’ vaccine uptake and report the proportion of
> patients who received the vaccine both prior to and after interventions. So
> I may have the following data
>
>   *   Outcomes in proportion: p_control_pre, p_control_post,
> p_experiment_pre, p_experiment_post
>   *   Sample sizes: n_control_pre, n_control_post, n_experiment_pre,
> n_experiment_post
>
> I am clear about how to calculate between-condition effect sizes and
> variances in the following manner. For instance, those for comparing the
> conditions at posttest would be:
>
>   *   Effect size: ln((p_experiment_post/(1 -
> p_experiment_post))/(p_control_post/(1 - p_control_post)))
>   *   Variance of effect size: 1/(n_experiement_post*p_experiment_post) +
> 1/(n_experiement_post*(1-p_experiment_post)) +
> 1/(n_control_post*p_control_post) + 1/(n_control_post*(1-p_control_post))
>
> My question is about how to calculate the effect size and its variance
> when I am also interested in within-condition growth. For instance, how to
> represent the prepost growth due to vaccination intervention for the
> experimental group? Perhaps even before asking this question, would it be
> reasonable to attempt the computation of such effect sizes and variances?
> Thank you very much!
>
> Best regards,
> Sicong (Zone)
>
> ------------------------------------------
> Sicong (Zone) Liu, Ph.D.
> Research Associate
> University of Pennsylvania
>
> 3620 Walnut Street,
> Philadelphia, PA 19104-6220
> ------------------------------------------
>
>
>         [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-meta-analysis mailing list @ R-sig-meta-analysis using r-project.org
> To manage your subscription to this mailing list, go to:
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>

	[[alternative HTML version deleted]]