[R-meta] calculate effect size and variance for prepost proportion data

[James Pustejovsky]

Hi Zone,

I have not been able to find a reference for the pre-post log odds ratio in
particular. I derived the formula using the delta method (same as Wei and
Higgins) and the properties of the multinomial distribution.

Perhaps others on the list know of a reference?

James

On Tue, Jan 3, 2023 at 2:13 PM Liu Sicong <64zone using gmail.com> wrote:

> Happy 2023 and thank you for your response, James!
>
>
>
> I wonder if you could point me to the reference of the formulas raised,
> especially the V(LOR) one? I checked Wei and Higgins (2013) but did not
> find such a formula explicitly expressed in the paper. Perhaps the V(LOR)
> is derived from their general method? Please let me know.
>
>
>
> Cheers,
>
> Zone
>
> -------------
>
>
>
>
>
> *From: *James Pustejovsky <jepusto using gmail.com>
> *Date: *Tuesday, January 3, 2023 at 10:43 AM
> *To: *Sicong Liu <64zone using gmail.com>
> *Cc: *"r-sig-meta-analysis using r-project.org" <
> r-sig-meta-analysis using r-project.org>
> *Subject: *Re: [R-meta] calculate effect size and variance for prepost
> proportion data
>
>
>
> Hi Zone,
>
>
>
> I think it is less common to use pre-post effect size measures with binary
> outcomes. In principle, it can be done, but my sense is that there is less
> benefit (in terms of precision improvement) from using a binary pre-test
> than there is from accounting for pre-tests with continuous outcomes.
>
>
>
> Wei and Higgins (2013; https://doi.org/10.1002/sim.5679) discuss the
> covariance between log odds ratios computed for different binary outcomes,
> which is closely related to the case you're looking at. In order to get an
> accurate estimate of the sampling variance of the pre-post log odds ratio,
> you will need to know the correlation between the pre-test outcome and the
> post-test outcome or, equivalently, the number of participants with the
> positive outcome at both pre-test and post-test.
>
>
>
> Say that that the outcome is school suspension (at any time) during 6th
> grade (pre) and 7th grade (post). Let P6 be the overall proportion of
> students suspended during 6th grade, P7 be the overall proportion of
> students suspended during 7th grade, and B be the proportion of students
> suspended during both 6th and 7th grades. Let N be the total sample size
> (which I'm assuming to be the same at both time points). The pre-post LOR is
>
>
>
> LOR = log[P7 / (1 - P7)] - log[P6 / (1 - P6)]
>
>
>
> And an estimate of its sampling variance is
>
>
> V(LOR) = [P6 (1 - P6) + P7 (1 - P7) - 2 (B - P6 * P7)] / [N P6 (1 - P6) P7
> (1 - P7)]
>
>
>
> As you can see, you'll need to know B to compute this. If this is not
> reported, you could use a conservative estimate (i.e., a probable
> over-estimate) of the sampling variance based on the assumption that the
> outcomes are independent (in which case B = P6 * P7 and the last term in
> the numerator drops out) but I'm not sure how useful that would be in your
> application.
>
>
>
> James
>
>
>
> On Mon, Jan 2, 2023 at 7:40 AM Liu Sicong <64zone using gmail.com> wrote:
>
> Happy 2023 All!
>
> I have some prepost proportion data. For instance, some clinical trials
> may intervene on patients’ vaccine uptake and report the proportion of
> patients who received the vaccine both prior to and after interventions. So
> I may have the following data
>
>   *   Outcomes in proportion: p_control_pre, p_control_post,
> p_experiment_pre, p_experiment_post
>   *   Sample sizes: n_control_pre, n_control_post, n_experiment_pre,
> n_experiment_post
>
> I am clear about how to calculate between-condition effect sizes and
> variances in the following manner. For instance, those for comparing the
> conditions at posttest would be:
>
>   *   Effect size: ln((p_experiment_post/(1 -
> p_experiment_post))/(p_control_post/(1 - p_control_post)))
>   *   Variance of effect size: 1/(n_experiement_post*p_experiment_post) +
> 1/(n_experiement_post*(1-p_experiment_post)) +
> 1/(n_control_post*p_control_post) + 1/(n_control_post*(1-p_control_post))
>
> My question is about how to calculate the effect size and its variance
> when I am also interested in within-condition growth. For instance, how to
> represent the prepost growth due to vaccination intervention for the
> experimental group? Perhaps even before asking this question, would it be
> reasonable to attempt the computation of such effect sizes and variances?
> Thank you very much!
>
> Best regards,
> Sicong (Zone)
>
> ------------------------------------------
> Sicong (Zone) Liu, Ph.D.
> Research Associate
> University of Pennsylvania
>
> 3620 Walnut Street,
> Philadelphia, PA 19104-6220
> ------------------------------------------
>
>
>         [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-meta-analysis mailing list @ R-sig-meta-analysis using r-project.org
> To manage your subscription to this mailing list, go to:
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>
>

	[[alternative HTML version deleted]]