[R-meta] calculate effect size and variance for prepost proportion data

[Liu Sicong]

Thank you for clarifying James!

Just one follow-up question:

  *   If I would like to transform the V(LOR) to Cohen�s d metric, does �V(LOR) * 3/Pi� still work? Thank you!

Cheers,
Zone
-------------


From: James Pustejovsky <jepusto using gmail.com>
Date: Tuesday, January 3, 2023 at 3:57 PM
To: Sicong Liu <64zone using gmail.com>
Cc: "r-sig-meta-analysis using r-project.org" <r-sig-meta-analysis using r-project.org>
Subject: Re: [R-meta] calculate effect size and variance for prepost proportion data

Hi Zone,

I have not been able to find a reference for the pre-post log odds ratio in particular. I derived the formula using the delta method (same as Wei and Higgins) and the properties of the multinomial distribution.

Perhaps others on the list know of a reference?

James

On Tue, Jan 3, 2023 at 2:13 PM Liu Sicong <64zone using gmail.com<mailto:64zone using gmail.com>> wrote:
Happy 2023 and thank you for your response, James!

I wonder if you could point me to the reference of the formulas raised, especially the V(LOR) one? I checked Wei and Higgins (2013) but did not find such a formula explicitly expressed in the paper. Perhaps the V(LOR) is derived from their general method? Please let me know.

Cheers,
Zone
-------------


From: James Pustejovsky <jepusto using gmail.com<mailto:jepusto using gmail.com>>
Date: Tuesday, January 3, 2023 at 10:43 AM
To: Sicong Liu <64zone using gmail.com<mailto:64zone using gmail.com>>
Cc: "r-sig-meta-analysis using r-project.org<mailto:r-sig-meta-analysis using r-project.org>" <r-sig-meta-analysis using r-project.org<mailto:r-sig-meta-analysis using r-project.org>>
Subject: Re: [R-meta] calculate effect size and variance for prepost proportion data

Hi Zone,

I think it is less common to use pre-post effect size measures with binary outcomes. In principle, it can be done, but my sense is that there is less benefit (in terms of precision improvement) from using a binary pre-test than there is from accounting for pre-tests with continuous outcomes.

Wei and Higgins (2013; https://doi.org/10.1002/sim.5679) discuss the covariance between log odds ratios computed for different binary outcomes, which is closely related to the case you're looking at. In order to get an accurate estimate of the sampling variance of the pre-post log odds ratio, you will need to know the correlation between the pre-test outcome and the post-test outcome or, equivalently, the number of participants with the positive outcome at both pre-test and post-test.

Say that that the outcome is school suspension (at any time) during 6th grade (pre) and 7th grade (post). Let P6 be the overall proportion of students suspended during 6th grade, P7 be the overall proportion of students suspended during 7th grade, and B be the proportion of students suspended during both 6th and 7th grades. Let N be the total sample size (which I'm assuming to be the same at both time points). The pre-post LOR is

LOR = log[P7 / (1 - P7)] - log[P6 / (1 - P6)]

And an estimate of its sampling variance is

V(LOR) = [P6 (1 - P6) + P7 (1 - P7) - 2 (B - P6 * P7)] / [N P6 (1 - P6) P7 (1 - P7)]

As you can see, you'll need to know B to compute this. If this is not reported, you could use a conservative estimate (i.e., a probable over-estimate) of the sampling variance based on the assumption that the outcomes are independent (in which case B = P6 * P7 and the last term in the numerator drops out) but I'm not sure how useful that would be in your application.

James

On Mon, Jan 2, 2023 at 7:40 AM Liu Sicong <64zone using gmail.com<mailto:64zone using gmail.com>> wrote:
Happy 2023 All!

I have some prepost proportion data. For instance, some clinical trials may intervene on patients� vaccine uptake and report the proportion of patients who received the vaccine both prior to and after interventions. So I may have the following data

  *   Outcomes in proportion: p_control_pre, p_control_post, p_experiment_pre, p_experiment_post
  *   Sample sizes: n_control_pre, n_control_post, n_experiment_pre, n_experiment_post

I am clear about how to calculate between-condition effect sizes and variances in the following manner. For instance, those for comparing the conditions at posttest would be:

  *   Effect size: ln((p_experiment_post/(1 - p_experiment_post))/(p_control_post/(1 - p_control_post)))
  *   Variance of effect size: 1/(n_experiement_post*p_experiment_post) + 1/(n_experiement_post*(1-p_experiment_post)) + 1/(n_control_post*p_control_post) + 1/(n_control_post*(1-p_control_post))

My question is about how to calculate the effect size and its variance when I am also interested in within-condition growth. For instance, how to represent the prepost growth due to vaccination intervention for the experimental group? Perhaps even before asking this question, would it be reasonable to attempt the computation of such effect sizes and variances? Thank you very much!

Best regards,
Sicong (Zone)

------------------------------------------
Sicong (Zone) Liu, Ph.D.
Research Associate
University of Pennsylvania

3620 Walnut Street,
Philadelphia, PA 19104-6220
------------------------------------------


        [[alternative HTML version deleted]]

_______________________________________________
R-sig-meta-analysis mailing list @ R-sig-meta-analysis using r-project.org<mailto:R-sig-meta-analysis using r-project.org>
To manage your subscription to this mailing list, go to:
https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis

	[[alternative HTML version deleted]]