Package {baselinr}

Title:	What Works Clearinghouse Standards for Education Impact Evaluations
Version:	0.6.0
Description:	Applies the group-design determinations of the What Works Clearinghouse (WWC) to education impact studies. Computes WWC effect sizes (Hedges' g with the small-sample correction, and the Cox index) and classifies baseline equivalence; classifies overall and differential attrition against the WWC attrition boundary; returns the group-design study rating; and reports the robustness of the baseline-equivalence verdict as a multiverse over the computation choices an analyst could defensibly make differently. Provides report-ready tables and Love plots.
License:	MIT + file LICENSE
Encoding:	UTF-8
Language:	en-US
Imports:	rlang, stats
Suggests:	ggplot2, gt, knitr, rmarkdown, testthat (≥ 3.0.0)
Config/testthat/edition:	3
VignetteBuilder:	knitr
URL:	https://github.com/zl1212-ship-it/baselinr, https://zl1212-ship-it.github.io/baselinr/
BugReports:	https://github.com/zl1212-ship-it/baselinr/issues
Config/roxygen2/version:	8.0.0
Depends:	R (≥ 3.5)
LazyData:	true
NeedsCompilation:	no
Packaged:	2026-08-22 20:30:56 UTC; yuxialiang
Author:	Yuxia Liang [aut, cre]
Maintainer:	Yuxia Liang <zl1212@uw.edu>
Repository:	CRAN
Date/Publication:	2026-08-22 23:20:37 UTC

baselinr: WWC-aligned baseline equivalence for education impact evaluations

Description

baselinr produces report-ready baseline equivalence tables for impact evaluations in education research, following the conventions of the What Works Clearinghouse (WWC). It takes a data frame, a treatment indicator, and a set of covariates, and reports, for each covariate, the appropriate standardized effect size (Hedges' g for continuous covariates, the Cox index for binary covariates) together with the WWC baseline-equivalence category.

Author(s)

Maintainer: Yuxia Liang zl1212@uw.edu (ORCID)

Authors:

• Yuxia Liang zl1212@uw.edu (ORCID)

See Also

Useful links:

• https://github.com/zl1212-ship-it/baselinr

• https://zl1212-ship-it.github.io/baselinr/

• Report bugs at https://github.com/zl1212-ship-it/baselinr/issues

Overall and differential attrition

Description

Computes overall and differential sample attrition for a two-group design, the inputs to the What Works Clearinghouse (WWC) attrition standard. Differential attrition is the absolute difference between the treatment and comparison attrition rates.

Usage

attrition(treatment, retained, na.rm = TRUE)

Arguments

Details

This function reports the attrition rates; it does not classify them. Compare the overall and differential rates against the WWC attrition boundary for your chosen response assumption (cautious or optimistic) in the Procedures Handbook.

Value

A one-row data frame with columns attrition_overall, attrition_treatment, attrition_comparison, and differential_attrition (all proportions).

References

What Works Clearinghouse (2022). Procedures Handbook (Version 5.0). U.S. Department of Education.

Examples

set.seed(1)
g <- rep(c(1, 0), each = 100)
kept <- rbinom(200, 1, ifelse(g == 1, 0.9, 0.8))
attrition(g, kept)

Classify a study under the WWC attrition standard

Description

Given a study's overall and differential attrition, classifies it as low or high attrition against the What Works Clearinghouse (WWC) attrition boundary (Standards Handbook Version 4.1, Table II.1). This is the classification that attrition() deliberately leaves to the user: attrition() reports the rates, attrition_boundary() applies the standard.

Usage

attrition_boundary(
  overall,
  differential,
  assumption = c("cautious", "optimistic")
)

Arguments

Details

The WWC uses one of two boundaries. The cautious boundary is applied when the intervention could plausibly affect attrition (for example, a dropout prevention program); the optimistic boundary when it is unlikely to (for example, a first-grade reading program). The applicable boundary is set by the review protocol, not chosen post hoc; the default here is the more conservative cautious boundary.

Value

A data frame, one row per input, with columns overall, differential, assumption, max_differential (the highest differential attrition still counted as low, as a proportion; NA where the overall rate is beyond the boundary), and attrition ("low" or "high").

References

What Works Clearinghouse (2020). Standards Handbook, Version 4.1, Table II.1. U.S. Department of Education.

Examples

# A study with 8% overall and 3 percentage-point differential attrition:
attrition_boundary(0.08, 0.03) # low under the cautious boundary

# 30% overall, 6-point differential: high if cautious, low if optimistic
attrition_boundary(0.30, 0.06, "cautious")
attrition_boundary(0.30, 0.06, "optimistic")

# Chained from attrition():
set.seed(1)
g <- rep(c(1, 0), each = 100)
kept <- rbinom(200, 1, ifelse(g == 1, 0.9, 0.82))
a <- attrition(g, kept)
attrition_boundary(a$attrition_overall, a$differential_attrition)

Baseline equivalence table for an impact evaluation

Description

Builds a report-ready baseline-equivalence table for a set of covariates, reporting group sample sizes, summaries, the appropriate standardized effect size, and the corresponding What Works Clearinghouse (WWC) equivalence category for each covariate. Continuous covariates use Hedges' g; binary covariates use the Cox index.

Usage

baseline_equivalence(data, treatment, covariates = NULL)

Arguments

Details

A covariate with exactly two unique non-missing values is treated as binary; any other numeric covariate is treated as continuous. A non-numeric covariate with more than two categories is not supported and raises an error.

Value

A data frame with one row per covariate and the columns: covariate; type ("continuous" or "binary"); n_treatment, n_comparison; mean_treatment, mean_comparison (group means for continuous covariates, event proportions for binary ones); sd_treatment, sd_comparison; effect_size (Hedges' g or Cox index, per type); and wwc_category.

References

What Works Clearinghouse (2022). Procedures Handbook (Version 5.0). U.S. Department of Education.

Examples

df <- data.frame(
  treat = c(1, 1, 1, 0, 0, 0),
  pretest = c(5, 6, 7, 4, 5, 6),
  female = c(1, 0, 1, 0, 0, 1)
)
baseline_equivalence(df, treatment = "treat")

WWC clustering correction for mismatched analyses

Description

Corrects the statistical significance of a finding for clustering, following the What Works Clearinghouse (WWC) procedure based on Hedges (2007). The correction applies when assignment was at the cluster level (classrooms or schools) but the analysis was at the student level, a "mismatch" that leaves the reported standard errors too small. It computes the t statistic implied by the effect size, then corrects both the t statistic and its degrees of freedom for clustering, and returns the clustering-corrected two-tailed p value and significance decision.

Usage

cluster_correction(
  g,
  n_treatment,
  n_comparison,
  m_treatment,
  m_comparison,
  icc = NULL,
  outcome = c("achievement", "other")
)

Arguments

Details

Because the correction can only reduce significance, the WWC applies it only to findings the study authors reported as statistically significant; a finding that was not significant stays not significant. This function computes the corrected values regardless; apply them where the WWC rules call for it.

Value

A data frame, one row per input, with columns t (ignoring clustering), t_corrected, df (corrected degrees of freedom), p_value (clustering-corrected, two-tailed), and significant (p_value < 0.05).

References

What Works Clearinghouse (2020). Procedures Handbook, Version 4.1, Appendix F. Hedges, L. V. (2007). Effect sizes in cluster-randomized designs. Journal of Educational and Behavioral Statistics, 32(4), 341-370.

Examples

# A finding with a moderate effect from a clustered design:
cluster_correction(
  g = 0.30, n_treatment = 200, n_comparison = 200,
  m_treatment = 10, m_comparison = 10, outcome = "achievement"
)

Cox index for a binary covariate

Description

Computes the What Works Clearinghouse (WWC) Cox index, a standardized effect size for a binary (dichotomous) covariate. The Cox index places the difference between two proportions on a scale comparable to Hedges' g, so it can be classified with the same baseline-equivalence thresholds.

Usage

cox_index(x, treatment, na.rm = TRUE)

Arguments

Details

The index is d_{Cox} = (\mathrm{logit}(p_t) - \mathrm{logit}(p_c)) / 1.65, where p_t and p_c are the proportions in the "event" category for the treatment and comparison groups.

Value

A single numeric value: the Cox index. Returns NA (with a warning) when a group proportion is exactly 0 or 1, where the index is undefined.

References

What Works Clearinghouse (2022). Procedures Handbook (Version 5.0). U.S. Department of Education.

Examples

x <- c(1, 1, 1, 1, 0, 1, 0, 0)
g <- c(1, 1, 1, 1, 0, 0, 0, 0)
cox_index(x, g)

Format a baseline equivalence table with gt

Description

Renders the result of baseline_equivalence() as a formatted gt table with rounded statistics and readable column labels. Requires the gt package.

Usage

gt_baseline(equivalence, decimals = 2)

Arguments

Value

A gt_tbl object.

Examples

if (requireNamespace("gt", quietly = TRUE)) {
  df <- data.frame(
    treat = c(1, 1, 1, 0, 0, 0),
    pretest = c(5, 6, 7, 4, 5, 6),
    female = c(1, 0, 1, 0, 0, 1)
  )
  tbl <- gt_baseline(baseline_equivalence(df, "treat"))
}

Hedges' g standardized mean difference

Description

Computes the standardized mean difference (Hedges' g) between a treatment and a comparison group for a single numeric covariate, using the pooled within-group standard deviation and the small-sample correction factor used by the What Works Clearinghouse (WWC).

Usage

hedges_g(x, treatment, na.rm = TRUE)

Arguments

Details

The correction factor is \omega = 1 - 3 / (4N - 9), where N = n_{treatment} + n_{comparison}.

Value

A single numeric value: Hedges' g. Positive when the treatment group mean exceeds the comparison group mean.

References

What Works Clearinghouse (2022). Procedures Handbook (Version 5.0). U.S. Department of Education.

Examples

x <- c(5, 6, 7, 4, 5, 6)
g <- c(1, 1, 1, 0, 0, 0)
hedges_g(x, g) # 0.8

Love plot of standardized effect sizes

Description

Plots the standardized effect size for each covariate from baseline_equivalence(), with reference lines at the What Works Clearinghouse (WWC) thresholds (0.05 and 0.25) and points coloured by WWC category. Requires the ggplot2 package.

Usage

love_plot(equivalence, signed = FALSE)

Arguments

Value

A ggplot object.

Examples

if (requireNamespace("ggplot2", quietly = TRUE)) {
  df <- data.frame(
    treat = c(1, 1, 1, 0, 0, 0),
    pretest = c(5, 6, 7, 4, 5, 6),
    female = c(1, 0, 1, 0, 0, 1)
  )
  love_plot(baseline_equivalence(df, "treat"))
}

Simulated tutoring program evaluation

Description

A small, simulated (not real) dataset for demonstrating baseline equivalence assessment in a quasi-experimental education evaluation. It represents 400 students: 200 who received a tutoring program and 200 comparison students who did not, with baseline covariates measured before the program and an outcome measured after. The treatment group is mildly positively selected, so the covariates span all three What Works Clearinghouse equivalence categories.

Usage

tutoring

Format

A data frame with 400 rows and 8 variables:

treat

Treatment indicator: 1 = received tutoring, 0 = comparison.

pretest

Baseline reading score (continuous).

attendance

Baseline attendance rate, 0-1 (continuous).

age

Age in years at baseline (continuous).

female

1 = female, 0 = not (binary).

frpl

Eligible for free or reduced-price lunch: 1 = yes (binary).

ell

English language learner: 1 = yes (binary).

posttest

Reading score after the program (continuous outcome).

Source

Simulated for package examples with data-raw/tutoring.R; not real student data.

Classify baseline equivalence under WWC standards

Description

Maps standardized effect sizes to the three What Works Clearinghouse baseline-equivalence categories. Sign is ignored; classification uses the absolute value of the effect size.

Usage

wwc_classify(es)

Arguments

Value

A character vector the same length as es:

• "satisfied" when ⁠|es| <= 0.05⁠ (no adjustment needed),

• "satisfied_with_adjustment" when ⁠0.05 < |es| <= 0.25⁠ (equivalence holds only if the covariate is adjusted for in the impact model),

• "not_satisfied" when ⁠|es| > 0.25⁠. NA inputs return NA.

References

What Works Clearinghouse (2022). Procedures Handbook (Version 5.0). U.S. Department of Education.

Examples

wwc_classify(c(0.03, 0.12, 0.80))

WWC group-design study rating

Description

Applies the What Works Clearinghouse (WWC) group-design rating logic to the two determinations this package supports, sample attrition and baseline equivalence, and returns the study's rating. This encodes the main decision path of the WWC Standards Handbook (Version 4.1, Section II): a randomized controlled trial (RCT) with low attrition can meet standards without reservations; an RCT with high attrition is held to the same baseline requirement as a quasi-experimental design (QED); and a QED, or a high attrition RCT, meets standards with reservations only if baseline equivalence is established, and otherwise does not meet standards.

Usage

wwc_rating(design, baseline, attrition = NULL)

Arguments

Details

This function covers the attrition-and-equivalence path only. A real WWC review also checks that random assignment was not compromised, that there are no confounding factors, and that the required baseline measures were used; those judgments are the reviewer's and are assumed satisfied here.

Value

A length-one character string: one of "Meets WWC Group Design Standards Without Reservations", "Meets WWC Group Design Standards With Reservations", or "Does Not Meet WWC Group Design Standards". The reasoning is attached as attr(x, "basis").

References

What Works Clearinghouse (2020). Standards Handbook, Version 4.1, Section II. U.S. Department of Education.

Examples

# Low-attrition RCT: meets without reservations regardless of baseline.
wwc_rating("rct", baseline = "not_satisfied", attrition = "low")

# High-attrition RCT hinges on baseline equivalence, like a QED.
wwc_rating("rct", baseline = "satisfied_with_adjustment", attrition = "high")

# QED that fails baseline equivalence does not meet standards.
wwc_rating("qed", baseline = "not_satisfied")

Robustness of the WWC baseline-equivalence verdict

Description

Reports how stable a baseline-equivalence verdict is to the computation choices a careful analyst might defensibly make differently. For each continuous covariate it recomputes the standardized difference under the cross of two choices, standardizing by the pooled versus the comparison-group standard deviation and applying the What Works Clearinghouse (WWC) small-sample correction or not, and records whether the covariate's WWC category changes. Binary covariates use the Cox index, which does not depend on these choices. It also reports whether the overall verdict changes.

Usage

wwc_robustness(data, treatment, covariates = NULL)

Arguments

Details

This is a multiverse, or specification-curve, view of a single WWC determination: it shows whether the verdict depends on which defensible choice is made.

Value

A data frame with one row per covariate and the columns covariate, type, category_default (the category under baselinr's default), the set of categories the covariate takes across the defensible choices, flips (whether that set has more than one category), and abs_es_min / abs_es_max (the range of the absolute effect size across choices). The overall verdict under each choice is attached as attr(x, "overall"), and attr(x, "overall_stable") is TRUE when the overall verdict is invariant.

References

What Works Clearinghouse (2022). Procedures Handbook (Version 5.0). U.S. Department of Education. Steegen, S., Tuerlinckx, F., Gelman, A., & Vanpaemel, W. (2016). Increasing transparency through a multiverse analysis. Perspectives on Psychological Science, 11(5), 702-712.

Examples

df <- data.frame(
  treat = c(1, 1, 1, 0, 0, 0),
  pretest = c(5, 6, 7, 4, 5, 6),
  female = c(1, 0, 1, 0, 0, 1)
)
r <- wwc_robustness(df, treatment = "treat")
r
attr(r, "overall")

Overall WWC baseline-equivalence verdict

Description

Summarizes a baseline_equivalence() table into a one-row overall assessment: how many covariates fall in each What Works Clearinghouse (WWC) category, the largest absolute effect size, and an overall verdict.

Usage

wwc_summary(equivalence)

Arguments

Details

The overall verdict follows the logic of the categories: if any covariate is "not_satisfied", baseline equivalence cannot be established ("not_satisfied"); otherwise, if any covariate requires adjustment, the verdict is "satisfied_with_adjustment" (equivalence holds only if those covariates are adjusted for in the impact model); otherwise "satisfied".

Value

A one-row data frame with columns n_covariates, n_satisfied, n_satisfied_with_adjustment, n_not_satisfied, max_abs_effect, and overall.

References

What Works Clearinghouse (2022). Procedures Handbook (Version 5.0). U.S. Department of Education.

Examples

df <- data.frame(
  treat = c(1, 1, 1, 0, 0, 0),
  pretest = c(5, 6, 7, 4, 5, 6),
  female = c(1, 0, 1, 0, 0, 1)
)
wwc_summary(baseline_equivalence(df, "treat"))