Model Validation: Latent Class Framework for Measuring Learning

Typical Workflow

This section demonstrates a complete analysis workflow from raw data to final estimates.

Step 1: Prepare Your Data

Your data should be two data frames with the same dimensions: - pre_test: Pre-test responses (0 = wrong, 1 = correct, optionally “d” for Don’t Know) - post_test: Post-test responses (same coding)

Each row is a respondent, each column is an item.

pre_test <- data.frame(
  item1 = c(1, 0, 0, 1, 0, 1, 0, 0, 1, 0),
  item2 = c(0, 0, 1, 1, 0, 0, 1, 0, 1, 0),
  item3 = c(1, 1, 0, 1, 0, 0, 0, 1, 1, 0)
)

post_test <- data.frame(
  item1 = c(1, 1, 0, 1, 1, 1, 0, 1, 1, 0),
  item2 = c(1, 0, 1, 1, 1, 0, 1, 0, 1, 1),
  item3 = c(1, 1, 1, 1, 0, 1, 0, 1, 1, 0)
)

Step 2: Compute Naive Learning Estimate

Before applying the LCA correction, compute the naive estimate for comparison:

naive_learning <- colMeans(post_test) - colMeans(pre_test)
cat("Naive learning estimates (biased downward):\n")
#> Naive learning estimates (biased downward):
print(round(naive_learning, 3))
#> item1 item2 item3 
#>   0.3   0.3   0.2
cat(sprintf("\nMean naive learning: %.3f\n", mean(naive_learning)))
#> 
#> Mean naive learning: 0.267

Step 3: Fit the LCA Model

Use item_lca_fit() to estimate the latent class proportions:

fit <- item_lca_fit(pre_test, post_test)
print(fit)
#> LCA Model Fit
#> ----------------------------------------
#> Items: 3  | Observations: 30 
#> Model: Without Don't Know 
#> 
#> Learning estimates:
#> item1 item2 item3 
#>   0.3   0.3   0.2 
#> 
#> Use summary() for parameter details, coef() to extract parameters.

Step 4: Extract and Interpret Results

The key output is the gk parameter—the proportion who learned:

summary(fit)
#> LCA Model Summary
#> ==================================================
#> Items: 3  | Observations: 30 
#> Model: Without Don't Know 
#> 
#> Parameter Estimates:
#> --------------------------------------------------
#>       item1 item2 item3
#> gg      0.3   0.3   0.3
#> gk      0.3   0.3   0.2
#> kk      0.4   0.4   0.5
#> gamma   0.0   0.0   0.0
#> 
#> Learning Estimates (gk):
#> --------------------------------------------------
#> item1 item2 item3 
#>   0.3   0.3   0.2 
#> 
#> Mean learning: 0.2667

cat("\nComparison: Naive vs. LCA-adjusted learning:\n")
#> 
#> Comparison: Naive vs. LCA-adjusted learning:
comparison <- data.frame(
  Item = colnames(pre_test),
  Naive = naive_learning,
  LCA_Adjusted = fit$learning,
  Difference = fit$learning - naive_learning
)
print(comparison, row.names = FALSE)
#>   Item Naive LCA_Adjusted    Difference
#>  item1   0.3          0.3 -5.071168e-09
#>  item2   0.3          0.3 -5.071168e-09
#>  item3   0.2          0.2 -2.032310e-08

The LCA-adjusted estimates should be larger than naive estimates because they correct for the downward bias from guessing.

Step 5: Get Standard Errors via Bootstrap

For inference, use lca_se() to obtain bootstrapped standard errors:

se_results <- lca_se(pre_test, post_test, n_boot = 100)

Step 6: Assess Model Fit

Check how well the model fits each item:

fit_stats <- fit_model(
  pre_test, post_test,
  fit$params["gamma", ],
  fit$params[c("gg", "gk", "kk"), ]
)
print(fit_stats)
#>            item1 item2 item3
#> chi-square    NA    NA    NA
#> p-value       NA    NA    NA

Non-significant p-values (> 0.05) indicate adequate fit.

Step 7: Compare Across Groups (Optional)

To compare learning between groups, fit the model separately:

group <- c(rep("treatment", 5), rep("control", 5))

pre_treat <- pre_test[group == "treatment", ]
post_treat <- post_test[group == "treatment", ]
pre_ctrl <- pre_test[group == "control", ]
post_ctrl <- post_test[group == "control", ]

fit_treat <- item_lca_fit(pre_treat, post_treat)
fit_ctrl <- item_lca_fit(pre_ctrl, post_ctrl)

cat("Treatment group learning:", round(mean(fit_treat$learning), 3), "\n")
#> Treatment group learning: 0.333
cat("Control group learning:", round(mean(fit_ctrl$learning), 3), "\n")
#> Control group learning: 0.2
cat("Difference:", round(mean(fit_treat$learning) - mean(fit_ctrl$learning), 3), "\n")
#> Difference: 0.133

The Latent Class Model

Three Latent Classes

The model assumes three mutually exclusive latent classes representing knowledge states before and after an informative process:

Class	Name	Interpretation
gg	guess-guess	Did not know before, did not know after (stable ignorance)
gk	guess-know	Did not know before, knows after (LEARNED)
kk	know-know	Knew before, knows after (stable knowledge)

The key simplifying assumption is no forgetting: we rule out know-to-guess transitions. This is reasonable for short-term learning interventions where forgetting is unlikely.

Let \(\gamma\) denote the probability of answering correctly when guessing. For a K-option multiple choice question with random guessing, \(\gamma = 1/K\).

Cell Probability Derivation

On a pre-post test, we observe a 2×2 transition matrix:

Pre Post	0 (Wrong)	1 (Right)
0 (Wrong)	\(n_{00}\)	\(n_{01}\)
1 (Right)	\(n_{10}\)	\(n_{11}\)

Each latent class generates a specific pattern of responses:

Class gg (guess both times):

Pre-test: correct with probability \(\gamma\), wrong with probability \((1-\gamma)\)
Post-test: correct with probability \(\gamma\), wrong with probability \((1-\gamma)\)

Class gk (guess pre, know post):

Pre-test: correct with probability \(\gamma\), wrong with probability \((1-\gamma)\)
Post-test: always correct (probability 1)

Class kk (know both times):

Pre-test: always correct (probability 1)
Post-test: always correct (probability 1)

The probability of each observable cell is the sum over latent classes:

\[ \begin{aligned} P(0 \to 0) &= gg \cdot (1-\gamma)(1-\gamma) \\ &= (1-\gamma)^2 \cdot gg \end{aligned} \]

\[ \begin{aligned} P(0 \to 1) &= gg \cdot (1-\gamma)\gamma + gk \cdot (1-\gamma) \cdot 1 \\ &= (1-\gamma)\gamma \cdot gg + (1-\gamma) \cdot gk \end{aligned} \]

\[ \begin{aligned} P(1 \to 0) &= gg \cdot \gamma(1-\gamma) \\ &= (1-\gamma)\gamma \cdot gg \end{aligned} \]

\[ \begin{aligned} P(1 \to 1) &= gg \cdot \gamma \cdot \gamma + gk \cdot \gamma \cdot 1 + kk \cdot 1 \cdot 1 \\ &= \gamma^2 \cdot gg + \gamma \cdot gk + kk \end{aligned} \]

Worked Example

Suppose the true parameters are \(gg = 0.35\), \(gk = 0.30\), \(kk = 0.35\), and \(\gamma = 0.25\).

gg <- 0.35
gk <- 0.30
kk <- 0.35
gamma <- 0.25

p00 <- (1 - gamma)^2 * gg
p01 <- (1 - gamma) * gamma * gg + (1 - gamma) * gk
p10 <- (1 - gamma) * gamma * gg
p11 <- gamma^2 * gg + gamma * gk + kk

cat("Cell probabilities:\n")
#> Cell probabilities:
cat(sprintf("  P(0→0) = %.4f\n", p00))
#>   P(0→0) = 0.1969
cat(sprintf("  P(0→1) = %.4f\n", p01))
#>   P(0→1) = 0.2906
cat(sprintf("  P(1→0) = %.4f\n", p10))
#>   P(1→0) = 0.0656
cat(sprintf("  P(1→1) = %.4f\n", p11))
#>   P(1→1) = 0.4469
cat(sprintf("  Sum    = %.4f\n", p00 + p01 + p10 + p11))
#>   Sum    = 1.0000

Note that the probabilities sum to 1, as they should.

Identification

The model has:

4 observable cell proportions
4 parameters: \(gg\), \(gk\), \(kk\), \(\gamma\)
1 constraint: \(gg + gk + kk = 1\)

This gives 4 observations to estimate 3 free parameters. The model is just-identified: we have exactly enough information to solve for the parameters, but no degrees of freedom for goodness-of-fit tests within a single item.

Implementation Verification

The guess package implements the likelihood function in guess_lik(). Let’s verify it matches our derivation:

guess_lik_manual <- function(gg, gk, kk, gamma, data) {
  vec <- numeric(4)
  vec[1] <- (1 - gamma) * (1 - gamma) * gg # P(0→0)
  vec[2] <- (1 - gamma) * gamma * gg + (1 - gamma) * gk # P(0→1)
  vec[3] <- (1 - gamma) * gamma * gg # P(1→0)
  vec[4] <- gamma * gamma * gg + gamma * gk + kk # P(1→1)

  -sum(data * log(vec))
}

test_data <- c(100, 150, 50, 200)

ll_manual <- guess_lik_manual(0.35, 0.30, 0.35, 0.25, test_data)

cat(sprintf("Manual implementation: %.4f\n", ll_manual))
#> Manual implementation: 645.1621

Parameter Recovery Demonstration

The strongest test of any estimator is whether it can recover known parameters from simulated data. We simulate data with known parameters and check if the fitted model recovers them.

true_params <- c(gg = 0.35, gk = 0.30, kk = 0.35, gamma = 0.25)

sim <- simulate_lca(
  n = 1000,
  n_items = 5,
  gg = true_params["gg"],
  gk = true_params["gk"],
  kk = true_params["kk"],
  gamma = true_params["gamma"],
  seed = 123
)

fit <- item_lca_fit(sim$pre, sim$post)

estimated <- c(
  gg = mean(fit$params["gg", ]),
  gk = mean(fit$params["gk", ]),
  kk = mean(fit$params["kk", ]),
  gamma = mean(fit$params["gamma", ])
)

comparison <- rbind(
  true = true_params,
  estimated = estimated,
  difference = estimated - true_params
)

knitr::kable(comparison,
  digits = 3,
  caption = "Parameter Recovery: True vs. Estimated"
)

Parameter Recovery: True vs. Estimated
	gg	gk	kk	gamma
true	0.350	0.300	0.350	0.250
estimated	0.342	0.302	0.356	0.241
difference	-0.008	0.002	0.006	-0.009

The estimated parameters should be close to the true values. Small differences are expected due to sampling variability.

Monte Carlo Validation

A single simulation might be lucky. To properly validate the estimator, we run many simulations and examine the distribution of estimates.

n_sims <- 100
n <- 500
n_items <- 2
true_params <- c(gg = 0.35, gk = 0.30, kk = 0.35, gamma = 0.25)

set.seed(789)
estimates <- matrix(NA, nrow = n_sims, ncol = 4)
colnames(estimates) <- names(true_params)

for (sim in seq_len(n_sims)) {
  sim_data <- simulate_lca(
    n = n, n_items = n_items,
    gg = true_params["gg"], gk = true_params["gk"],
    kk = true_params["kk"], gamma = true_params["gamma"]
  )

  tryCatch(
    {
      fit <- item_lca_fit(sim_data$pre, sim_data$post)
      estimates[sim, ] <- c(
        mean(fit$params["gg", ]),
        mean(fit$params["gk", ]),
        mean(fit$params["kk", ]),
        mean(fit$params["gamma", ])
      )
    },
    error = function(e) NULL
  )
}

Bias Assessment

An unbiased estimator has \(E[\hat{\theta}] = \theta\). We check this by comparing mean estimates to true values:

mean_estimates <- colMeans(estimates, na.rm = TRUE)
bias <- mean_estimates - true_params
rel_bias <- 100 * bias / true_params

bias_table <- data.frame(
  Parameter = names(true_params),
  True = true_params,
  Mean_Estimate = mean_estimates,
  Bias = bias,
  Relative_Bias_Pct = rel_bias
)

knitr::kable(bias_table,
  digits = 4, row.names = FALSE,
  caption = "Bias Assessment from Monte Carlo Simulation"
)

Bias Assessment from Monte Carlo Simulation
Parameter	True	Mean_Estimate	Bias	Relative_Bias_Pct
gg	0.35	0.3508	0.0008	0.2196
gk	0.30	0.2961	-0.0039	-1.3012
kk	0.35	0.3531	0.0031	0.8957
gamma	0.25	0.2532	0.0032	1.2690

The bias should be close to zero for all parameters.

Standard Error Assessment

The standard deviation of estimates across simulations approximates the true standard error. For well-behaved estimators, this should decrease with \(\sqrt{n}\):

se_estimates <- apply(estimates, 2, sd, na.rm = TRUE)
rmse <- sqrt(colMeans(
  (estimates - matrix(true_params,
    nrow = n_sims,
    ncol = 4, byrow = TRUE
  ))^2,
  na.rm = TRUE
))

se_table <- data.frame(
  Parameter = names(true_params),
  SE = se_estimates,
  RMSE = rmse
)

knitr::kable(se_table,
  digits = 4, row.names = FALSE,
  caption = "Standard Errors from Monte Carlo Simulation"
)

Standard Errors from Monte Carlo Simulation
Parameter	SE	RMSE
gg	0.0236	0.0235
gk	0.0289	0.0290
kk	0.0290	0.0291
gamma	0.0259	0.0260

Coverage Assessment

For valid confidence intervals, 95% CIs should contain the true parameter 95% of the time:

coverage <- numeric(4)
for (j in 1:4) {
  ci_lower <- estimates[, j] - 1.96 * se_estimates[j]
  ci_upper <- estimates[, j] + 1.96 * se_estimates[j]
  coverage[j] <- mean(true_params[j] >= ci_lower &
    true_params[j] <= ci_upper, na.rm = TRUE)
}

coverage_table <- data.frame(
  Parameter = names(true_params),
  Coverage_95 = coverage
)

knitr::kable(coverage_table,
  digits = 3, row.names = FALSE,
  caption = "95% CI Coverage from Monte Carlo Simulation"
)

95% CI Coverage from Monte Carlo Simulation
Parameter	Coverage_95
gg	0.98
gk	0.94
kk	0.93
gamma	0.95

Coverage should be approximately 0.95 for all parameters.

Visualization

hist(estimates[, "gk"],
  breaks = 20,
  main = "Distribution of Learning (gk) Estimates",
  xlab = "Estimated gk", col = "lightblue", border = "white"
)
abline(v = true_params["gk"], col = "red", lwd = 2, lty = 2)
legend("topright", legend = c("True value"), col = "red", lty = 2, lwd = 2)

Distribution of gk (learning) estimates across simulations

Sample Size Effects

The precision of estimates should improve with sample size, following the \(1/\sqrt{n}\) rule:

sample_sizes <- c(100, 250, 500, 1000)
true_params <- c(gg = 0.35, gk = 0.30, kk = 0.35, gamma = 0.25)
n_sims_quick <- 50

set.seed(456)
rmse_by_n <- matrix(NA, nrow = length(sample_sizes), ncol = 4)
colnames(rmse_by_n) <- names(true_params)

for (s in seq_along(sample_sizes)) {
  n <- sample_sizes[s]
  estimates_n <- matrix(NA, nrow = n_sims_quick, ncol = 4)

  for (sim in seq_len(n_sims_quick)) {
    sim_data <- simulate_lca(
      n = n, n_items = 2,
      gg = true_params["gg"], gk = true_params["gk"],
      kk = true_params["kk"], gamma = true_params["gamma"]
    )

    tryCatch(
      {
        fit <- item_lca_fit(sim_data$pre, sim_data$post)
        estimates_n[sim, ] <- c(
          mean(fit$params["gg", ]),
          mean(fit$params["gk", ]),
          mean(fit$params["kk", ]),
          mean(fit$params["gamma", ])
        )
      },
      error = function(e) NULL
    )
  }

  rmse_by_n[s, ] <- sqrt(colMeans(
    (estimates_n - matrix(true_params,
      nrow = n_sims_quick,
      ncol = 4,
      byrow = TRUE
    ))^2,
    na.rm = TRUE
  ))
}

sample_size_table <- data.frame(
  n = sample_sizes,
  RMSE_gk = rmse_by_n[, "gk"],
  RMSE_ratio = c(NA, rmse_by_n[-nrow(rmse_by_n), "gk"] / rmse_by_n[-1, "gk"])
)

knitr::kable(sample_size_table,
  digits = 3, row.names = FALSE,
  caption = "RMSE of gk by Sample Size (ratio should be ~sqrt(2) for doubling n)"
)

RMSE of gk by Sample Size (ratio should be ~sqrt(2) for doubling n)
n	RMSE_gk	RMSE_ratio
100	0.058	NA
250	0.034	1.680
500	0.025	1.375
1000	0.017	1.499

The RMSE ratio between adjacent sample sizes should be approximately \(\sqrt{2} \approx 1.41\) when sample size doubles, confirming the \(1/\sqrt{n}\) efficiency pattern.

DK Model Extension

When “Don’t Know” responses are available, the model extends to a 9-cell transition matrix with 7 latent classes:

Class	From	To	Interpretation
gg	guess	guess	Stable ignorance
gk	guess	know	Learned
gd	guess	DK	Became aware of ignorance
kk	know	know	Stable knowledge
dg	DK	guess	Stopped confessing, started guessing
dk	DK	know	Learned
dd	DK	DK	Persistent uncertainty

Two of the nine conceivable transitions are absent. The model is identified by the assumption that people do not lose knowledge over a short informative process, which sets know→guess and know→DK to zero. Without that restriction there would be 9 parameters against 8 free cell probabilities, and no dataset of any size could separate them.

The learning estimate in the DK model is \(gk + dk\): those who learned the item from guessing, plus those who learned it from confessed ignorance.

sim_dk <- simulate_lca_dk(
  n = 500, n_items = 1,
  gg = 0.25, gk = 0.15, gd = 0.10,
  kk = 0.20, dg = 0.10, dk = 0.10,
  dd = 0.10, gamma = 0.25,
  seed = 456
)

fit_dk <- item_lca_fit(sim_dk$pre, sim_dk$post)

cat("True learning (gk): 0.15\n")
#> True learning (gk): 0.15
cat(sprintf("Estimated gk: %.3f\n", fit_dk$params["gk", 1]))
#> Estimated gk: 0.132
cat(sprintf("\nTrue total learning (gk + kd): %.2f\n", 0.15 + 0.10))
#> 
#> True total learning (gk + kd): 0.25
cat(sprintf("Estimated total: %.3f\n", fit_dk$learning[1]))
#> Estimated total: 0.239

Using validate_recovery()

The package provides a convenience function for Monte Carlo validation:

results <- validate_recovery(
  true_params = c(gg = 0.35, gk = 0.30, kk = 0.35, gamma = 0.25),
  n = 500,
  n_items = 2,
  n_sims = 100,
  seed = 789
)

print(results)

Individual-Level Learning Recovery

Beyond aggregate parameter recovery, we can assess how well the LCA model recovers which specific individuals learned. This is important because the posterior P(learned | data) provides individual-level diagnostic information.

The Key Insight

The LCA model leverages the joint transition structure across all items:

A person with (pre=0, post=1) on one item could be: truly learned OR lucky guess
The LCA uses ALL items’ transition patterns to separate these cases
Cross-sectional methods treat each timepoint independently, losing this information

Computing Posterior Class Probabilities

For each individual with response vector Y = {(y_pre_j, y_post_j)}, Bayes’ rule gives:

\[P(\text{class} = gk \mid Y) \propto P(\text{class} = gk) \times \prod_j P(y_{\text{pre},j}, y_{\text{post},j} \mid \text{class} = gk, \gamma_j)\]

The package provides posterior_class_probs() to compute these:

sim <- simulate_lca(
  n = 500, n_items = 5, gk = 0.30, gamma = 0.25,
  seed = 123, return_classes = TRUE
)

fit <- person_item_lca_fit(sim$pre, sim$post)

posteriors <- posterior_class_probs(fit)
head(posteriors)
#>           P_gg         P_gk      P_kk
#> 1 1.000000e+00 0.0000000000 0.0000000
#> 2 1.172450e-03 0.9988275503 0.0000000
#> 3 8.831195e-07 0.0007523428 0.9992468
#> 4 1.172450e-03 0.9988275503 0.0000000
#> 5 1.172450e-03 0.9988275503 0.0000000
#> 6 1.000000e+00 0.0000000000 0.0000000

The key estimand is P(gk | data) = P(learned | data):

p_learned_lca <- posterior_learned(fit)

cor_with_truth <- cor(p_learned_lca, as.numeric(sim$learned))
cat(sprintf("LCA correlation with true learning: %.3f\n", cor_with_truth))
#> LCA correlation with true learning: 1.000

Comparison with a Cross-Sectional Score

The cross-sectional baseline transforms proportion correct at each timepoint and then computes a bounded difference score. It is not a fitted IRT model:

p_learned_cs <- cross_sectional_learning_score(sim$pre, sim$post)

cor_cs <- cor(p_learned_cs, as.numeric(sim$learned))
cat(sprintf("Cross-sectional correlation: %.3f\n", cor_cs))
#> Cross-sectional correlation: 0.758

cat(sprintf("\nLCA advantage: %.3f\n", cor_with_truth - cor_cs))
#> 
#> LCA advantage: 0.242

Monte Carlo Comparison

We can systematically compare the two approaches:

comparison <- compare_learning_recovery(
  n = 500, n_items = 5, gk = 0.30, gamma = 0.25,
  n_sims = 50, seed = 456
)

summary_stats <- summarize_learning_comparison(comparison)
knitr::kable(summary_stats,
  digits = 3,
  caption = "LCA vs. Cross-Sectional Learning Recovery"
)

LCA vs. Cross-Sectional Learning Recovery
metric	mean	sd
cor_lca	0.999	0.002
cor_cs	0.756	0.018
lca_advantage	0.243	0.018

Effect of Gamma (Guessing Rate)

As gamma rises, response patterns contain less information about the latent class. The simulation compares how the two recovery measures degrade:

set.seed(789)

gammas <- c(0.15, 0.25, 0.35)
results_by_gamma <- data.frame()

for (g in gammas) {
  res <- compare_learning_recovery(
    n = 500, n_items = 5, gk = 0.30, gamma = g,
    n_sims = 30, seed = NULL
  )
  res$gamma <- g
  results_by_gamma <- rbind(results_by_gamma, res)
}

agg <- aggregate(lca_advantage ~ gamma,
  data = results_by_gamma,
  FUN = function(x) c(mean = mean(x), se = sd(x) / sqrt(length(x)))
)
agg <- do.call(data.frame, agg)
names(agg) <- c("gamma", "mean", "se")

barplot(agg$mean,
  names.arg = agg$gamma,
  main = "LCA Advantage by Guessing Rate",
  xlab = "Gamma (guessing probability)",
  ylab = "Correlation advantage (LCA - CS)",
  col = "steelblue"
)

LCA advantage by gamma level

Interpreting Posterior Probabilities

The posterior probabilities provide actionable individual-level information:

sim <- simulate_lca(
  n = 500, n_items = 5, gk = 0.30, gamma = 0.25,
  seed = 101, return_classes = TRUE
)
fit <- person_item_lca_fit(sim$pre, sim$post)
posteriors <- posterior_class_probs(fit)

posteriors$true_class <- sim$true_class

kk_subset <- posteriors[posteriors$true_class == "kk", ]
gk_subset <- posteriors[posteriors$true_class == "gk", ]
gg_subset <- posteriors[posteriors$true_class == "gg", ]

cat("Mean posteriors by true class:\n")
#> Mean posteriors by true class:
cat(sprintf(
  "  kk class: P_kk=%.3f, P_gk=%.3f, P_gg=%.3f\n",
  mean(kk_subset$P_kk), mean(kk_subset$P_gk), mean(kk_subset$P_gg)
))
#>   kk class: P_kk=0.999, P_gk=0.001, P_gg=0.000
cat(sprintf(
  "  gk class: P_kk=%.3f, P_gk=%.3f, P_gg=%.3f\n",
  mean(gk_subset$P_kk), mean(gk_subset$P_gk), mean(gk_subset$P_gg)
))
#>   gk class: P_kk=0.000, P_gk=0.999, P_gg=0.001
cat(sprintf(
  "  gg class: P_kk=%.3f, P_gk=%.3f, P_gg=%.3f\n",
  mean(gg_subset$P_kk), mean(gg_subset$P_gk), mean(gg_subset$P_gg)
))
#>   gg class: P_kk=0.000, P_gk=0.011, P_gg=0.989

Conclusion

This vignette demonstrates that the guess package implementation: 1. Correctly implements the cell probability formulas from the latent class model 2. Produces unbiased parameter estimates 3. Has valid standard errors with proper coverage 4. Shows the expected \(1/\sqrt{n}\) efficiency gains

The latent class model provides a principled way to adjust estimates of learning for guessing bias, recovering the true proportion who learned from pre-post test data.

References

Cor, K. and Sood, G. (2018). Measuring Learning. Working paper. https://gsood.com/research/papers/k_gains.pdf

Cor, K. and Sood, G. (2016). Adjusting for Guessing. Working paper. https://gsood.com/research/papers/guess.pdf

Session Info

sessionInfo()
#> R version 4.6.0 (2026-04-24)
#> Platform: aarch64-apple-darwin23
#> Running under: macOS Tahoe 26.5.2
#> 
#> Matrix products: default
#> BLAS:   /Library/Frameworks/R.framework/Versions/4.6/Resources/lib/libRblas.0.dylib 
#> LAPACK: /Library/Frameworks/R.framework/Versions/4.6/Resources/lib/libRlapack.dylib;  LAPACK version 3.12.1
#> 
#> locale:
#> [1] C/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
#> 
#> time zone: America/Los_Angeles
#> tzcode source: internal
#> 
#> attached base packages:
#> [1] stats     graphics  grDevices utils     datasets  methods   base     
#> 
#> other attached packages:
#> [1] guess_0.7.0
#> 
#> loaded via a namespace (and not attached):
#>  [1] cli_3.6.6           knitr_1.51          rlang_1.3.0        
#>  [4] xfun_0.59           otel_0.2.0          jsonlite_2.0.0     
#>  [7] backports_1.5.1     future.apply_1.20.2 listenv_1.0.0      
#> [10] htmltools_0.5.9     sass_0.4.10         rmarkdown_2.31     
#> [13] evaluate_1.0.5      jquerylib_0.1.4     fastmap_1.2.0      
#> [16] numDeriv_2016.8-1.1 yaml_2.3.12         lifecycle_1.0.5    
#> [19] compiler_4.6.0      codetools_0.2-20    Rsolnp_2.0.1       
#> [22] Rcpp_1.1.2          future_1.70.0       truncnorm_1.0-9    
#> [25] digest_0.6.39       R6_2.6.1            parallelly_1.48.0  
#> [28] parallel_4.6.0      checkmate_2.3.4     bslib_0.11.0       
#> [31] tools_4.6.0         globals_0.19.1      cachem_1.1.0