---
title: "Summarizing transparency across a corpus"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Summarizing transparency across a corpus}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(collapse = TRUE, comment = "#>")
has_ggplot <- requireNamespace("ggplot2", quietly = TRUE)
```

The detector functions (`rt_all_pmc()`, `rt_data_code_pmc()`) describe **one
article at a time**. Most studies of research transparency instead ask
corpus-level questions: across thousands of articles, how often is each practice
present? Is it improving over time? Does it differ by journal or article type?

This vignette shows how to go from per-article detector output to that kind of
summary, using `rt_summary()`, `rt_score()` and `rt_plot()`.

## From one article to many

Running a detector on a single article returns a one-row table of indicators:

```{r}
library(rtransparency)

xml <- system.file(
  "extdata", "PMID32171256-PMC7071725.xml", package = "rtransparency"
)
one <- rt_all_pmc(xml)
one[, c("pmid", "is_coi_pred", "is_fund_pred", "is_register_pred")]
```

To study a corpus you run a detector over many files and stack the rows;
`purrr::map_dfr(files, rt_all_pmc)` returns all ten
indicators per article in one pass. The result is one row per article with the
indicator columns `is_coi_pred`, `is_fund_pred`, `is_register_pred`,
`is_open_data`, `is_open_code`, `is_novelty_pred`, `is_replication_pred`,
`is_ai_pred`, `is_open_access` and `is_reporting_pred`.
`is_ai_pred` is `NA` for articles published before 2023, and `rt_summary()`
drops those `NA`s, so the AI-disclosure prevalence is computed only over the
articles where the indicator applies. `is_open_access` and `is_reporting_pred`
are summarized by `rt_summary()` too, but are not part of the five openness
practices counted by `rt_score()`.

This package ships a small **simulated** table of that shape, `rt_demo`, so the
rest of the vignette runs without downloading anything:

```{r}
data(rt_demo)
head(rt_demo)
```

## Prevalence of each indicator

`rt_summary()` reports, for each indicator, how many articles were assessed, how
many were positive, the apparent prevalence and its 95% confidence interval:

```{r}
s <- rt_summary(rt_demo)
knitr::kable(
  s[, c("label", "n_articles", "n_detected", "percent", "conf_low", "conf_high")],
  digits = 1,
  col.names = c("Indicator", "Assessed", "Detected", "%", "CI low", "CI high")
)
```

### Correcting for detector error

A text-mining detector is not perfect, so the **observed** prevalence is a biased
estimate of the **true** prevalence. `rt_summary()` corrects for this using each
detector's sensitivity and specificity estimates (the Rogan-Gladen estimator).
The correction is on by default and adds `adj_percent`, `adj_low` and
`adj_high`:

```{r}
knitr::kable(
  s[, c("label", "percent", "adj_percent", "adj_low", "adj_high")],
  digits = 1,
  col.names = c("Indicator", "Apparent %", "Corrected %", "CI low", "CI high")
)
```

The accuracy values come from [`rt_accuracy`](../reference/rt_accuracy.html):

```{r}
rt_accuracy
```

AI-use disclosure and open-access licensing have no bundled accuracy estimate,
so their corrected values are `NA`. Each row of `rt_accuracy` records where its
estimate comes from (`source`) and the validation counts behind it (`tp`, `fn`,
`tn`, `fp`).

**The corrected interval carries the validation's uncertainty.** A detector's
sensitivity and specificity are themselves estimates from a finite hand-labeled
sample. By default (`adj_interval = "simulation"`) `rt_summary()` draws the
apparent prevalence, the sensitivity and the specificity from their Jeffreys
posteriors, corrects each draw and reports the percentile interval, so the
corrected interval widens when the validation sample is small. For rare
indicators (registration, code sharing, replication) this matters most: their
correction subtracts the false-positive rate `1 - specificity`, which is only
known to within a percentage point or so. `adj_interval = "fixed"` gives the
narrower interval that treats the accuracy as known.

**Registration is summarized over the articles its detector assesses.** When
the data carry `is_research` (as `rt_all_pmc()` output does), the registration
denominator is restricted to research articles and reviews, the types the
detector assesses; set `register_assessed_only = FALSE` to count every article.

To use your own validation (or the published `oddpub` values for data and
code), pass any table with `variable`, `sensitivity` and `specificity` columns:

```{r}
# Keep only the point estimates: the bundled validation counts would no longer
# match an edited sensitivity (rt_summary() warns when they disagree).
my_acc <- rt_accuracy[, c("variable", "sensitivity", "specificity")]
my_acc$sensitivity[my_acc$variable == "is_open_data"] <- 0.758
rt_summary(rt_demo, indicators = "is_open_data", accuracy = my_acc)[,
  c("label", "percent", "adj_percent")]
```

## How many practices per article

`rt_score()` adds a per-article count of the openness practices met (conflicts
of interest, funding, registration, data and code). Tabulating it shows how many
articles meet zero, one, two ... of the five practices:

```{r}
scored <- rt_score(rt_demo)
knitr::kable(
  as.data.frame(table(`Practices met` = scored$n_indicators)),
  col.names = c("Practices met", "Articles")
)
```

## Subgroups

Pass `by` to summarize within a grouping column, such as article type:

```{r}
by_type <- rt_summary(rt_demo, by = "type", adjust = FALSE)
knitr::kable(
  by_type[by_type$indicator == "is_open_data",
          c("type", "label", "n_articles", "percent")],
  digits = 1,
  col.names = c("Type", "Indicator", "Assessed", "%")
)
```

## Plots

`rt_plot()` returns a `ggplot`, so it composes with the usual ggplot2 layers.
The default is a prevalence bar chart:

```{r, eval = has_ggplot, fig.width = 7, fig.height = 3.5, fig.alt = "Bar chart of the prevalence of each transparency indicator"}
library(ggplot2)
rt_plot(rt_demo) + ggtitle("Transparency indicators in rt_demo")
```

Use `type = "trend"` with a year column to see prevalence over time:

```{r, eval = has_ggplot, fig.width = 7, fig.height = 4, fig.alt = "Line chart of each transparency indicator's prevalence by year"}
rt_plot(rt_demo, type = "trend", year = "year")
```

The AI-disclosure line begins only in 2023, because the indicator is `NA`
before then; the rising data-sharing and AI lines illustrate the kind of trend
these summaries are meant to surface. Restrict a plot to particular indicators
with `indicators =`, for example to follow AI-use disclosure on its own:

```{r, eval = has_ggplot, fig.width = 7, fig.height = 3.5, fig.alt = "Line chart of AI-use disclosure prevalence by year from 2023"}
rt_plot(rt_demo, type = "trend", year = "year", indicators = "is_ai_pred") +
  ggtitle("Disclosure of generative-AI use, 2023 onward")
```

Set `adjusted = TRUE` in either plot to show the error-corrected prevalence
instead of the apparent prevalence.

## Putting it together

A typical analysis is therefore: run a detector over your corpus, stack the
rows, then

```r
results <- purrr::map_dfr(xml_files, rt_all_pmc)
rt_summary(results)                       # prevalence + corrected prevalence
rt_score(results)                         # per-article practice count
rt_plot(results, type = "trend", year = "year")
```

For the per-indicator detection methodology, see
`vignette("rtransparency")`.