[R-meta] meta-analysis vs. re-analysis

[James Pustejovsky]

Elli,

These are very interesting questions. You will find relevant references using the terms “individual participant data meta-analysis” (e.g. Riley, Lambert, & Abo-Zaid, 2010, https://www.jstor.org/stable/pdf/25674217.pdf) and “integrative data analysis” (Curran & Hussong, 2009,
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2777640/). A recent review is by Debray and colleagues (2015, https://onlinelibrary.wiley.com/doi/abs/10.1002/jrsm.1160).

Regarding whether you need to conduct sensitivity analysis or consider the potential for publication bias, I would argue that these remain important even though you have analyzed the individual-level data. Just having access to the IPD does not alleviate the possibility that the samples you have identified might not represent the full body of relevant research conducted on your topic. One paper that discusses these issues is Ahmed, Sutton, & Riley (2012; https://doi.org/10.1136/bmj.d7762). 

James

> On Sep 7, 2019, at 4:41 PM, Elli J. Theobald <ellij using uw.edu> wrote:
> 
> Greetings!
> 
> I am wondering about the differences (and some citations to support the
> logic) between meta-analyses and re-analyses. We conducted a systematic
> review of the literature, then of the papers we decided to include in our
> study, we contacted the authors and requested their raw data. (I think that
> the reason for this decision is irrelevant to my question but I would be
> happy to explain if that is helpful.)
> 
> With their raw data, we answered the question(s) we were interested in by
> fitting hierarchical Bayesian regression models (controlling for study and
> other clustering/non-independence elements within each study. We were
> interested in student performance within different types of classrooms.)
> 
> My intuition is that:
> 1) This is a re-analysis of the data, not a meta-analysis (because there
> was nothing meta about it!)
> 2) We do not need to or show any of the typical quality
> assurance/sensitivity analyses, like fail safe number, funnel plots, etc. I
> am not even entirely sure how we would do this given that our unit of
> observation is finer-grain than study. (We re-analyzed the student-level
> data the original papers published.) We have included a histogram of the
> means from each study to show that it is roughly normal in shape, to show
> that we don't have crazy sampling.
> 
> Can anyone weigh in? Am I distinguishing re-analysis from meta-analysis
> appropriately (or is the semantics debate unwarranted)? Can you point me to
> a good citation(s) that distinguish re-analyses from meta-analyses? And is
> my intuition correct that we don't need to provide additional sensitivity
> analyses?
> 
> Thanks so much for your help!
> Cheers.
> Elli
> 
> 
> -- 
> <http://www.biology.washington.edu/users/elli-jenkins>Elli J. Theobald, PhD
> Research Scientist
> Biology Education Research Group
> Department of Biology
> University of Washington, Seattle
> https://sites.google.com/site/ellijtheobald
> <https://sites.google.com/site/ellijtheobald/home>
> 
>    [[alternative HTML version deleted]]
> 
> _______________________________________________
> R-sig-meta-analysis mailing list
> R-sig-meta-analysis using r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis

	[[alternative HTML version deleted]]