[R-meta] meta-analysis vs. re-analysis

[Elli J. Theobald]

Hi James -

Thank you for your prompt response! I will check out these citations and
get back with more questions! Thank you.

Cheers.
Elli

On Sat, Sep 7, 2019 at 6:21 PM James Pustejovsky <jepusto using gmail.com> wrote:

> Elli,
>
> These are very interesting questions. You will find relevant references
> using the terms “individual participant data meta-analysis” (e.g. Riley,
> Lambert, & Abo-Zaid, 2010, https://www.jstor.org/stable/pdf/25674217.pdf
> <https://www.jstor.org/stable/pdf/25674217.pdf?casa_token=rRC6UCpidkoAAAAA:Tqo4pIznk-dIPn521YGrvI9Kg3fMo-7eDCbbbMnjMVMxiUsXOrZ6XyY22wRLWoKaY5hONfBWaBGFTark_zXp8EKb6qkTZKU73XQbocMlULNikKA-l8vN>)
> and “integrative data analysis” (Curran & Hussong, 2009,
> https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2777640/). A recent review
> is by Debray and colleagues (2015,
> https://onlinelibrary.wiley.com/doi/abs/10.1002/jrsm.1160).
>
> Regarding whether you need to conduct sensitivity analysis or consider the
> potential for publication bias, I would argue that these remain important
> even though you have analyzed the individual-level data. Just having access
> to the IPD does not alleviate the possibility that the samples you have
> identified might not represent the full body of relevant research conducted
> on your topic. One paper that discusses these issues is Ahmed, Sutton, &
> Riley (2012; https://doi.org/10.1136/bmj.d7762).
>
> James
>
> On Sep 7, 2019, at 4:41 PM, Elli J. Theobald <ellij using uw.edu> wrote:
>
> Greetings!
>
> I am wondering about the differences (and some citations to support the
> logic) between meta-analyses and re-analyses. We conducted a systematic
> review of the literature, then of the papers we decided to include in our
> study, we contacted the authors and requested their raw data. (I think that
> the reason for this decision is irrelevant to my question but I would be
> happy to explain if that is helpful.)
>
> With their raw data, we answered the question(s) we were interested in by
> fitting hierarchical Bayesian regression models (controlling for study and
> other clustering/non-independence elements within each study. We were
> interested in student performance within different types of classrooms.)
>
> My intuition is that:
> 1) This is a re-analysis of the data, not a meta-analysis (because there
> was nothing meta about it!)
> 2) We do not need to or show any of the typical quality
> assurance/sensitivity analyses, like fail safe number, funnel plots, etc. I
> am not even entirely sure how we would do this given that our unit of
> observation is finer-grain than study. (We re-analyzed the student-level
> data the original papers published.) We have included a histogram of the
> means from each study to show that it is roughly normal in shape, to show
> that we don't have crazy sampling.
>
> Can anyone weigh in? Am I distinguishing re-analysis from meta-analysis
> appropriately (or is the semantics debate unwarranted)? Can you point me to
> a good citation(s) that distinguish re-analyses from meta-analyses? And is
> my intuition correct that we don't need to provide additional sensitivity
> analyses?
>
> Thanks so much for your help!
> Cheers.
> Elli
>
>
> --
> <http://www.biology.washington.edu/users/elli-jenkins>Elli J. Theobald,
> PhD
> Research Scientist
> Biology Education Research Group
> Department of Biology
> University of Washington, Seattle
> https://sites.google.com/site/ellijtheobald
> <https://sites.google.com/site/ellijtheobald/home>
>
>    [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-meta-analysis mailing list
> R-sig-meta-analysis using r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>
>

-- 
<http://www.biology.washington.edu/users/elli-jenkins>Elli J. Theobald, PhD
Research Scientist
Biology Education Research Group
Department of Biology
University of Washington, Seattle
https://sites.google.com/site/ellijtheobald
<https://sites.google.com/site/ellijtheobald/home>

	[[alternative HTML version deleted]]