[R-meta] Meta-analysis with small number of effects: 3 options

[Michael Dewey]

Dear Zhouhan

I would go for option 2, exclude them. If there are only four of them it 
is unlikely to make a big difference if you include or exclude and it 
would be simpler to exclude them.

Note I did not look at the output as I did not want to potentially make 
a data-driven decision.

Michael


On 27/03/2024 18:30, Zhouhan Jin via R-sig-meta-analysis wrote:
> Hello R meta Community,
> 
> I'm meta-analyzing 55 effects from 27 longitudinal studies. However, there are only 3 studies that collectively offer 4 post-test2 effects.
> 
> Given the unreliable nature of post-test2 results, I'm considering 3 options:
> 
> 1- Include these 4 effects in analyses but exclude them from interpretation (and/or data display).
> 
> 2- Exclude these 4 effects from data and of course analyses altogether.
> 
> 3- Merge post-test1 and post-test2 categories into one category (called post-test) but then methodologically adjust for the fact that the 3 studies with post-test2 effects likely took longer to complete than other studies.
> 
> Which one seems more reasonable? (results below)
> 
> #Option1: Removing 4 delayed effects from data display but not data:
>   time Mean SE Df Lower Upper t p-value Sig.
> 1 baseline -0.191 0.159 18.000 -0.526 0.144 -1.200 0.246
> 2 posttest1 0.719 0.126 18.000 0.455 0.982 5.723 0.000 ***
> 3 posttest2 0.384 0.323 18.000 -0.295 1.063 1.187 0.250
> 
> #Option2:  Removing 4 delayed effects altogether from data:
>   time Mean SE Df Lower Upper t p-value Sig.
> 1 baseline -0.182 0.163 19.000 -0.523 0.160 -1.115 0.279
> 2 posttest1 0.722 0.127 19.000 0.456 0.987 5.681 0.000 ***
> 
> #Option3: After merge:
>   time Mean SE Df Lower Upper t p-value Sig.
> 1 baseline -0.178 0.157 19.000 -0.507 0.151 -1.132 0.272
> 2 posttest 0.690 0.120 19.000 0.438 0.942 5.735 0.000 ***
> 
> Thanks so much and I look forward to hearing from you.
> 
> Best wishes,
> 
> Zhouhan
> 
> 
> 
> 	[[alternative HTML version deleted]]
> 
> _______________________________________________
> R-sig-meta-analysis mailing list @ R-sig-meta-analysis using r-project.org
> To manage your subscription to this mailing list, go to:
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
> 

-- 
Michael