[R-meta] Does trim and fill method correct for data falsification or lower quality of small studies?

[towhidi]

Dear all,

The asymmetry in a funnel plot can be caused by factors other than 
publication bias, such as data falsification or poorer quality in 
smaller trials. However, the Cochrane Handbook mentions that "the trim 
and fill method does not take into account reasons for funnel plot 
asymmetry other than publication bias".

I do not understand why it cannot account for data falsification or poor 
quality of small trials, assuming that these characteristics are 
associated with study size. For data falsification, the true observed 
effect size (before the fraudulent change in the data) for these studies 
converges on the true underlying effect size. But the falsified data 
move these data points to the right side, and, using the trim and fill 
method, this bias is neutralized by imputing their counterparts on the 
other side. Of course, the confidence intervals will be biased, because 
we are imputing data points that do not exist (which narrows the CI) and 
that the bias arose from data falsification or low quality has added to 
the estimated sampling variance (which widens the CI). Also, it changes 
the weights, especially in the random-effects model.

But, isn't the point estimate a corrected estimate, assuming that data 
falsification has caused the asymmetry?

The same argument may apply to the bias that arises from low-quality 
studies. However, if this is correct, I think that acknowledging this 
and interpreting the CIs with even more caution is more logical than 
assuming that the asymmetry is caused solely by publication bias and 
that misconduct and low quality of small studies have nothing to do with 
it.

Is this correct? Or I am missing something?

Thank you.


-- 
Ali Zia-Tohidi MSc
Clinical Psychology
University of Tehran