[R-meta] publication bias

[Teresa Luther]

Hello everyone,

I am wondering whether evaluation of publication bias through visual funnel plot inspection and performing Egger’s regression test is meaningful with less than 10 studies (Suggested cutoff by Sterne et al. ( 2011).
To give a little bit more context: I have run a total of 5 meta-analyses depending on the outcome of a metabolit (either measured throughout the day: analyses on morning concentration, afternoon concentration and evening concentration, or measured after stress induction: analyses on physical stress, social stress and metabolic stress). Some studies provided values for more than one outcome (e.g. metabolit concentration in the morning and after physical stress induction, or metabolit concentration after a physical and after a social stressor) and were included in more than one analysis.
The 3 analyses on concentration throughout the day include 56, 19 and 11 studies, respectively. The two analyses on stress include 8, 4 and 8 studies, respectively.
My thesis advisor suggested to construct funnel plots and perform Egger’s test for each individual study, but I have quite frequently read that assessment of publication bias is not advisable with less than 10 studies in an analysis. So, when I would apply this rule of thumb, I would assess publication bias only in the 3 analyses on metabolit concentration throughout the day. In some publications I have seen that they assessed potential of publication with less than 10 studies (either by setting another cutoff of for example 5 studies or by stating something like “the results have to be interpreted with caution as only few studies were included”- however in my opinion such a note seems a little weird in my case as I would show the reader 3 funnel plots and then question the results and interpretability).

I would be very interested in you opinion. 

Thank you very much in advance and best wishes,
Teresa