[R-meta] Calculation of p values in selmodel

[Will Hopkins]

According to your documentation, Wolfgang, the selection models in selmodel
are based on the p values of the study estimates, but these are computed by
assuming the study estimate divided by its standard error has a normal
distribution, whereas significance in the original studies of mean effects
of continuous variables would have been based on a t distribution. It could
make a difference when sample sizes in the original studies are ~10 or so,
because some originally non-significant effects would be treated as
significant by selmodel. For example, with a sample size of 10, a mean
change has 9 degrees of freedom, so a p value of 0.080 (i.e.,
non-significant, p>0.05) in the original study will be given a p value of
0.049 (i.e., significant, p<0.05) by selmodel. Is this issue likely to make
any real difference to the performance of selmodel with meta-analyses of
realistic small-sample studies? I guess that only a small (negligible?)
proportion of p values will fall between 0.05 and 0.08, in the worst-case
scenario of a true effect close to the critical value and with only 9
degrees of freedom for the SE. If it is an issue, you could include the SE's
degrees of freedom in the rma object that gets passed to selmodel.

 

Will


	[[alternative HTML version deleted]]