[R-meta] An issue with selmodel( type="step")

[Viechtbauer, Wolfgang (NP)]

I am working on some updates to selmodel() and one of those changes is that the function now continues to run even if an interval contains no p-values. As discussed previously, the corresponding delta estimate will then either try to drift to 0 or to infinity. Results should be treated with caution (as noted in the output from the function).

Best,
Wolfgang

> -----Original Message-----
> From: R-sig-meta-analysis <r-sig-meta-analysis-bounces using r-project.org> On Behalf
> Of Will Hopkins via R-sig-meta-analysis
> Sent: Tuesday, March 26, 2024 20:47
> To: 'R Special Interest Group for Meta-Analysis' <r-sig-meta-analysis using r-
> project.org>
> Cc: Will Hopkins <willthekiwi using gmail.com>
> Subject: [R-meta] An issue with selmodel( type="step")
>
> Wolfgang, according to the documentation "there must be at least one
> observed p-value within each interval to fit this model. If this is not the
> case, an error will be issued." When I tried it with steps=(0.025) for a
> simulated meta-analysis in which all estimates were significant (p<0.05), it
> issued an error ("One or more intervals do not contain any observed
> p-values"), but the analysis nevertheless produced a result. When I ran it
> with 2500 such simulations, it produced point estimates for the fixed
> effects with 2456 simulations, and confidence limits with 2399. See below
> for a typical result. The selection model results show 0.0000 for the
> estimated probability of non-significant p values, as expected, so how is it
> able to make adjustments?
>
> Type="steps" seems to be about as good as type="beta" for bias and coverage
> with this particular set of study characteristics, but the beta type runs so
> slowly that I have only used it for 100 simulations for comparison so far.
> It took more than an hour, and only 51/100 produced confidence limits for
> the fixed effects, so in these respects the steps type is definitely
> better).
>
> Will
>
> Mixed-Effects Model (k = 22; tau^2 estimator: ML)
>
> tau^2 (estimated amount of residual heterogeneity): 0.2541 (SE = 0.2288)
> tau (square root of estimated tau^2 value):         0.5041
>
> Test for Residual Heterogeneity:
> LRT(df = 1) = 4.9738, p-val = 0.0257
>
> Test of Moderators (coefficients 1:2):
> QM(df = 2) = 94.2813, p-val < .0001
>
> Model Results:
>
>                estimate      se    zval    pval    ci.lb   ci.ub
> xxx$SexFemale    2.9331  0.3022  9.7046  <.0001   2.4359  3.4302  ***
> xxx$SexMale      1.1294  0.7376  1.5312  0.1257  -0.0838  2.3425
>
> Test for Selection Model Parameters:
> LRT(df = 1) = 22.8488, p-val < .0001
>
> Selection Model Results:
>
>                      k  estimate   se  zval  pval  ci.lb  ci.ub
> 0     < p <= 0.025  22    1.0000  ---   ---   ---    ---    ---
> 0.025 < p <= 1       0    0.0000   NA    NA    NA     NA     NA